Snowflake Data Engineer 100% Remote en Derevo

At Derevo we seek to empower companies and people to unlock the value of data in their organizations. We do this through the implementation of analytics processes and platforms with an approach that covers the complete cycle that they need to carry out to achieve it. Derevo started in 2010 with a simple idea, to create more than a company, a community and a space where everyone has the opportunity to build a dream. Do you want to know more about the vacancy?

Snowflake Data Engineer

Summary:

The desired profile should have hands-on experience in designing, establishing, and maintaining data management and storing systems. Skilled in collecting, processing, cleaning, and deploying large datasets, understanding ER data models, and integrating with multiple data sources. Efficient in analyzing, communicating, and proposing different ways of building Data Warehouses, Data Lakes, End-to-End Pipelines, and Big Data solutions to clients, either in batch or streaming strategies.

Technical Proficiencies:

• SQL:

Data Definition Language, Data Manipulation Language, Intermediate/advanced queries for analytical purpose, Subqueries, CTEs, Data types, Joins with business rules applied, Grouping and Aggregates for business metrics, Indexing and optimizing queries for efficient ETL process, Stored Procedures for transforming and preparing data, SSMS, DBeaver

 

• Azure:

Intermediate/Advanced knowledge in

 

Azure Storage Account:

Provision Azure Blob Storage or Azure Data Lake instances

Build efficient file systems for storing data into folders with static or parametrized names, considering possible security rules and risks

Experience identifying use cases for open-source file formats like parquet, AVRO, ORC

Understanding optimized column-oriented file formats vs optimized row-oriented file formats

Implementing security configurations through Access Keys, SAS, AAD, RBAC, ACLs

 

Azure Data Factory:

Provision Azure Data Factory instances

Use Azure IR, Self-Hosted IR, Azure-SSIS to establish connections to distinct data sources

Use of Copy or Polybase activities for loading data

Build efficient and optimized ADF Pipelines using linked services, datasets, parameters, triggers, data movement activities, data transformation activities, control flow activities and mapping data flows

Build Incremental and Re-Processing Loads

Understanding and applying best practices for Source Control with Azure Repos Git integration

 

• Snowflake:

Use of staging, bronze, silver and gold layers. Load structured and semi-structured data with Copy command in Batch, Micro-Batch and Continuous strategies. Internal / External Stages, Storage Integration.

Experience with working and querying parquet files

Optimizing the Data Warehouse: Scaling up, Scaling down, Caching and Clusterization. Monitoring and analyzing query execution plans, query profiles, query statistics and implementing clustering keys to optimize response times and credit consumption through Resource Monitor

Transform data with ANSI SQL. Continuous transformations with Streams and Tasks. Enhanced extensibility with Functions and Stored Procedures

Understanding of star and snowflake schemas, Watermark date columns, Slowly changing dimensions, ACID compliance and big data massively parallel processing architectures

Types of Temporary, Transient and Permanent Tables, Materialized Views. Time Travel, Copy Options

Security Role Access Control, Dynamic Data Masking, Fail Safe

Understanding business and technical requirements to build the best Snowflake architecture solution

Data Sharing

Connecting other tools

Best Practices