Lead Engineer - Data [T500-10873]
hyderabad
Job Summary:
We are looking for a Lead Data Engineer for the Enterprise Data Organization to design, build and manage data pipelines (Data ingestion, data transformation, data distribution, quality rules, data storage etc.) for Azure cloud-based data platform. The candidate will require to possess strong technical, analytical, programming and critical thinking skills.
Responsibilities:
Work closely with Product owners/managers to understand the business needs
Collaborate with architects, data modelers and other team members to design technical details of data engineering jobs / processes to fulfillment business needs
Lead sessions with data engineers and provide necessary guidance to solve technical challenges
Develop new and / or enhance existing data processing (Data Ingest, Data Transformation, Data Store, Data Management, Data Quality) components
Actively contribute towards setting up and refining data engineering best practices
Help data engineers adhere to coding standards, best practices, etc. and produce production deployable code that is robust, scalable and reusable
Support and troubleshoot the data environments, tune performance, etc.
Document technical artifacts for developed solutions
Good interpersonal skills; comfort and competence in dealing with different teams within the organization. Requires an ability to interface with multiple constituent groups and build sustainable relationships.
Versatile, creative temperament, ability to think out-of-the box while defining sound and practical solutions. Ability to master new skills.
Familiar with Agile practices and methodologies
Knowledge and Abilities:
Advanced SQL queries, scripts, stored procedures, materialized views, and views (5+ yrs experience)
Focus on ELT to load data into database and perform transformations in database (5+ yrs experience)
Ability to use analytical SQL functions
Snowflake experience a plus.
Experience building dimensional Data marts, Data lakes and warehouses. (5+ yrs experience)
Cloud Data Warehouse solutions experience (Snowflake, Azure DW, or Redshift); data modeling, analysis, programming - (3+ years' experience with one or more cloud platforms)
Experience with DevOps models utilizing a CI/CD tool (2+ yrs experience)
Work in hands-on Cloud environment in Azure Cloud Platform (ADLS, Blob)
Talend, Apache Airflow, Azure Data Factory