Data Engineer

Data Engineer – AWS & Databricks

Role Overview

The Client Data Engineer will be responsible for the design, development, validation, optimization, and ongoing ownership of data solutions primarily using AWS and Databricks . The role will work closely with internal teams, business SMEs, architects, governance teams, Power BI/analytics teams, and implementation partners.

The ideal candidate should have 8+ years of experience in data engineering, ETL/ELT, and data integration , with strong hands-on expertise in Databricks, AWS, SQL, Python, PySpark, and Spark .

Key Responsibilities

• Design, develop, and support AWS and Databricks data pipelines using SQL, Python, PySpark, and Spark.
• Develop and maintain Bronze, Silver, and Gold data layers using Databricks and Delta Lake.
• Work with source-system data, business rules, mappings, and integration requirements.
• Review and validate data solutions delivered by implementation partners.
• Perform data profiling, reconciliation, testing, and troubleshooting.
• Support data quality, metadata, lineage, and Unity Catalog activities.
• Optimize Databricks pipelines and queries for performance, scalability, and cost efficiency.
• Participate in code reviews, CI/CD, deployment, and production support.
• Maintain technical documentation and support knowledge transfer to internal teams.
• Collaborate closely with BI and analytics teams on downstream reporting and data requirements.
• Support integration with Tableau and Power BI for reporting and analytics use cases.

Required Skills

• 8+ years of experience in data engineering, ETL/ELT, or data integration.
• Strong hands-on experience with Databricks .
• Strong experience with AWS data services and cloud-based data platforms .
• Strong SQL and Python skills.
• Strong experience with PySpark / Apache Spark .
• Experience designing and implementing data pipelines and data transformation frameworks.
• Strong understanding of data integration, transformation, reconciliation, and data quality.
• Strong analytical, troubleshooting, and communication skills.

Preferred Skills

• Databricks, Delta Lake, and Unity Catalog .
• AWS Databricks / Databricks on AWS experience.
• Medallion/Lakehouse architecture.
• Experience with Tableau reporting and Power BI .
• Git, CI/CD, and DevOps practices.
• Experience with data governance, metadata, lineage, and data quality frameworks.
• Databricks certification is a plus.