Data Engineer

Join Toptal’s Elite Network of Freelance Talent

Toptal is an exclusive network of the world’s top freelance software developers, designers, finance experts, product managers, and project managers. As one of the fastest-growing fully remote networks globally, we empower professionals to thrive in their freelance careers while working with leading companies around the world.

Role: Data Engineer

Responsibilities:

• Collaborate with teams of experienced actuaries, developers and business experts to deliver projects
• Design and propose end-to-end data pipelines (ingestion to storage to consumption) for data projects as well as databases to support web-based applications
• Implementation of data warehouses and data marts to serve data consumers
• Execute implementation of the designs and bring it from design stages to implementation ,followed by operationalization to maintenance
• Database design and modelling. To be able to organize data at both macro and micro level and provide logical data models for the consumers
• Database performance tuning and data lifecycle management
• Assist in support and enhancements of existing data pipelines and databases

Must-Have:

• 5+ years of total experience working with data integration teams.
• 1+ years of in-depth experience developing data pipelines within an Apache Spark environment (preferably Databricks).
• 1+ years of active hands-on work with Databricks, demonstrating in-depth experience.
• Good experience in building data pipelines that ingests various types of data sources into a database or data lake and populating a structured warehouse or data mart that fuels the data consumer.
• Fluent in both complex SQL query performance tuning and database performance tuning
• Good knowledge of data modelling, database technologies, data warehousing.
• Have experience in building data warehouses/data marts
• Understand the importance of performance and able to implement best practices in ensuring performance and maintainability for data-centric projects
• This person is required to be dynamic and be a quick eager learner as knowledge about new technologies will need to be acquired on the spot
• Experience with BI and analytics tools like Power BI or Tableau.

Nice-to-Have:

• Experience integrating AI into data engineering.
• Certifications in Databricks, cloud computing or database management.
• Experience developing data solutions using native IaaS and PaaS solutions on AWS (RDS, S3).
• Experience in any object-oriented programming language.

Additional Details

• Location: Hybrid in Bangalore, India (2-3 days/week in-office)
• Overlap: Singapore timezone
• Type: 40h/week contract
• Duration: 12+ months
• Authorization: Applicants must be authorized to work in their country of residence without employer sponsorship