Data Engineer

Job description

Role Overview

Seeking a Data Engineer to build and optimize scalable data solutions for Fraud Detection, Reporting, and Machine Learning. This individual contributor role focuses on real-time streaming, large-scale data processing, cloud platforms, and analytics engineering. The ideal candidate has strong expertise in Apache Spark, Kafka, Snowflake, and AWS/Azure.

Key Responsibilities:

• Build and maintain scalable batch and real-time data pipelines for fraud analytics.
• Develop and optimize Spark-based data processing frameworks.
• Design Kafka-based streaming and event-driven architectures.
• Implement efficient data ingestion frameworks and integrate multiple data sources.
• Build and support Snowflake data warehouse and data lake solutions.
• Develop data models for reporting, analytics, ML, and fraud investigations.
• Design scalable cloud solutions using AWS and/or Azure.
• Ensure data quality, governance, reliability, monitoring, and compliance.
• Partner with Data Scientists, Fraud Analysts, and Engineering teams to deliver data-driven solutions.

Required Qualifications:

• 4+ years of experience in Data Engineering or Big Data Engineering.
• 2+ years of experience building large-scale distributed data processing systems.
• Experience supporting production-grade analytics and ML platforms.

Core Skills (Must Have)

• Apache Spark: Large-scale data processing, performance tuning, batch and streaming pipelines.
• Apache Kafka: Event streaming, real-time data processing, and messaging architectures.
• Snowflake: Data warehousing, data modeling, ELT, performance optimization, and security.
• AWS / Azure: Experience with cloud-native data platforms and services such as:
• AWS: S3, Kinesis, DynamoDB, RDS, Glue, EMR, Redshift
• Azure: ADLS, ADF, Event Hubs, Synapse, Databricks, Azure SQL
• Strong programming skills in Python, Scala, or Java.

Preferred: Experience in Fraud Analytics, data lakes, ML pipelines, and modern data platform architectures.