Job Description

We are looking for a Data Engineer to design, build, and maintain scalable data pipelines on AWS and Databricks. You will combine data engineering with DevOps practices (CI/CD, infrastructure as code, monitoring) to deliver reliable, automated, and secure data platforms that support analytics, reporting, and machine learning.

Key Responsibilities

Design and develop batch and streaming data pipelines using Databricks (PySpark, Spark SQL, Delta Lake) on AWS
Build and optimize ETL/ELT workflows to ingest data from databases, APIs, files, and event streams
Implement lakehouse architecture using Delta Lake, Unity Catalog, and the medallion model (bronze/silver/gold)
Manage and orchestrate workflows with Databricks Workflows, Airflow, or AWS Step Functions
Provision and manage cloud infrastructure using Terraform or CloudFormation
Build and maintain CI/CD pipelines for data code and infrastructure (GitHub Actions, GitLab