Lead Data Engineer
EPAM
Job Description
We are building a next-generation analytics platform and need a Lead Data Engineer to shape scalable ingestion, transformation, and governed data products on Databricks. You will combine hands-on delivery with technical leadership and architecture to enable self-service analytics and AI-assisted engineering.
Responsibilities
Design scalable data ingestion and transformation frameworks on Databricks
Define and enforce Databricks best practices for medallion layers and operational excellence
Build batch and streaming pipelines using Apache Spark and PySpark
Deliver end-to-end data products from ingestion through governed consumption layers
Develop trusted analytical datasets and semantic models for reporting and self-service analytics
Implement data quality, lineage, monitoring, and observability standards
Lead proof-of-concepts and evaluate emerging capabilities in the Databricks ecosystem
Mentor engineers and drive reusable frameworks