Job Description

Responsibilities:

Build scalable ETL/ELT solutions for high-volume structured and semi-structured data, including transformation, cleansing, validation, reconciliation and data-quality controls.
Design, develop, test and optimize data pipelines using PySpark, Python and advanced SQL.
Develop and maintain cloud data warehouse and lakehouse solutions, including staging, curated, fact, dimension and reporting layers.
Work with Microsoft Fabric Lakehouse, Warehouse, OneLake, Data Factory pipelines and Notebooks, with Azure Data Factory, Azure Data Lake Storage and Power BI integration where required.
Apply Spark and SQL optimization techniques and troubleshoot job failures, data issues and performance bottlenecks.
Follow coding, testing, source-control and code-review standards; use Git and Azure DevOps and contribute to CI/CD, SIT, UAT, releases and production deployment.