Senior Data Engineer
User Experience Researchers Pte.Ltd
AI assistant
Job Description
Key Responsibilities:
- Design, develop and implement enterprise-scale data pipelines, data ingestion and integration workflows, data transformation processes and data storage solutions across cloud and enterprise data platforms.
- Design and implement scalable, resilient and secure data architectures, including data lakes, data warehouses, streaming platforms and time-series data platforms.
- Integrate structured and unstructured data from multiple sources, including databases, APIs, files, streaming sources, Internet of Things (IoT) and Operational Technology (OT) systems.
- Develop and maintain data engineering solutions using SQL, Python or equivalent programming languages, ETL frameworks, data orchestration and distributed data processing technologies.
- Implement data modelling, version control, automated testing and Continuous Integration/Continuous Deployment (CI/CD) practices to ensure reliable data pipeline