Data Engineer
EEU Software
AI assistant
Job Description
We are seeking an experienced Data Engineer to join our team.
What you'll need to know and do:
- Develop and maintain a Python/FastAPI microservices system running on Kubernetes, backing a production ETL pipeline (MongoDB → Kafka → Pandas → IBM Cloud Object Storage)
- Own the full lifecycle: API design, schema migrations, and infrastructure debugging under real production pressure.
- API design and implementation (FastAPI, SQLAlchemy/SQLModel, PostgreSQL)
- ETL pipelines using fan-out/fan-in patterns, data validation (Pandera)
- Concurrency control in distributed processing (TTL index locking when scaling multi-pod)
- Infrastructure debugging: Kafka partition imbalance, OOM kills across pods, Helm/K8s deployment failures, Vault auth, proxy/urllib3 routing issues
- Schema migrations and data-conflict resolution (constraint violations, ON CONFLICT patterns)
- MongoDB and