Data Quality Engineer, MPID Team

SHOPEE IP SINGAPORE PRIVATE LIMITED

📍 Singapore On-Site ⏳ 15h ago
AI assistant

Job Description

Job Description:

Design, build, and maintain layered Hive/Spark SQL data models across ODS, DWD, DWS and ADS, including shared dimension tables. Define each table's grain, keys, business-date and partition logic, metric definitions, dependencies, and backfill and release procedures.

Develop and operate production batch ETL pipelines on Spark and Hive, including partner and source onboarding, incremental loads, scheduled runs, and historical backfills. Use staging and checkpoint patterns where needed so retries do not corrupt published data or SCD2 history.

Plan and execute data-store or platform migrations, including Hive-to-ClickHouse migrations where required. Design partition strategies, write migration scripts, reconcile source and target data, and validate query behavior across regions.

Build reusable cross-region models and parameterized pipelines for onboarding markets, partners, and business lines. Standardize schemas and metric logic to reduce