Data Engineer
LanceSoft, Inc.
AI assistant
Job Description
- Strong hands-on experience in Python programming, including Pandas, NumPy, and data processing libraries.
- Strong experience with PySpark for distributed data processing and large-scale data transformation.
- Extensive knowledge of SQL, including complex queries, joins, CTEs, window functions, and query optimization.
- Hands-on experience in ETL/ELT development, data integration, data migration, and pipeline orchestration.
- Strong experience with Snowflake, including Snowflake SQL, virtual warehouses, stages, streams, tasks, and Snowpipe.
- Good understanding of relational databases such as SQL Server, Oracle, PostgreSQL, or MySQL.
- Experience in data warehousing concepts, dimensional modeling, fact and dimension tables, and schema design.
- Understanding of batch processing, incremental loads, data validation, and error handling.
- Experience with Git, version control, debugging, and code