Senior AI Solution Engineer with MS Foundry, MCP

We are seeking a Senior AI Solution Engineer with expertise in MS Foundry and MCP to build automated data ingestion pipelines and data scraping mechanisms on Azure Databricks that fuel the enterprise AI platform and semantic layer. This role focuses on gathering structured and unstructured data, automating workflows via Azure DevOps CI/CD pipelines, and exposing clean data models formatted via Open Knowledge Format directly to Databricks semantic layers, AI Search, and AI agent runtimes.

Responsibilities

• Build high-throughput data ingestion pipelines and data scraping engines using Python, PySpark, and Databricks Auto Loader to collect unstructured document content, external registry feeds, and enterprise data
• Structure and expose ingested data within the Databricks semantic layer, organize datasets using Open Knowledge Format standards for seamless consumption by AI Search, Databricks Genie Spaces, and LLM reasoning