AI Data Ops Engineer

Client of Faze 3 Consulting

What you'll own:

• Engineer batch and stream pipelines using Fabric Data Pipelines / Azure Data Factory, Synapse, or Databricks.

• Implement data quality rules, schema validation, de-duplication, SCD, and reconciliation checks.

• Operationalize lineage, cataloging, and classifications with Microsoft Purview; enforce RBAC and access patterns.

• Automate CI/CD via Azure DevOps or GitHub with environment promotion, infrastructure-as-code (Bicep/Terraform), and secrets management via Key Vault.

• Build feature stores and model-serving data contracts in partnership with MLOps and AI engineering teams.

• Own reliability: alerts, runbooks, on-call rotation, and cost and performance optimization.

• Review and finalize vendor-delivered data pipelines and data architecture for AI projects; ensure compliance with client standards for security, performance, and reliability; approve production readiness.

• Collaborate with delivery partner squads on interface specifications, test data, and delivery checkpoints; support SIT/UAT and production cutover.

• Define and enforce Data Contracts and SLAs per priority dataset (schema, refresh frequency, quality thresholds, reconciliation checks, and consumer expectations).

• Own data incident management: classification, RCA, corrective actions, and prevention of recurring nonconformities.

• Formalize the handshake with AI/ML/DevOps engineers on feature and embedding pipelines, monitoring hooks, and release gates for data-dependent AI deployments.

Core skills and tools required:

• Python and PySpark, SQL, Lakehouse patterns, medallion architecture.

• Microsoft Purview: catalog, lineage, classifications; data privacy controls and masking.

• CI/CD with Azure DevOps or GitHub; IaC with Bicep or Terraform; Docker basics.

• Observability: Kusto/KQL, Azure Monitor, Log Analytics; performance tuning and FinOps.

• Production incident response and RCA discipline.