Senior QA Engineer VaporVM

Job Title: Senior QA Engineer (Arabic Speaker)

Location: Riyadh, Saudi Arabia

Role Overview:

We seek a Senior QA Engineer
to lead the design, testing, validation, and automation of QA strategies across
our Data & AI programs. The role spans testing data pipelines, AI/ML
models, Generative AI (LLM) applications, and multi-agent (Agentic AI)
workflows, ensuring solutions meet the highest standards of accuracy,
reliability, fairness, security, and compliance.

The ideal candidate will bring
hands-on expertise in testing LLM-based applications and Agentic AI systems,
along with strong foundations in data quality assurance and AI/ML model
validation.

Key Responsibilities:

Define and enforce QA frameworks,
processes, and standards for data pipelines, AI/ML models, GenAI/LLM
applications, and agent-based AI systems.

Develop and automate data
validation tests (integrity, consistency, lineage, schema drift).

Design and execute ML model
validation tests, including performance, fairness, bias, and reproducibility.

Implement automated prompt-response
validation for GenAI/LLM applications (semantic similarity, factuality,
hallucination detection).

Integrate QA processes into CI/CD,
MLOps, and LLMOps workflows.

Develop monitoring and alerting
systems for data drift, model decay, and pipeline failures.

Define, track, and report on AI
evaluation metrics.

Ensure compliance with AI
governance, ethics, security, and privacy requirements.

Document QA processes, maintain
reusable test cases, and support audits.

Skills & Qualifications:

5+ years of experience in Quality
Assurance, with at least 2+ years focused on Data, AI/ML, GenAI, or Agentic AI
testing.

Proven experience testing LLM/GenAI
applications (prompt testing, RAG pipelines, grounding validation,
hallucination detection).

Experience testing Agentic AI
workflows (multi-agent orchestration, decision-making validation, safety
guardrails).

Strong Python skills and
proficiency with testing frameworks (PyTest, unittest).

Knowledge of ML evaluation metrics
(precision, recall, F1, ROC, fairness, bias testing).

Familiarity with AI governance,
compliance, and privacy testing.

Strong analytical skills, with
attention to detail and quality.

Excellent communication and
collaboration skills for cross-functional work.

Apply Now