Quality Assurance Engineer

AI71

Non-Deterministic Testing: Architect automated frameworks to evaluate Generative AI outputs for hallucination, consistency, and factual accuracy against "Gold Standard" datasets.

RAG Evaluation: Implement automated metrics (e.g., RAGAS, faithfulness, answer relevance) to verify that Retrieval-Augmented Generation pipelines accurately cite technical and regulatory documentation.

Prompt Regression: Design regression suites to monitor "prompt drift," ensuring model updates do not degrade the quality of AI-generated engineering documents.

Enterprise Integration: Build robust tests to validate data consistency between AI agents and critical systems (e.g., SAP S/4HANA, Ariba), ensuring the integrity of Bill of Materials (BOM) and financial data.

Performance Benchmarking: Design tests to validate latency and throughput for forecasting models and risk-scoring engines using tools like Locust, JMeter, or K6.

API & Security Validation: Automate testing of secure API gateways, verifying Role-Based Access Control (RBAC) and PII redaction logic before data reaches AI models.

V-Model Alignment: Map automated test cases to "System Requirements" to create digital evidence for formal Verification and Validation (V&V) reports.

Stage Gate Compliance: Prepare "Test Readiness" packages for formal reviews, providing quantitative evidence that systems are stable enough to move from MVP to Production.

Defect Lifecycle Management: Manage the feedback loop between Requirements Quality Assistants and development teams, tracing AI logic defects back to specific model versions.

Core Automation: Expert proficiency in Python (Pytest) and standard libraries (Selenium/Playwright, Requests).

AI Evaluation: Hands-on experience with LLM evaluation frameworks (e.g., DeepEval, TruLens) and "Ground Truth" dataset management.

Performance Engineering: Proficiency in crafting Performance Test Plans and implementations (Locust, K6, etc.).

Data Validation: Expertise in SQL and data quality tools (e.g., Great Expectations) for Data Lakehouses and Vector Databases.

CI/CD & DevOps: Strong experience integrating quality gates into GitLab CI/CD pipelines.

Engineering Practices: Deep understanding of modern QE practices, including Shift Left, Test Pyramid, and Mono-repo architectures.

Experience: 5+ years in QA Automation, with 2+ years focused on complex data-driven applications, ML models, or AI agents.

Domain Expertise: Background in Defense, Aerospace, or highly regulated industries is a strong plus. Familiarity with IV&V processes is highly desirable.

Analytical Mindset: Ability to define pass/fail criteria for probabilistic systems and communicate "Confidence Levels" to engineering leadership.

Final Line of Defense: You define whether an AI agent is "trusted" to negotiate a contract or design a critical system component in a safety-critical environment.

Tax-Free Compensation: A market-leading, 100% tax-free salary and benefits package.

Full Relocation Support: Comprehensive assistance including flights, housing support, and visa sponsorship for you and your family to Abu Dhabi.

Pioneering Work: Set the global standard for how defense organizations validate intelligent systems using world-leading models.