Product-Aligned Agentic Quality Engineering Lead (Automation & Agentic AI)

We are hiring a senior Quality Engineering leader who owns end-to-end quality, release sign-off, and automation strategy across our platform — including AI agents, web applications, APIs, and databases.

This role blends:

• Quality Ownership
• Automation & Agentic Testing Leadership
• Release & Go-Live Authority
• Product-aligned execution (PO-style ownership)

You will start by establishing strong manual and exploratory testing foundations, then systematically drive automation-first quality, including AI agent validation, critic agents, eval frameworks, and safety guardrails.

This is not a passive QA role.

You are accountable for production quality.

Key Responsibilities

End-to-End Quality Ownership & Release Sign-Off

• Own quality outcomes across the full delivery lifecycle
• Act as release authority, driving go/no-go decisions
• Define and enforce quality gates across features, agents, and releases
• Partner with Product and Engineering to balance speed and quality
• Reduce production defects through proactive automation and validation

Automation-First Quality Strategy

• Define and own the automation roadmap (UI, API, DB, agentic systems)
• Build and maintain hybrid automation frameworks using Python
• Drive CI/CD-integrated automated testing pipelines
• Ensure automation coverage continuously replaces repetitive manual testing
• Guide aligned Quality Engineers in automation implementation

Agentic AI & LLM Quality Engineering

• Own quality for AI agents, tool calls, and multi-step workflows
• Validate:
• Agent behavior & reasoning paths
• Tool invocation correctness
• Hallucinations & unsafe outputs
• RAG accuracy and grounding
• Define and execute agent evaluation (evals) frameworks
• Collaborate with Critic Engineers to design:
• Critic agents
• Automated agent validation
• Benchmark datasets & scoring rubrics
• Lead innovation in agent testing methodologies

Manual, Exploratory & Scenario-Based Testing

• Drive high-quality manual and exploratory testing where automation is not yet viable
• Design test scenarios covering edge cases, failure modes, and real-world flows
• Ensure strong test data strategies and defect lifecycle management
• Use manual testing strategically — not as a long-term crutch

Product-Aligned Ownership

• Work as a quality-focused Product Owner partner
• Align closely with Product Managers on acceptance criteria and readiness
• Participate in sprint planning, refinement, and release discussions
• Ensure quality is embedded into feature design, not added later

Cross-Team Collaboration

• Work closely with:
• Product
• Engineering
• AgenticOps
• RAG & AI teams
• Act as the single quality voice across delivery discussions
• Mentor and guide aligned Quality Engineers and contributors

Required Skills & Experience

Quality & Manual Testing

• 5+ years of strong hands-on experience in manual and exploratory testing
• Deep understanding of SDLC/STLC, test planning, traceability, and defect management
• Strong API testing experience (Postman, REST clients)

Automation (UI, API, Database)

• Strong experience in:
• UI automation (Playwright / Selenium / Cypress)
• API automation (pytest, requests, REST Assured)
• Database validation using SQL
• Experience designing end-to-end automation frameworks
• CI/CD integration (GitHub Actions, GitLab, Jenkins)

Agentic AI & LLM Testing

• Hands-on experience testing LLMs and AI agents
• Understanding of:
• Agent workflows
• Tool calling
• RAG pipelines
• Guardrails & safety validation
• Experience designing or executing eval frameworks

Python, MCP & Cloud

• Strong Python skills for automation, utilities, and validation
• Understanding of MCP servers, agent tools, and orchestration
• Cloud exposure (GCP / AWS / Azure) in QA or automation contexts

What Success Looks Like

• Releases go out predictably and confidently
• Automation coverage grows sprint after sprint
• Agent failures are caught before production
• Quality becomes a shared responsibility, led by you
• Manual testing effort reduces as automation and agent critics mature

Why This Role Is Special

• Owns release sign-off
• Drives automation-first quality
• Shapes how agentic AI is tested
• Works closely with Product & CTO
• Leads innovation in AI testing
• Not a test executor role
• Not a checkbox QA position