Your role
As a SDET focused on Agentic AI, you are a strong software engineer with deep test automation skills and a passion for quality in AI-powered systems. You will: design and build scalable Python + Playwright automation for UI, API, and end-to-end flows; test complex agentic workflows, including multi-step reasoning, tool invocation, and orchestration across multiple services; telp define testing strategies for deterministic and non-deterministic systems, and contribute best practices for LLM and Agentic AI quality across the organization; you’ll be hands-on in code, involved in design and architecture discussions, and a key voice for testability, observability, and quality.
What you’ll do
- Design, develop, and maintain scalable automation frameworks using Playwright and Python for UI, API, and end-to-end testing.
- Build automated tests for Agentic AI workflows, including multi-step reasoning, tool invocation, and autonomous decision flows.
- Validate AI-driven systems using deterministic testing approaches, mocks, simulators, and contract testing where applicable.
- Develop test strategies for LLM-based features, including prompt validation, response evaluation, guardrails, and failure modes.
- Implement API, integration, and system-level tests across microservices and AI orchestration layers.
- Collaborate with AI/ML engineers to test model integrations, inference pipelines, and agent execution frameworks.
- Set up and maintain CI/CD pipelines using GitHub Actions, enabling fast feedback and reliable releases.
- Analyze test results, derive quality KPIs, and clearly communicate risks and release readiness.
- Provide detailed root-cause and failure analysis to accelerate defect resolution.
- Participate in design and architecture reviews, advocating for testability, observability, and quality-first design.
- Mentor junior SDETs and QA engineers, raising the bar on automation, AI testing practices, and engineering excellence.
Skills you’ll bring
- 4+ years of experience as an SDET, Software Engineer, or Test Automation Engineer in SaaS environments.
- Strong programming experience in Python (preferred) and familiarity with JavaScript/TypeScript.
- Hands-on experience with Playwright (preferred) or modern UI automation frameworks.
- Solid experience testing API-first and microservices-based architectures.
- Practical understanding of AI/ML or LLM-based systems, including:
- Agentic AI concepts (agents, tools, planners, memory, orchestration).
- Deterministic vs non-deterministic testing strategies.
- Prompt-based systems and response evaluation.
- Experience with mocking, stubbing, and simulation to test complex integrations.
- Strong knowledge of REST APIs, server-side testing, and integration testing.
- Experience working with CI/CD pipelines, preferably GitHub Actions.
- Familiarity with Cloud technologies and containerized environments (Docker).
- Excellent written and verbal communication skills; comfortable working across geographies.
Nice to Have
- Experience testing Agentic AI frameworks (e.g., LangChain, LlamaIndex, or similar concepts).
- Knowledge of AI evaluation techniques (golden datasets, semantic similarity, rule-based validation, replay testing).
- Experience with communication platforms, contact centers, or VoIP technologies.
- Experience with performance, load, and reliability testing for AI-powered systems.
Technologies We Use
- Languages: Python, Java, JavaScript.
- Automation: Playwright, Selenium, TestNG.
- API Testing: REST Assured, Postman.
- CI/CD: GitHub Actions (GHA).
- Dev Tools: Git, GitHub, Jira.
- Infrastructure: Docker, Cloud-based services.
