Why This Job is Featured on The SaaS Jobs
This AI Engineer role stands out in the SaaS landscape because it sits inside an enterprise-facing, platform-style product where conversational AI is delivered as a repeatable service, not a one-off model deployment. The emphasis on automated evaluation, safety, and reliability reflects a maturing segment of SaaS: AI features being operationalised into governed workflows that can support large, high-stakes customer experience programs.
From a career standpoint, the work maps closely to how modern SaaS companies productise machine learning. Building test pipelines, defining quality metrics, and integrating checks into CI/CD are durable skills across subscription software, where releases are continuous and regressions can impact retention. Experience monitoring drift, bias, and hallucinations also translates to broader AI governance and MLOps practices that many SaaS organisations are now formalising.
The position is best suited to engineers who enjoy the intersection of software engineering discipline and applied ML, especially those who prefer measurable outcomes over research novelty. It will fit someone who likes turning ambiguous “model quality” questions into repeatable tests, dashboards, and feedback loops, and who is interested in how LLM capabilities become dependable product behaviour.
The section above is editorial commentary from The SaaS Jobs, provided to help SaaS professionals understand the role in a broader industry context.
Job Description
About the Company:
Netomi is the leading agentic AI platform for enterprise customer experience. We work with the largest global brands like Delta Airlines, MetLife, MGM, United, and others to enable agentic automation at scale across the entire customer journey. Our no-code platform delivers the fastest time to market, lowest total cost of ownership, and simple, scalable management of AI agents for any CX use case. Backed by WndrCo, Y Combinator, and Index Ventures, we help enterprises drive efficiency, lower costs, and deliver higher quality customer experiences.
Want to be part of the AI revolution and transform how the world’s largest global brands do business? Join us!
About the Role
A Senior AI Engineer focuses on building automated systems to evaluate, test, and enhance the performance of conversational AI models (LLMs, chatbots, voice assistants). They sit at the intersection of machine learning, software engineering, and QA automation, ensuring that AI systems produce high-quality, safe, and reliable responses at scale.
\n
Responsibilities- Build Python-based pipelines for automated quality testing of AI responses.
- Integrate LLMs into automated evaluation frameworks (e.g., using GPT-based evaluators, embeddings, or custom scoring).
- Automate regression and stress testing for conversational AI flows.
- Define evaluation metrics (relevance, factuality, coherence, safety, empathy).
- Implement both rule-based and AI-driven quality checks.
- Monitor model drift, bias, and hallucinations using automated workflows.
- Work with APIs, SDKs, and CI/CD pipelines to embed automated AI evaluation in production.
- Develop monitoring dashboards to visualize conversation quality.
- Collaborate with ML engineers, product managers, and QA teams to close the feedback loop.
- Experiment with prompt engineering and automated prompt-testing frameworks.
- Explore reinforcement learning, self-critique models, or human-in-the-loop automation for continuous improvement.
- Automate compliance and policy adherence checks for enterprise AI systems.
Requirements- Strong 4-5 Years of experience in Python development (automation, scripting, data handling).
- Experience with LLMs/NLP frameworks.
- Understanding of MLOps / AI deployment pipelines.
\n
Netomi is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.
