Why This Job is Featured on The SaaS Jobs
This Senior Software Developer role sits at the intersection of SaaS reliability and modern AI delivery, focusing on the platform layer that turns models, including LLMs, into dependable production services. The listing signals a mature, high-throughput environment, with infrastructure supporting large volumes of queries and a broad internal user base across applied science and engineering functions.
For a SaaS career, the standout value is in building durable platform primitives that other product teams rely on: observability, ease of use, and operational correctness under sustained load. Experience gained here translates well to other SaaS companies investing in AI features, because it emphasizes repeatable deployment patterns, scalable distributed design, and the practical constraints of serving real-time inference rather than experimentation. The mention of agentic workflows also points to emerging platform concerns that many SaaS organizations are only beginning to formalize.
This role tends to suit engineers who prefer backend depth over feature-surface work, and who enjoy debugging production systems with measurable performance and reliability targets. It is a strong match for someone comfortable collaborating across disciplines and iterating on platform ergonomics as internal adoption grows.
The section above is editorial commentary from The SaaS Jobs, provided to help SaaS professionals understand the role in a broader industry context.
Job Description
Build the backbone of AI at scale
Want to see your work power millions of machine learning predictions every day? Join our Machine Learning Platform team and help bring models, including Large Language Models (LLMs), into production in a reliable and scalable way.
You’ll contribute to a platform that trains thousands of models and serves hundreds of millions of queries daily, while growing your expertise in distributed systems, machine learning infrastructure and agentic workflows.
As a Backend Developer, you will:
Contribute to the development lifecycle, including coding, testing, and deployment
Analyze and help improve the ease of use, observability and reliability of our AI platform
Participate in maintaining services handling millions of requests and LLM calls per day
Collaborate with applied scientists, data engineers, and developers to integrate agentic applications into production
Contribute to evolving our AI platform to support the growth and scale of agentic workflows for Coveo users
Suggest improvements to existing processes and use AI coding tools to support your development workflow
Here is what will qualify you for the role:
5+ years of experience in backend development in a cloud environment (Python or Java/Spring preferred, AWS an asset)
Good understanding of distributed systems and scalable application design
Ability to solve problems and debug systems with guidance when needed
Comfortable learning new tools, technologies, and best practices
What will make you stand out:
Familiarity with Terraform, AWS Bedrock, OpenSearch serverless or other vector databases, LiteLLM or other LLM inference gateways
Exposure to machine learning workflows or agentic workflows
Interest in scalable backend services, large-scale realtime systems, agentic workflows and applications or realtime inference at scale
You don’t need to check every single box; passion goes a long way and we appreciate that skillsets are transferable.
Send us your application, we want to hear from you!
Join the Coveolife!
We encourage all qualified candidates to apply regardless of, for example, age, gender, disability, gaps in CV, national or ethnic background.
This job description was written by humans, assisted by AI. We may leverage technology in our hiring process to help us see the person behind the resume.
Coveo is committed to providing accessible employment practices. If you require accommodation due to a disability at any point during the recruitment process, please contact HR@Coveo.com to discuss your needs.
