Why This Job is Featured on The SaaS Jobs
This Product Manager role sits at the increasingly important intersection of SaaS platform engineering and production ML. Rather than shipping a single customer-facing feature, the remit is to shape internal “paved road” capabilities for AI observability and evaluation, which are becoming foundational as SaaS products embed LLMs and agentic workflows. The focus on instrumentation defaults, data contracts, and debugging workflows reflects a mature platform approach to operationalizing AI at scale.
For a SaaS career, the work builds durable product instincts around reliability, measurement, and governance, areas that tend to differentiate senior product operators in subscription businesses. Owning success metrics such as coverage, time-to-first-trace, and MTTR creates strong fluency in outcome-driven product management. The cross-functional surface area, spanning engineering leaders plus privacy and security considerations, mirrors the constraints that recur across modern SaaS environments as AI features move into production.
This role is best suited to product managers who enjoy technical stakeholder management and productizing developer workflows, even without writing production code. It fits professionals who prefer evidence-based iteration, clear definitions of “done,” and long-horizon platform roadmapping. Interest in ML development patterns and pragmatic evaluation methods will matter more than domain-specific industry background.
The section above is editorial commentary from The SaaS Jobs, provided to help SaaS professionals understand the role in a broader industry context.
Job Description
We are looking for a Product Manager to join the ML/AI Platform team at Spotify.
The team helps various groups build, deliver, and run ML and AI-enabled experiences at scale. We want a Product Manager to oversee AI Observability and Evaluation. These platform tools assist teams in tracking LLM behavior during production, quickly resolving issues, measuring performance, and consistently refining our AI products.
This is an internal platform role with highly technical customers (ML engineers, backend engineers, data scientists). You will partner closely with engineering leaders to define the paved road for instrumenting, monitoring, and operating AI workloads end-to-end.
\n
What You'll Do- Own the product roadmap for Spotify’s AI observability capabilities across instrumentation, data contracts, and debugging workflows.
- Collaborate with stakeholders to develop the roadmap for Spotify’s AI/Agentic evaluation capabilities via LLM-as-judges and other approaches.
- Build golden path instrumentation defaults (SDKs/libraries/templates) that make LLM workloads observable by default.
- Partner with engineering to deliver root-cause workflows across LLM chains/agents (what changed, where time/cost is spent, which step fails, which prompt/template/model version regressed).
- Drive adoption with internal teams through discovery, pilots, documentation, and ongoing enablement.
- Define and track success metrics (coverage, time-to-first-trace, regressions caught pre-prod, MTTR, cost anomalies detected early).
- Partner with Privacy/Security to establish safe logging and retention guidance that supports both compliance and debuggability.
Who You Are- 2+ years of experience as a Product Manager, Technical Product Manager, or equivalent product ownership role.
- Familiar with ML and AI development flows, and comfortable reasoning about LLM application patterns (prompting, RAG/agents, evaluation basics).
- Aware of LLM evaluation approaches (golden datasets, regression testing, human-in-the-loop review, LLM-as-a-judge).
- Comfortable partnering with engineers on technical tradeoffs and system design, even if you are not writing production code.
- Strong product fundamentals: discovery, prioritization, roadmap communication, and stakeholder alignment.
- Metrics-oriented and pragmatic; you can define outcomes and iterate based on evidence.
Where You'll Be- This role is based in Toronto.
- We offer you the flexibility to work where you work best! There will be some in person meetings, but still allows for flexibility to work from home.
\n
The Canadian base range for this position is 98,155 - 140,222 CAD plus equity.
Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.
At Spotify, we are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone. We have ways to request reasonable accommodations during the interview process and help assist in what you need. If you need accommodations at any stage of the application or interview process, please let us know - we’re here to support you in any way we can.
Spotify transformed music listening forever when we launched in 2008. Our mission is to unlock the potential of human creativity by giving a million creative artists the opportunity to live off their art and billions of fans the chance to enjoy and be passionate about these creators. Everything we do is driven by our love for music and podcasting. Today, we are the world’s most popular audio streaming subscription service.
