The Opportunity
We're looking for a talented Scala Spark Application Developer to join our early stage startup team and drive the core development of our platform's dataplane. Unlike typical Scala/Spark roles that focus on services or project-based work, this position is uniquely centered on building a sophisticated product platform. You will design and develop the dataplane of our platform - a cutting-edge Spark-based analytics engine with embedded inductive learning capabilities that forms the foundation of our data quality intelligence.
In this role, you'll architect and implement the mathematical and computational core that powers our anomaly detection algorithms. You'll work with complex statistical models, real-time data processing, and concurrent systems that operate at enterprise scale across multiple cloud environments. Our platform doesn't just process data - it learns from it, and you'll be building the brain that makes that possible.
What You'll Do
Design and implement the core dataplane architecture for our Spark-based analytics engine
Develop sophisticated statistical algorithms and inductive learning models for automated data quality inference
Build high-performance, concurrent data processing pipelines using Scala and Spark that handle enterprise-scale workloads
Implement asynchronous processing patterns and reactive systems for real-time anomaly detection and alerting
Collaborate with our CTO & our customers to invent & embed statistical models directly into the Spark execution engine
Optimize query performance and data processing efficiency across distributed computing clusters
Design and implement advanced SQL generation and optimization for complex analytical workloads
Build robust error handling and fault tolerance mechanisms for mission-critical data processing
Establish best practices for functional programming, concurrent system design, and mathematical computation accuracy
Mentor team members on Scala, functional programming principles, and advanced Spark optimization techniques
Collaborate cross-functionally with product, infrastructure, and customer success teams to deliver a world-class platform
Give & receive constructive feedback that makes the overall team and product stronger
Who You Are
Enjoys working with excellent team members while maintaining high personal standards
Collaborates extremely well with teammates – we want people to have fun working with you
Has desire and ability to work both independently on complex algorithmic problems
Comes with an optimistic attitude towards problem solving and continuous learning
Brings experience and passion for building high-performance, mathematically rigorous systems
Has a deep appreciation for functional programming principles and their application to distributed computing
Takes a thoughtful approach to architectural decisions, balancing performance, maintainability, and mathematical correctness
Has strong analytical thinking skills and enjoys perfecting algorithms that solve real-world data problems
Requirements
Experience: 3+ years of experience with Scala development and distributed systems
Core Technologies: 2+ years of hands-on Apache Spark development experience
Mathematical Foundation: Strong background in statistics, linear algebra, and mathematical analysis
SQL Mastery: Advanced SQL skills with experience in query optimization and performance tuning
Concurrent Programming: Proficiency in asynchronous and concurrent programming patterns in Scala
Functional Programming: Deep understanding of functional programming concepts and their practical application
Data Engineering: Experience with large-scale data processing, ETL pipelines, and data quality frameworks
Database Systems: Familiarity with multiple datastores (PostgreSQL, Snowflake, RedShift, etc.)
Version Control: Proficiency with Git and collaborative development workflows
Nice to Have
ZIO Experience: Hands-on experience with ZIO for functional effects and concurrent programming
Machine Learning: Background in ML model development and statistical learning theory
Streaming Systems: Experience with real-time data streaming (Kafka, Kinesis, etc.)
Cloud Platforms: Experience with cloud-native Spark deployments (EMR, Dataproc, Databricks)
Performance Optimization: Experience with JVM tuning and Spark performance optimization
Academic Background: Advanced degree in Computer Science, Mathematics, Statistics, or related field
Product Development: Experience building data platform products (vs. project-based analytics)
Startup Experience: Previous experience working in fast-paced startup environments
Enterprise Experience: Experience working with enterprise-scale data systems and customers
Why Work Here?
Mission
We founded Qualytics based on our collective experiences working at a number of early-stage technology companies grappling with challenges related to data quality. We realized how few products are available to help organizations maintain quality & trust in their data pipelines. We're here to change that with a platform capable of detecting and proactively responding to data anomalies before they result in ML model drift, corruption of your aggregated calculations, or other pollution of your downstream systems. We believe that data is the most valuable resource to companies, and that resource isn't useful unless it's reliable.
Values
Qualytics is a values-driven organization, and we are excited about what we do. We're flexible, honest, hardworking, and collaborative. As a team, we bring our diverse backgrounds, beliefs, and experiences together to create better individual work experiences. We support and challenge one another to bring out the best in each of us, and it seems like we have a little fun along the way. We are lifelong learners, and support ongoing training and professional development on a wide variety of topics.
Perks
Competitive salary and equity: we pay up for great talent
Unlimited vacation – we trust you to take what you need – we clear the calendars on common holidays so you don't feel like you're going to miss an important meeting if you take off
All the benefits: medical, dental, and vision insurance, the works
Work with a passionate and dedicated team
Fun environment with like minded individuals who enjoy a company giphy conversation on slack more than they should
Please note that while the position is remote, we currently require applicants to be located in the United States and cannot offer visa sponsorship at this time.
