Job Description
Join Our Team at Litera: Where Legal Technology Meets Excellence
Litera has been at the forefront of legal technology innovation for over 25 years, crafting legal software to amplify impact and maximize efficiency. Developed by the best legal minds in the industry, our comprehensive suite of integrated legal tools is both powerful and user-friendly and simplifies the way modern firms manage core legal workflows, secure collaboration, and organize firm knowledge and experience. Every day, we help more than 2.3 million legal professionals focus on their craft. Litera: Less busy work, more of your life’s work.
About the Role
We are seeking an experienced LLM Engineer to join our newly established LLM Ops Team. In this role, you will be responsible for managing the complex lifecycle of Large Language Models from development to deployment, monitoring, and continuous improvement.
Key Responsibilities
Model Development & Training
Fine-tune pre-trained models for specific use cases
Curate and prepare datasets for training
Manage training infrastructure, resources, and computational environments
Implement optimization techniques to improve model performance
Deployment & Serving
Develop and manage APIs for model serving
Scale infrastructure to handle varying demand loads
Build and maintain the GenAI middleware/sidecar layer
Integrate LLMs with existing systems and data sources
Monitoring & Evaluation
Track performance metrics including latency and throughput
Monitor quality metrics such as hallucination rates and accuracy
Optimize costs associated with model inference and training
Create and maintain dashboards for real-time performance insights
Testing & Quality Assurance
Create and maintain golden datasets for benchmark testing
Implement statistical validation methods for model outputs
Set up similarity matching criteria for response evaluation
Develop confidence score thresholds for production systems
Feedback Loops & Iteration
Design and implement user feedback collection systems
Establish continuous improvement processes
Create A/B testing frameworks for model and feature evaluation
Conduct trace analysis to identify areas for performance optimization
Safety & Compliance
Implement content moderation systems
Detect and mitigate bias in model outputs
Ensure regulatory compliance in AI systems
Develop output validation frameworks
Prompt Management
Version and store prompts systematically
Create and maintain prompt templates
Set up playground environments for prompt testing
Abstract prompts from application code for better maintainability
Qualifications
Experience with LLM development, fine-tuning, and deployment
Strong programming skills, particularly in Python
Experience with Kubeflow, Apache Airflow, or other LLM Pipeline technology
Experience with Azure OpenAI, AWS Sagemaker, and/or Vertex AI
Understanding of machine learning operations and MLOps principles
Knowledge of infrastructure scaling and optimization
Experience with AI monitoring tools and dashboard creation
Familiarity with AI safety, bias detection, and compliance requirements
Strong problem-solving abilities and analytical thinking
Familiarity with ISO 27001 and SOC2 Certification
Join Our Team
Be part of building the future of AI operations as we establish robust systems for managing the entire LLM lifecycle. This role offers the opportunity to work at the cutting edge of artificial intelligence implementation and operations.
Why Join Litera?
The company culture: We emphasize helping each other grow, doing the right thing always, and being part of a journey to amplify impact, creating an exciting and fulfilling work environment
Commitment to Employees: Our people commitment is based on what employees love most about being part of the team, focusing on tools that matter to the difference-makers in the legal world and amplifying their impact
Global, Dynamic, and Diverse Team: Our is a global company with ambitious goals and unlimited opportunities, offering a dynamic and diverse work environment where employees can grow, listen, empathize, and problem-solve together
Comprehensive Benefits Package: Experience peace of mind with our health insurance, retirement savings plans, generous paid time off, and a supportive work-life balance. We invest in your well-being and future, ensuring a rewarding career journey.
Career Growth and Development: We provide career paths and opportunities for professional development, allowing employees to progress through various technical and leadership roles
California, Colorado, Connecticut, Illinois, Maryland, Nevada, New Jersey, New York, Ohio, Rhode Island, or Washington Residents Only: The salary range for California, Colorado, Connecticut, Illinois, Maryland, Nevada, New Jersey, New York, Ohio, Rhode Island, or Washington residents is $100,000 to $132,000. Pay is based on several factors including but not limited to education, work experience, certifications, etc. In addition to your salary, Litera offers benefits such as a comprehensive benefits package, incentive and recognition programs, and 401k contribution (all benefits are subject to eligibility requirements).
Litera is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
