Gantry is building product testing and analytics for LLM-powered applications. We’re developing the most reliable, trustworthy way to evaluate LLM apps, and workflows to integrate those evaluations into the product development process. You can think of it like unit testing + Mixpanel for AI app builders. Gantry was founded by
Josh Tobin, former OpenAI researcher and co-founder of The Full Stack, and Vicki Cheung, former founding engineer at OpenAI and Compute team lead at Lyft. At Gantry, we believe AI practitioners should have access to tools that feel as magical as the ones they develop for their end-users. As an Applied AI Researcher, you will be responsible for identifying opportunities for AI-powered product experiences with product teams, as well as prototyping and working with the engineering team to implement the algorithms behind those tools. Some examples of the kinds of problems you can expect to work on include: Automated evaluation of large language models, model-augmented error analysis/failure mode detection, and fine-tuning large language models for specific use cases.This is an interdisciplinary role that will involve skill at developing statistical algorithms, training models, and lightweight prototyping and implementation.
Responsibilities
- Conduct innovative research and implement algorithms in the field of machine learning and LLMs
- Develop and train large scale machine learning and deep learning models.
- Design and implement practical solutions for complex AI/ML problems.
- Develop metrics and prototypes that can be used to drive business decisions.
- Work closely with our engineering team to integrate successful models and algorithms into our products.
- Stay up-to-date with the latest research and technologies in AI and machine learning, and be ready to implement them.
Requirements
- Proven experience in applying models in real-world applications and collaborating with product engineering teams.
- Strong knowledge of Python and experience with machine learning/deep learning frameworks like PyTorch.
- Extensive experience in training or fine-tuning language models such as GPT, BERT, etc.
- Demonstrated experience in delivering ML/AI projects from conceptualization to deployment.
- Experience or interest in prompt engineering and model chaining.
- Strong problem-solving skills, creative thinking, and the ability to thrive in a fast-paced, innovative, and collaborative environment.
- Excellent communication and teamwork skills, and the ability to explain complex AI/ML concepts to non-experts.
- Advanced degree (Ph.D. preferred) in Computer Science, Machine Learning, Artificial Intelligence, or a related quantitative field.
- Record of publications in top-tier AI/ML conferences or journals is a plus.
Gantry Systems, Inc. is an equal opportunity employer. We celebrate diversity, are committed to creating an inclusive environment for all employees, and intend to consider qualified applicants with criminal histories. Most of the team is based in San Francisco, but we are building a remote-friendly company and welcome applicants from anywhere in the US or Canada.Apply for this job
