Research Engineer, Superalignment

About the Team

• 
  Scalable oversight: How can we best leverage AI systems to assist evaluation of other AI systems on difficult tasks?
• 
  Generalization: Can we understand and control how our models generalize from easy tasks that humans can supervise to hard tasks that humans cannot?
• 
  Automated interpretability: Can we use AI to explain how LLMs work internally?
• 
  Robustness: How can we train our models to be aligned in worst-case situations?
• 
  Adversarial testing: If we deliberately train deceptively aligned models as testbeds,  can our oversight techniques, interpretability tools, and evaluations detect this misalignment?

About the Role

• 
  Writing performant and clean code for ML training.
• 
  Independently running and analyzing ML experiments to diagnose problems and understand which changes are real improvements.
• 
  Writing clean non-ML code, for example when building interfaces to let workers interact with our models or pipelines for managing human data.
• 
  Collaborating closely with a small team to balance the need for flexibility and iteration speed in research with the need for stability and reliability in a complex long-lived project.
• 
  Understanding our high-level research roadmap to help plan and prioritize future experiments.
• 
  Implement experiments to measure the effectiveness of scalable oversight techniques such as AI-assisted feedback and Debate
• 
  Studying generalization to see when AI systems trained on easy problem can solve hard problems
• 
  Managing large datasets from interpretability experiments and creating visualizations to explore interpretability data
• 
  Investigating situations when training against a reward signal causes model outputs to deteriorate
• 
  Exploring methods to understand and predict model behaviors, such as finding inputs causing anomalous circuits or catastrophic outputs
• 
  Designing novel approaches for using LLMs in alignment research

You might thrive in this role if you:

• 
  Are excited about OpenAI’s mission of building safe, universally beneficial AGI and are aligned with OpenAI’s charter
• 
  Want to use your engineering skills to push the frontiers of what state-of-the-art language models can accomplish
• 
  Possess a strong curiosity about aligning and understanding ML models, and are motivated to use your career to address this challenge
• 
  Enjoy fast-paced, collaborative, and cutting-edge research environments
• 
  Have experience implementing ML algorithms (e.g., PyTorch)
• 
  Can develop data visualization or data collection interfaces (e.g., JavaScript, Python)
• 
  Want to ensure that powerful AI systems stay under human control

About OpenAI