Berkeley, California
About METR
METR is a non-profit which does empirical research to determine whether frontier AI models pose a significant threat to humanity. It’s robustly good for civilization to have a clear understanding of what types of danger AI systems pose, and know how high the risk is. You can learn more about our goals from our videos (overall goals, recent update).
Some highlights of our work so far:
- Establishing autonomous replication evals: Thanks to our work, it’s now taken for granted that autonomous replication (the ability for a model to independently copy itself to different servers, obtain more GPUs, etc) should be tested for. For example, labs pledged to evaluate for this capability as part of the White House commitments.
- Pre-release evaluations: We’ve worked with OpenAI and Anthropic to evaluate their models pre-release, and our research has been widely cited by policymakers, AI labs, and within government.
- Inspiring lab evaluation efforts: Multiple leading AI companies are building their own internal evaluation teams, inspired by our work.
- Early commitments from labs: Anthropic credited us for their recent Responsible Scaling Policy (RSP), and OpenAI recently committed to releasing a Risk-Informed Development Policy (RDP). These fit under the category of “evals-based governance”, wherein AI labs can commit to things like, “If we hit capability threshold X, we won’t train a larger model until we’ve hit safety threshold Y”.
We’ve been mentioned by the UK government, Obama, and others. We’re sufficiently connected to relevant parties (labs, governments, and academia) that any good work we do or insights we uncover can quickly be leveraged.
About the role
You’ll design and implement experiments to improve and validate METR’s evaluation protocols, to ensure that they provide accurate measurements of the level of risk posed by highly capable models. You’ll help answer questions including:
- How much more capable might agents become with additional post-training enhancements? How much “safety margin” do we need to leave?
- Can we design a continuous scale of task difficulty that results in a predictable relationship between increased inputs and increased performance?
- Can we predict what kinds of agent failures are fixable by a small amount of fine-tuning versus ones which might represent more fundamental limitations?
- How capable are the strongest open-source agents?
Your work will ensure that METR’s evaluation protocols can robustly predict whether a new frontier model poses catastrophic risks.
You’ll help with our:
- Research direction
- Understand what kinds of abilities we need to be evaluating for, and what properties we most need our evaluations to have
- Find the most promising directions to explore; design experiments and research roadmap
- Collaborate with the threat modeling team, sharing your ML domain expertise to help us evaluate models for AI R&D skills
- Research execution
- Rapidly execute experiments, obtain reliable results
- Design sensible pipelines and workflows (know which things are going to be reused and need to be good versus what things it's ok to do scrappily)
- Quickly interpret results - recognize what is signal vs noise, notice when things don't look right and there might be a bug, know where to look for bugs in ML experiments
- Know how much work different approaches are likely to be and how promising they are; when you have uncertainties, get information as quickly as possible
What we’re looking for
An ideal candidate would be a machine learning researcher with extensive experience working with frontier LLMs and a track record of successful execution-heavy research projects.
- (Required) Have at least 1 research project related to machine learning in which you played a major role
- Have rigorous, legible research that’s written up well - carefully addressing validity, confounders, overfitting etc.
Some bonus attributes include:
- Threat-model-relevant subject-matter expertise
- Large model inference and training compute optimization
- non-standard hardware setups
- Complicated distributed system setups for many LM agents
- Cybersecurity
- Cybercrime, fraud, KYC, authentication
- Experience working at a frontier lab
- Comfortable managing external partnerships
- Skilled at hiring and managing a small team
- Identify our talent needs and recruit world-class employees
- Harness them effectively, providing direction and motivation and fostering their growth
Logistics
Deadline to apply: None. Applications will be reviewed on a rolling basis.
Salary range for research scientist: $158,000—$365,000 USD
Apply for this job
We encourage you to apply even if your background may not seem like the perfect fit! We would rather review a larger pool of applications than risk missing out on a promising candidate for the position. If you lack US work authorization and would like to work in-person (preferred), we can likely sponsor a cap-exempt H-1B visa for this role.
We are committed to diversity and equal opportunity in all aspects of our hiring process. We do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We welcome and encourage all qualified candidates to apply for our open positions.
Registering interest is short/quick — there’s just a single required question, which can be answered with a few bullet points. Register interest!