AF - ML Safety Research Advice - GabeM by Gabe M

Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ML Safety Research Advice - GabeM, published by Gabe M on July 23, 2024 on The AI Alignment Forum.
This is my advice for careers in empirical ML research that might help AI safety (ML Safety). Other ways to improve AI safety, such as through AI governance and strategy, might be more impactful than ML safety research (I generally think they are). Skills can be complementary, so this advice might also help AI governance professionals build technical ML skills.
1. Career Advice
1.1 General Career Guides
Preventing an AI-related catastrophe - 80,000 Hours
A Survival Guide to a PhD (Andrej Karpathy)
How to pursue a career in technical AI alignment - EA Forum
AI safety technical research - Career review - 80,000 Hours
Beneficial AI Research Career Advice
2. Upskilling
2.1 Fundamental AI Safety Knowledge
AI Safety Fundamentals - BlueDot Impact
AI Safety, Ethics, and Society Textbook
Forming solid AI safety threat models helps you select impactful research ideas.
2.2 Speedrunning Technical Knowledge in 12 Hours
Requires some basic coding, calculus, and linear algebra knowledge
Build Intuition for ML (5h)
Essence of linear algebra - 3Blue1Brown (3h)
Neural networks - 3Blue1Brown (2h)
Backpropagation, the foundation of deep learning (3h)
Neural Networks: Backpropagation - CS 231N (0.5h)
The spelled-out intro to neural networks and backpropagation: building micrograd (2.5h)
Transformers and LLMs (4h)
[1hr Talk] Intro to Large Language Models (1h)
The Illustrated Transformer - Jay Alammar (1h)
Let's build GPT: from scratch, in code, spelled out. (2h)
2.3 How to Build Technical Skills
Traditionally, people take a couple of deep learning classes.
Stanford CS 224N | Natural Language Processing with Deep Learning (lecture videos)
Practical Deep Learning for Coders - Practical Deep Learning (fast.ai)
Other curricula that seem good:
Syllabus | Intro to ML Safety
Levelling Up in AI Safety Research Engineering [Public]
ARENA
Maybe also check out recent topical classes like this with public lecture recordings: CS 194/294-267 Understanding Large Language Models: Foundations and Safety
Beware of studying too much.
You should aim to understand the fundamentals of ML through 1 or 2 classes and then practice doing many manageable research projects with talented collaborators or a good mentor who can give you time to meet.
It's easy to keep taking classes, but you tend to learn many more practical ML skills through practice doing real research projects.
You can also replicate papers to build experience. Be sure to focus on key results rather than wasting time replicating many experiments.
"One learns from books and reels only that certain things can be done. Actual learning requires that you do those things." -Frank Herbert
Note that ML engineering skills will be less relevant over time as AI systems become better at writing code.
A friend didn't study computer science but got into MATS 2023 with good AI risk takes. Then, they had GPT-4 write most of their code for experiments and did very well in their stream.
Personally, GitHub Copilot and language model apps with code interpreters/artifacts write a significant fraction of my code.
However, fundamental deep learning knowledge is still useful for making sound decisions about what experiments to run.
2.4 Math
You don't need much of it to do empirical ML research.
Someone once told me, "You need the first chapter of a calculus textbook and the first 5 pages of a linear algebra textbook" to understand deep learning.
You need more math for ML theory research, but theoretical research is not as popular right now.
Beware mathification: authors often add unnecessary math to appease (or sometimes confuse) conference reviewers.
If you don't understand some mathematical notation in an empirical paper, you can often send a screenshot to an LLM chatbot f...