好文集ChatGPT Plus¥119

A year as a Research Engineer working on RSI at Meta

文章目录

I never thought I would work on RSI at Meta as a Research Engineer.

I don’t come from an academic background, and to be honest, I started my career in technical support, fixing printers.

And yet, I get to work with brilliant people on one of our time's most impactful technologies.

What does “Research Engineer” mean?

I remember reading on Reddit that AI Research is the end-game of Data Science, and I largely agree.

If you've worked as a Data Scientist (sexiest job title in the world) then you already have what it takes: data, exploration, a good engineering base. Where Data Scientists might be commercially oriented, researchers fully concentrate on early ideas and bets with long-term pay-off.

Building reliable software and maintaining infrastructure is still a big part of making Research reproducible, and 90% of your time is spent writing code. The other 10% is you coming up with ideas, testing them, and repeating until you strike gold.

My team has Research Scientists, Software Engineers, Research Engineers, and Data Scientists, but we all largely do the same thing (Members of Research Staff lol?). Honestly, the title matters less than whether you can help answer the research question.

What I work on

I work on research agents: teaching agents to solve the world's hardest problems with tokens and compute.

The way I describe RSI (Recursive Self-Improvement) to family and friends is: teach AI Agents how to train the next generation of models and self-evolve. The recursive ambition is that each improvement helps the system find the next one. One route updates the model itself; another improves the prompts, tools, memory, or code around it. There’s still a lot to figure out.

I wrote more about this in From punch cards to a Terminator: Self-improving agents in 2026.

文章配图

Two routes to self-improvement: updating the foundation model or the scaffold around it. Figure 1 from Self-Improvements in Modern Agentic Systems: A Survey, also featured in my previous X article. Original source · My previous article

How can I get into AI research?

This might be the section you’ve been looking for :)

I’d prioritise having papers that recruiters can find and hiring managers can read. They give people something concrete to assess: how you can contribute to a research effort. It can be scaling on top of existing technologies (scaling agents, improving architectures) or trying out completely new things to revolutionise the field.

Working on a benchmark, winning a Kaggle competition, or building an agent harness is great experience and work to show. Be ready to explain what you did, what you measured, and why you made those choices.

For a starting project, pick one task and build a small agent harness around it. Run a baseline, change one thing, inspect the traces, and write up the result. Include what failed and what you’d try next.

That gives you something concrete to discuss in an interview, as well as practice with the skills below.

What skills matter in research engineering?

These are the areas I’d build a solid understanding of.

1. LLM architecture and the training cycle

You might work on pre-training, mid-training, or post-training. Whether you’re creating data, running evaluations, or launching training jobs, understanding the model helps you decide what to change and why.

When a benchmark score improves, can you explain what might have caused it? When training fails, do you know where to start looking?

2. Agent orchestration and development

Learn how to turn a research goal into a workflow an agent can execute: choosing tools, managing context, running experiments, and recovering from failures.

Start with a task you understand well. Follow the agent from its initial instructions to a result you can check. That’s a good way to discover where the harness needs work.

3. Working with coding agents

Can you use Codex, Claude, or another coding agent to work through a complex problem and produce high-quality, validated PRs?

Your team might have shared workflows, or you might build your own. Either way, you still need to define the task, review the implementation, run the checks, and verify the result.

4. Evaluation principles

Understand how benchmarks are designed, what they measure, and where they can mislead you.

Take PostTrainBench. It gives agents small base models, one H100 GPU, and ten hours to improve performance through post-training. Its evaluations include GPQA, GSM8K, and HealthBench. Other examples might be nanochat, Program Bench, SWE Bench, etc.

Read the methodology and inspect the traces alongside the leaderboard. Ask what the agent was allowed to do, how the score was calculated, and whether the result supports the conclusion being drawn.

5. Data basics still matter

Coding agents don’t remove the need to understand your data.

Inspect examples. Check quality, coverage, and whether training data overlaps with evaluation items. Before attributing an improvement to a new method, understand what changed in the data too.

Looking at data and traces can help you catch failures a final score hides. It would have helped me avoid some of the mistakes I made when I started. Start looking at something like Langfuse.

Research Taste

I remember a conversation with colleagues about what differentiates junior and senior researchers. A lot of it came down to taste.

Everyone learns this the hard way. Some results you haven't checked, a question you didn't prepare for. Early in my career, I learned a hard lesson: I vibed too hard - I ran a great-looking experiment, but when I was asked why the result had changed, I couldn’t explain it properly. I eventually checked the results and found that there was a bug that made my results look better than they actually were.

That day, I got +0.1 in taste. I started preparing for potential questions in advance and getting my answers ready. What had I changed? Why did I think it would help? What evidence did I have for my explanation?

For me, taste means making those calls with better judgment. It also means presenting results so someone else can question them.

My First Paper: Research Preference Models

Our RPM paper tackles a practical research question: how do you choose promising experiments when running every candidate is expensive?

Figure 2 shows the main results. It shows how inference-only and agentic RPM compare with “No RPM,” both over time and at the end of the run. The error bars show 95% confidence intervals (a very common topic of discussion in the team).

That’s the trade-off I want a reader to see quickly: how much better our selection methods perform than the unguided baseline, how close they get to those ceilings, and the uncertainty around the results. A lot of our experiments never led anywhere and weren't included - but they taught us a useful lesson.

Some intuition comes from doing more experiments: learning which architectures to try, recognising common training failures, and estimating what the next run will cost in time and compute.

文章配图

Figure 2 from AI Research Preference Models: RPM-augmented AIRA-dojo evaluation on AIRS-Bench, including 95% confidence intervals and oracle reference lines. Image supplied by the author. Source

Would I recommend Research Engineering?

Yes, if you enjoy applying research papers, exploring different approaches, and designing experiments to improve results.

You need to be comfortable with failure and quick to learn from it. That’s a big part of what makes the work interesting to me.

Some public work I’ve contributed to at Meta:

If you’re trying to get into the field, pick a small experiment, run a baseline, inspect what happened, and explain what you’d try next.

全文完
查看原文