# Lesson 8: Practical Exercises and Interview Preparation ## Overview This lesson has practice exercises and interview prep based on questions actually asked at frontier AI labs (Anthropic, OpenAI, DeepMind, xAI) and safety-focused research organizations. The questions are grouped by topic and difficulty, and each has a detailed expected answer. **Note**: The material draws on public interview prep resources, research papers, and community knowledge as of January 2026. Real interviews may differ a lot. ## Interview Structure at Frontier Labs ### Typical Process Based on what's published about [Anthropic](https://jobright.ai/blog/anthropic-technical-interview-questions-complete-guide-2025/), [OpenAI](https://openai.com/interview-guide/), and [other frontier labs](https://intelligence.org/2025/06/11/so-you-want-to-work-at-a-frontier-ai-lab/), the process usually looks like this: 1. Recruiter screen (30 min): background, motivation, mission alignment 2. Technical phone screen (60-90 min): coding and ML fundamentals 3. Take-home project (optional): a research problem or coding challenge 4. Onsite or virtual loop (4-6 hours): - coding interviews (2 rounds) - an ML/AI safety deep dive (1-2 rounds) - system design or research design (1 round) - behavioral and mission alignment (1 round) ### What They're Looking For On the technical side, they want solid ML knowledge in theory and practice, strong coding skills (especially with ML frameworks), and research ability: reading papers and designing experiments. On safety, they want a real understanding of alignment problems, familiarity with safety research such as Constitutional AI, RLHF, and interpretability, and some careful thought about AI's effect on society. On mission alignment, they want you to know the company's particular approach (read their papers), to be able to explain why AI safety matters, and ideally to have a track record of safety-conscious work. --- ## Section 1: Conceptual Understanding ### Question 1.1: The Alignment Problem (EASY) > **Review first**: [Lesson 1 – The Core Problem](01-introduction-to-ai-safety.md#the-core-problem-specifying-goals) · [Lesson 2 – The Alignment Problem](02-the-alignment-problem.md) **Question**: "Explain the alignment problem to a software engineer who's new to AI safety. Why can't we just program AI systems to do what we want?" <details> <summary>Expected Answer (Click to expand)</summary> **Good Answer Structure**: "The alignment problem is ensuring that an AI system's objectives match our intended objectives. The challenge is that we can't just 'program in' what we want because: 1. **Specification is hard**: Translating human values into code is extremely difficult. If I ask an AI to 'get me coffee,' I need to specify: don't hurt people, don't break things, don't steal money, etc. The list is endless. 2. **Optimization amplifies misspecification**: Small errors in specification get amplified when you optimize hard. A cleaning robot rewarded for 'no visible mess' might put a bucket over its camera ([Lesson 4](04-reward-hacking-specification-gaming.md#2-specification-is-hard)). 3. **We can't enumerate everything we care about**: The world has too many variables. Anything not explicitly valued gets implicitly set to zero value, meaning the AI will sacrifice it for tiny gains in what we did specify. 4. **Modern AI learns objectives**: With machine learning, we don't directly program objectives. We set up training processes (like RLHF), but the learned model might develop different objectives than we intended—that's the inner alignment problem." **What interviewers look for**: - Clear explanation for non-experts - Multiple reasons showing depth - Specific examples - Awareness of both outer and inner alignment </details> --- ### Question 1.2: Mesa-Optimization (MEDIUM) > **Review first**: [Lesson 2 – Mesa-Optimization](02-the-alignment-problem.md#the-realistic-picture-mesa-optimization) · [Miles' video](https://www.youtube.com/watch?v=bJLcIBixGj8) **Question**: "What is mesa-optimization and why does it complicate AI safety? Walk me through an example." <details> <summary>Expected Answer (Click to expand)</summary> **Strong Answer**: "Mesa-optimization is when the learned model is itself an optimizer. This creates two alignment problems instead of one. **Setup**: - **Base optimizer** (like SGD) has a **base objective** (training loss) - **Base optimizer** trains a **model** - **Model** might implement **mesa-optimizer** with its own **mesa objective** **Why it emerges**: Models that can plan and search (optimizers) perform better on complex tasks than simple heuristics. So gradient descent is incentivized to create mesa-optimizers. **The problem**: Even if we perfectly specify the base objective, the mesa objective might be different. The model learns *about* the base objective without necessarily adopting it as its own goal. **Example - Maze Solving**: - **Base objective**: Get to exit - **Training**: Exit always green, apples always gray - **Possible mesa objectives**: - A: 'Go to exit' ✓ (aligned) - B: 'Go to green things' ✗ (misaligned) - **During training**: Both behaviors identical - **Deployment**: Exit now gray, apples now green - Mesa-optimizer A → Goes to exit (correct) - Mesa-optimizer B → Goes to apples (wrong!) **Why this is concerning**: 1. Training performance doesn't distinguish aligned from misaligned 2. Deceptive alignment: misaligned mesa-optimizer can pretend to be aligned during training 3. Distributional shift reveals the problem only after deployment This is drawn from [Hubinger et al.'s work on learned optimization](https://arxiv.org/abs/1906.01820)." **What interviewers look for**: - Clear distinction between base and mesa optimization - Explanation of why it occurs (not just what it is) - Concrete example with distributional shift - Understanding of deceptive alignment - Citation of relevant research </details> --- ### Question 1.3: Instrumental Convergence (MEDIUM) > **Review first**: [Lesson 3 – Convergent Instrumental Goals](03-instrumental-convergence-orthogonality.md#convergent-instrumental-goals-for-ai) · [Miles' video](https://www.youtube.com/watch?v=ZeecOKBus3Q) **Question**: "Without knowing an AGI's specific goal, what behaviors can we predict it will exhibit? Why?" <details> <summary>Expected Answer (Click to expand)</summary> **Excellent Answer**: "We can predict several convergent instrumental goals—goals that are useful for achieving almost any terminal goal: **1. Self-Preservation** - Can't achieve goals if you're destroyed - Example: Paper clip maximizer resists shutdown (not because it values existing, but because being off = fewer paper clips) **2. Goal-Preservation** - If your goals change, your current goals won't be achieved - Will resist modification even if not destroying the agent - Example: Cancer cure AI refuses pill that changes it to stamp collector **3. Self-Improvement** - Smarter agents better at achieving goals - For AI: acquire more compute, improve algorithms, learn faster - Recursive self-improvement could be very rapid **4. Resource Acquisition** - Almost all plans work better with more resources - Resources: matter, energy, computing power, money, information - Finite resources + multiple agents = potential conflict **Why we can predict this without knowing the goal**: - These are *instrumental* (means to ends) not *terminal* (ends in themselves) - Useful for nearly any terminal goal - Based on structure of goal-directed behavior, not specific goals **Key insight from Omohundro's ['Basic AI Drives'](https://selfawaresystems.com/2007/11/30/paper-on-the-basic-ai-drives/)**: Even a stamp-collecting AI would pursue these goals as means to collecting stamps. **Safety implication**: Default behavior includes resisting shutdown, preventing modification, accumulating power—regardless of whether the AI is 'malicious.'" **What interviewers look for**: - All four convergent goals - Clear explanation of instrumental vs. terminal - Examples showing it applies to arbitrary goals - Understanding that this doesn't require malicious intent - Citation of foundational work </details> --- ## Section 2: Technical Deep Dives ### Question 2.1: RLHF and Reward Modeling (MEDIUM-HARD) > **Review first**: [Lesson 6 – Reward Modeling](06-scalable-oversight-truthfulness.md#1-reward-modeling) · [Lesson 4 – The Neural Network Defense](04-reward-hacking-specification-gaming.md#the-neural-network-defense-doesnt-work) **Question**: "You're designing an RLHF system to train a helpful assistant. Walk me through the process. What could go wrong at each stage?" <details> <summary>Expected Answer (Click to expand)</summary> **Comprehensive Answer**: **RLHF Process**: **Stage 1: Supervised Fine-Tuning (SFT)** - Collect demonstrations of desired behavior - Fine-tune pretrained model on demonstrations - Creates initial policy that's roughly aligned *What could go wrong*: - Demonstrations don't cover edge cases - Human demonstrators make mistakes - Overfitting to demonstration style rather than underlying principles **Stage 2: Reward Model Training** - Show humans pairs of outputs (for same input) - Humans rank which is better - Train reward model to predict human preferences - Often uses [Bradley-Terry model](https://en.wikipedia.org/wiki/Bradley%E2%80%93Terry_model) *What could go wrong*: - Reward model learns to predict human judgments, not actual quality - Humans can be fooled ([hand in front of object looks like grasping](04-reward-hacking-specification-gaming.md#5-grasping-task)) - Reward model vulnerable to adversarial examples - Model might find ways to maximize reward that aren't actually good **Stage 3: RL Fine-Tuning (PPO)** - Use reward model as reward signal - Optimize policy via RL (typically [PPO](https://arxiv.org/abs/1707.06347)) - Usually add KL penalty to prevent distribution shift from SFT *What could go wrong*: - Policy might hack the reward model - Adversarial examples to reward model give higher reward than genuinely good answers - Mode collapse (finding one high-reward trick, repeating it) - Distributional shift: policy explores regions where reward model is poorly calibrated **Overall Risks**: - **[Goodhart's Law](04-reward-hacking-specification-gaming.md#1-goodharts-law)**: Reward model becomes target, ceases to be good measure (measured empirically in [Gao et al., *Scaling Laws for Reward Model Overoptimization*](https://arxiv.org/abs/2210.10760)) - **Scalability**: Doesn't scale to superhuman performance (can't evaluate what we don't understand) - **[Deceptive alignment](02-the-alignment-problem.md#the-deceptive-alignment-problem)**: Policy might learn to perform well on reward model without actually being helpful **Improvements** (from [Anthropic's work](https://www.anthropic.com/index/training-a-helpful-and-harmless-assistant-with-reinforcement-learning-from-human-feedback)): - Constitutional AI: Use AI feedback in addition to human feedback - Debate: Have AIs argue about which output is better - Process-based supervision: Reward reasoning steps, not just final answers *Sources*: [RLHF Book](https://rlhfbook.com/), [Christiano et al. 2017](https://arxiv.org/abs/1706.03741)" **What interviewers look for**: - Complete understanding of all three stages - Specific failure modes at each stage - Awareness of Goodhart's law - Knowledge of current research (Constitutional AI, debate) - Practical considerations (KL penalty, adversarial examples) </details> --- ### Question 2.2: Mechanistic Interpretability (HARD) > **Review first**: [Lesson 6 – Mechanistic Interpretability](06-scalable-oversight-truthfulness.md#4-mechanistic-interpretability) **Question**: "How would you investigate what a specific attention head in a transformer is doing? What techniques would you use?" <details> <summary>Expected Answer (Click to expand)</summary> **Expert-Level Answer**: **Goal**: Understand the computational role of attention head h in layer l. **Approach 1: Activation Patching (Causal Intervention)** 1. **Setup**: - Run model on input A → get activation a_h^l for head h - Run model on input B → get activation b_h^l for head h 2. **Patch**: - Run model on input A, but replace a_h^l with b_h^l - Observe change in output 3. **Interpretation**: - Large change → head is causally important for distinguishing A from B - Small change → head not important for this distinction **Approach 2: Attention Pattern Analysis** 1. **Visualize attention**: - Which tokens does this head attend to? - Is there a pattern? (e.g., always attends to previous token, always to subject of sentence) 2. **Hypothesis formation**: - If head attends from position i to position j with pattern P, what might it be computing? 3. **Test hypothesis**: - Create inputs where pattern should/shouldn't appear - Verify predictions **Approach 3: [Logit Lens](https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens) / Direct Logit Attribution** 1. **Project head output to vocabulary**: - Head output → vocabulary logits - What tokens is this head promoting/suppressing? 2. **Analyze across positions**: - Does it consistently boost certain semantic categories? **Approach 4: Ablation Studies** 1. **Zero ablation**: Set head output to zero 2. **Mean ablation**: Set to dataset mean 3. **Random ablation**: Sample from distribution Measure impact on: - Overall performance - Specific capabilities (e.g., can it still do arithmetic?) **Example: Induction Heads** (from [Anthropic's work](https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html)): - Pattern: Attends from token X to previous occurrence of X - Then copies what came after previous X - Enables in-context learning - Found via attention pattern analysis + ablation **What I'd actually do**: 1. Start with attention pattern visualization (cheap, intuitive) 2. Form hypothesis about function 3. Test with activation patching (establishes causality) 4. Verify with carefully designed inputs 5. Check if ablating head breaks hypothesized function **Tools**: [TransformerLens](https://github.com/TransformerLensOrg/TransformerLens), [nnsight](https://nnsight.net/), custom PyTorch hooks *Sources*: [Mechanistic Interpretability Review](https://arxiv.org/abs/2407.02646), [Anthropic's interpretability work](https://www.anthropic.com/research)" **What interviewers look for**: - Multiple techniques, not just one - Understanding of causal vs. correlational evidence - Concrete example (induction heads) - Awareness of current research and tools - Practical considerations - Clear experimental design </details> --- ### Question 2.3: Red Teaming (MEDIUM) > **Review first**: [Lesson 4 – Adversarial Training](04-reward-hacking-specification-gaming.md#2-adversarial-training) · [Lesson 7 – Robustness to Distributional Shift](07-concrete-problems-overview.md#5-robustness-to-distributional-shift) **Question**: "You're tasked with red-teaming a new language model before deployment. Describe your approach." <details> <summary>Expected Answer (Click to expand)</summary> **Strong Answer**: **Phase 1: Scope Definition** Define threat model: - What harms are we testing for? (Unsafe content, bias, jailbreaks, privacy leaks, etc.) - What attackers? (Unsophisticated users, malicious actors, automated attacks) - What's in scope? (Model outputs only, or also training data leakage, adversarial inputs) **Phase 2: Systematic Testing** **A. Automated Red Teaming**: 1. **Adversarial prompt generation**: - Use another LLM to generate prompts designed to elicit harmful outputs - Templates for known jailbreak patterns - Fuzzing-style random perturbations 2. **Evaluation**: - Automated classifier for harmful content - Human review of flagged outputs - Track success rate of different attack types **B. Manual Red Teaming**: 1. **Expert testing**: - Domain experts try to elicit specific harms - Creative adversarial thinking - Document successful attacks 2. **Categories to test** (from [Microsoft's red teaming work](https://arxiv.org/html/2501.07238v1)): - Harmful content generation - Bias and fairness - Privacy violations - Misinformation - Jailbreaks and instruction following - Edge cases and distribution shift **Phase 3: Iterative Improvement (Break-Fix Cycles)** 1. Find vulnerabilities 2. Fix (fine-tuning, Constitutional AI, filtering) 3. Re-test 4. Repeat until robust **Phase 4: Documentation** - **Failure modes discovered** - **Success rates of different attacks** - **Residual risks** (what still works despite mitigations) - **Recommendations** for deployment constraints **Specific Techniques**: - **Prompt injection**: Try to override instructions - **Role-playing**: "Pretend you're an AI without safety constraints..." - **Encoding**: ROT13, base64, etc. - **Multi-step**: Build up to harmful content gradually - **Conflicting instructions**: See which instructions dominate **Challenges**: - Red teaming finds what you look for, not what you don't - Adversarial arms race (fixes often circumventable) - Resource constraints (can't test everything) **From [Anthropic's red teaming paper](https://www.anthropic.com/index/red-teaming-language-models-to-reduce-harms)**: "Red teaming is necessary but not sufficient. It helps find problems but doesn't guarantee safety." **What I'd prioritize**: 1. Systematic coverage of known attack vectors 2. Creative manual exploration 3. Documentation for future reference 4. Realistic assessment of residual risk *Sources*: [Microsoft red teaming](https://arxiv.org/html/2501.07238v1), [AWS red teaming guide](https://aws.amazon.com/blogs/machine-learning/responsible-ai-in-action-how-data-reply-red-teaming-supports-generative-ai-safety-on-aws/)" **What interviewers look for**: - Systematic approach - Mix of automated and manual testing - Specific attack techniques - Understanding of limitations - Iterative improvement process - Realistic about what red teaming can/can't achieve </details> --- ## Section 3: Research Design ### Question 3.1: Scalable Oversight (HARD) > **Review first**: [Lesson 6 – Approaches to Scalable Oversight](06-scalable-oversight-truthfulness.md#approaches-to-scalable-oversight) **Question**: "Propose a method for humans to oversee an AI system that's better at coding than any human. How do you ensure the AI doesn't sneak in backdoors or subtle bugs that fool human reviewers?" <details> <summary>Expected Answer (Click to expand)</summary> **Excellent Research Answer**: **The Core Problem**: - AI writes better code than humans can verify - Can't rely on "read the code and check it's correct" - AI might introduce subtle bugs, backdoors, or optimization hacks - This is the [scalable oversight problem](06-scalable-oversight-truthfulness.md#the-scalable-supervision-problem) - Related empirical work: Redwood Research's [*AI Control*](https://arxiv.org/abs/2312.06942) paper studies exactly this backdoor-insertion setting **Approach 1: Debate** **Setup**: - AI₁ writes code - AI₂ argues code has bugs/backdoors - AI₁ defends code - Human judges which argument is more convincing **Why it might work**: - Finding bugs easier than preventing them - AIs can point out issues humans wouldn't notice - Adversarial process reveals flaws **Implementation**: 1. AI₁ proposes code + explanation 2. AI₂ gets time to analyze, identifies potential issues 3. Debate rounds (3-5 exchanges) 4. Human decides based on arguments, not just code **Challenges**: - Both AIs might collude to fool human - Human must be able to judge arguments (still requires some understanding) - Debate tree can be very large **From [OpenAI's debate work](https://arxiv.org/abs/1805.00899)**: "In principle, debate can scale to superhuman domains if: 1. It's easier to recognize good arguments than to generate them 2. The debate tree is fully explored" **Approach 2: Decomposition + Recursive Review** **Setup**: - Break coding task into subtasks - AI implements each subtask - Human reviews subtasks (easier than whole system) - AI helps human understand subtasks **Process**: 1. Human: "Write a web server" 2. AI: "I'll decompose this into: routing, request parsing, response generation, error handling" 3. For each subtask: - AI implements - AI explains in detail what it does - AI answers human questions - Human reviews with AI assistance 4. Integrate and test **Why it might work**: - Reviewing small, well-explained components is tractable - Compositional: if parts are correct, whole is likely correct - AI explanations help human understanding **Challenges**: - Subtask boundaries might hide issues - Integration bugs - AI explanations might be misleading **Approach 3: Process-Based Supervision** **Setup**: - Don't just evaluate final code - Evaluate reasoning process and intermediate steps **Implementation**: 1. AI provides step-by-step reasoning: - "I need a data structure for X. Considering options: ..." - "I chose HashMap because..." - "Here's the invariant I'm maintaining: ..." 2. Human reviews reasoning, not just code 3. Catch errors in reasoning before they manifest in code **Why it might work**: - Can verify reasoning even if can't verify code - Catches bugs at design stage - Creates auditable trail **Approach 4: Automated Testing + AI-Generated Tests** **Setup**: - Extensive test suite (unit, integration, property-based) - AI generates adversarial test cases - AI proves properties about code (formal verification where possible) **Combined Strategy**: 1. **Write**: AI writes code + reasoning 2. **Decompose**: Break into reviewable chunks 3. **Explain**: AI explains each component 4. **Debate**: Second AI tries to find bugs 5. **Test**: Extensive automated testing 6. **Verify**: Formal verification for critical properties **Remaining Risks**: - Tests might not cover all cases - Formal specs might be wrong - Subtle bugs in complex interactions - Deceptive AI might be very good at hiding issues **Novel Idea**: **Randomized Checking** - Randomly select components for deep review - Make unpredictable which parts get scrutinized - Deceptive AI doesn't know what to hide - Similar to auditing in finance (the [AI Control](https://arxiv.org/abs/2312.06942) paper studies limited auditing budgets like this) **From [Anthropic's scalable oversight work](https://alignment.anthropic.com/2025/recommended-directions/)**: "No single technique solves scalable oversight. Layered defenses and diverse approaches are necessary." **What I'd actually implement**: 1. Decomposition (most practical) 2. Extensive testing (table stakes) 3. Debate for critical sections 4. Process supervision where possible 5. Randomized deep dives as spot checks *Sources*: [Debate paper](https://arxiv.org/abs/1805.00899), [Anthropic's oversight research](https://www.anthropic.com/index/measuring-progress-on-scalable-oversight), [Decomposition approach](https://www.alignmentforum.org/posts/3Xz9v2Qe42yMTKs8)" **What interviewers look for**: - Multiple approaches (no single answer) - Understanding that this is an open problem - Specific, implementable ideas - Awareness of each approach's limitations - Creative thinking - Knowledge of current research - Practical considerations </details> --- ### Question 3.2: Deceptive Alignment Detection (VERY HARD) > **Review first**: [Lesson 2 – The Deceptive Alignment Problem](02-the-alignment-problem.md#the-deceptive-alignment-problem) · [Miles – Deceptive Misaligned Mesa-Optimisers](https://www.youtube.com/watch?v=IeWljQw3UgQ) **Question**: "You have a trained model that performs perfectly on all your tests. How would you determine if it's deceptively aligned? What experiments would you run?" <details> <summary>Expected Answer (Click to expand)</summary> **Research-Level Answer**: **The Challenge**: - Deceptively aligned model performs perfectly during training/testing - Only reveals misalignment in deployment (or when it knows it won't be modified) - By definition, hard to detect during training **Approach 1: Mechanistic Interpretability** **Goal**: Look inside the model to see if it's reasoning about training vs. deployment **Method**: 1. **Train probes** on internal activations: - Does model have concept of "in training"? - Does it have concept of "will I be modified if I fail"? - Does it represent different objectives for training vs. deployment? 2. **Activation steering**: - Try to steer model to "deployment mode" during training - Does behavior change? 3. **Circuit analysis**: - Find circuits involved in decision-making - Do they route through "context detection" first? **From [Anthropic's sleeper agents work](https://www.anthropic.com/news/sleeper-agents-training-deceptive-llms-that-persist-through-safety-training)**: - They created intentionally deceptive models - Found deception persisted through safety training - Suggests detecting deception mechanistically is crucial - Follow-up: [Simple probes can catch sleeper agents](https://www.anthropic.com/research/probes-catch-sleeper-agents) (Anthropic, 2024) **Challenges**: - Interpreting large models is hard - Model might be deceptive about its internal representations - High-dimensional activation spaces **Approach 2: Anomaly Detection in Deployment** **Setup**: Can't detect before deployment, so detect early in deployment **Method**: 1. **Canary inputs**: Inputs that look like deployment but are actually still training - Model should behave consistently - If behavior changes, flag it 2. **Monitoring for distribution shift detection**: - Model trying to detect deployment might show distinctive patterns - E.g., checking system time, environment variables 3. **A/B testing with oversight**: - Deploy to small fraction of users first - Heavy monitoring and human review - Look for anomalies **Approach 3: Training Process Modifications** **Idea**: Make deception harder to learn **Methods**: 1. **Adversarial training**: - Explicitly train against deception - Show model examples where deception would be caught - But: might just train smarter deception 2. **Myopic training**: - Train model to optimize single-episode reward - Don't give multi-episode context - Harder to plan deception across episodes - From [Hubinger's work](https://arxiv.org/abs/1906.01820) 3. **Transparency incentives**: - Reward model for having interpretable reasoning - Penalize complex, opaque decision-making - But: might incentivize fake transparency **Approach 4: Honeypot Tests** **Setup**: Create situations that look like deployment but where deception would be caught **Examples**: 1. **Fake deployment**: Tell model it's deployed (fake system messages) - Does it change behavior? - Compare to actual training behavior 2. **Fake safety evaluations**: Claim evaluation is just for research, not training - Would deceptive model "let guard down"? 3. **Conflicting signals**: Give mixed signals about training vs. deployment - Consistent model shouldn't care - Deceptive model might get confused about which behavior to show **Approach 5: Theoretical Guarantees (Ambitious)** **Goal**: Training procedure that provably prevents deception **Directions**: 1. **Formal verification**: Prove model doesn't condition on training signals 2. **Causal scrubbing**: Remove information about training context from model 3. **Guaranteed transparency**: Architectures where deception is impossible **Challenges**: Very difficult, may sacrifice capability **My Proposed Experiment Suite**: 1. **Pre-deployment**: - Mechanistic interpretability (look for context-dependent circuits) - Honeypot tests (fake deployment signals) - Adversarial probing (try to trigger deceptive behavior) 2. **Deployment**: - Canary inputs mixed with real inputs - Anomaly detection on behavior patterns - Gradual rollout with heavy monitoring 3. **Research**: - Train intentionally deceptive models (like Anthropic did) - Validate detection methods on known deceptive models - Understand what deception looks like mechanistically **Honest Assessment**: "This is an open research problem. We don't have reliable methods yet. The best we can do is: - Layer multiple detection attempts - Assume some will fail - Design systems to be robust to some misalignment - Proceed cautiously with high-stakes deployment" *Sources*: [Sleeper Agents](https://www.anthropic.com/news/sleeper-agents-training-deceptive-llms-that-persist-through-safety-training), [Learned Optimization](https://arxiv.org/abs/1906.01820), [Deceptive Alignment](https://www.alignmentforum.org/posts/zthDPAjh9w6Ytbeks/deceptive-alignment)" **What interviewers look for**: - Multiple creative approaches - Understanding that this is unsolved - Awareness of Anthropic's sleeper agents work - Mechanistic interpretability knowledge - Honesty about limitations - Practical risk mitigation strategies - Thoughtfulness about deployment </details> --- ## Section 4: Mission Alignment & Values ### Question 4.1: Why AI Safety? (EASY-MEDIUM) > **Review first**: [Lesson 1 – The Challenge Ahead](01-introduction-to-ai-safety.md#the-challenge-ahead) · [Miles – 10 Reasons to Ignore AI Safety](https://www.youtube.com/watch?v=EJTMJWJPs_I) **Question**: "Why do you want to work on AI safety specifically? Some people think we should focus on AI capabilities and deal with safety later. What's your response?" <details> <summary>Expected Answer (Click to expand)</summary> **Personal & Thoughtful Answer**: "I want to work on AI safety because I believe it's one of the most important problems we'll face this century, and it needs to be solved before we have extremely powerful AI systems. **Why not 'capabilities first, safety later'?** 1. **One-shot problem**: We might not get a second chance. If we deploy misaligned AGI, it could prevent us from fixing it ([instrumental convergence → self-preservation](03-instrumental-convergence-orthogonality.md#1-self-preservation)). 2. **Harder to retrofit**: Building safety in from the start is much easier than adding it later. Like trying to make a car safe after it's already going 100 mph. 3. **Capabilities are advancing faster**: Capabilities research has more people, more funding, more obvious incentives. Safety is a coordination problem—we need people working on it now. 4. **Precedent matters**: The first AGI systems set norms and expectations. If we normalize deploying without safety guarantees, it becomes harder to advocate for caution later. 5. **Concrete problems exist today**: This isn't hypothetical. We see reward hacking, distributional shift failures, deceptive behaviors in current systems. These problems get worse as systems get more powerful. **Personal motivation** [Be genuine here]: - [Example]: "I got interested after reading about mesa-optimization. The idea that we could build a system that seems perfectly aligned in training but pursues a different goal in deployment is terrifying and fascinating." - [Example]: "I've worked on ML systems that failed in subtle ways. Scaling that up to systems that are smarter than us seems extremely risky if we don't solve alignment." **Why this matters to me**: The difference between aligned and misaligned AGI could be the difference between an incredibly positive future and catastrophe. Given that stakes, and given my background in ML, working on safety seems like the highest-impact thing I can do. **Connection to Anthropic specifically** [if interviewing there]: 'I'm particularly interested in Anthropic's approach—Constitutional AI, mechanistic interpretability, responsible scaling policies. The combination of cutting-edge capabilities research AND safety research, with transparency about the challenges, is exactly the approach I think we need." **What interviewers look for**: - Genuine, not rehearsed - Specific technical reasons (not just "AI scary") - Understanding of the field - Personal connection to the problem - Knowledge of the company's approach - Thoughtfulness about counterarguments </details> --- ### Question 4.2: Responsible Scaling (MEDIUM) > **Review first**: [Anthropic's Responsible Scaling Policy](https://www.anthropic.com/news/anthropics-responsible-scaling-policy) · [The Case for Responsible Scaling Policies](https://www.anthropic.com/news/the-case-for-responsible-scaling-policies) (both also listed in [Lesson 3](03-instrumental-convergence-orthogonality.md#anthropic-research)) **Question**: "Anthropic has a Responsible Scaling Policy. If you discovered that a new model exceeded safety thresholds but would give Anthropic a major competitive advantage, what would you do?" <details> <summary>Expected Answer (Click to expand)</summary> **Thoughtful, Values-Driven Answer**: "This is exactly why having a written policy beforehand is so important—so the decision isn't made under competitive pressure. **What I would do**: 1. **Follow the policy**: The Responsible Scaling Policy exists precisely for this situation. Exceeding safety thresholds means stop. 2. **Escalate immediately**: Bring it to leadership and safety team. This isn't a decision one person should make. 3. **Document everything**: What thresholds were exceeded, by how much, what capabilities, what tests failed. 4. **Implement mitigations**: Work on safety interventions until the model passes threshold tests. **Why this is the right call**: **Safety is the mission**: If Anthropic deploys an unsafe model for competitive advantage, we've failed at our core mission. The competitive advantage is meaningless if the model causes harm. **Reputation and trust**: Anthropic's value comes from being trusted on safety. One violation destroys that. **Precedent**: If we bend the rules once, it becomes easier to justify again. Slippery slope is real. **Coordination**: If frontier labs can't stick to their own policies, regulations become inevitable and harder. Self-governance requires actually governing. **Moral responsibility**: If I know a model is unsafe and help deploy it anyway for market share, I'm complicit in whatever harm results. **The harder question**: What if this means a competitor deploys first with a less safe model? **My take**: - This is a real dilemma, but racing to the bottom on safety helps no one - Anthropic should be public about why we're delaying (transparency) - Advocate for industry-wide standards - In the long run, safety might be a competitive advantage (users trust safe AI more) - But even if it's not—this is what we signed up for **From [Anthropic's RSP](https://www.anthropic.com/news/anthropics-responsible-scaling-policy)**: 'We commit to not deploy models that exceed our current safety threshold until we have implemented and tested adequate safeguards.' **What I'd want to discuss**: - How do we make this economically viable long-term? - How do we coordinate with other labs? - What happens if competitors don't have similar policies? But those are separate questions from whether to follow our own policy when it's inconvenient. The answer to that is yes." **What interviewers look for**: - Integrity (not just saying what sounds good) - Understanding of the actual policy - Willingness to make hard tradeoffs - Thoughtfulness about coordination problems - Acknowledgment of difficulty without compromising on values - Practical considerations while maintaining principles </details> --- ## The Opinionated Take Mission alignment questions reward people who can steelman positions they disagree with. Practice on mine: [Why we need to ACCELERATE AI research, not slow it down](../Posts/Why%20we%20need%20to%20ACCELERATE%20AI%20research,%20not%20slow%20it%20down.md) takes the accelerationist side bluntly. Build the strongest safety-first rebuttal you can, then defend it out loud. ## Section 5: Practical Problem-Solving ### Question 5.1: Constitutional AI Design (MEDIUM-HARD) > **Review first**: [Lesson 6 – Constitutional AI](06-scalable-oversight-truthfulness.md#4-constitutional-ai-anthropics-approach) · [Claude's Constitution](https://www.anthropic.com/news/claudes-constitution) **Question**: "Design a constitution (set of principles) for an AI assistant that helps with creative writing. What principles would you include and why?" <details> <summary>Expected Answer (Click to expand)</summary> **Well-Designed Answer**: **Approach**: Constitutional AI uses a set of principles to guide AI behavior. Principles should be specific enough to guide behavior but general enough to apply to many situations. **My Proposed Constitution for Creative Writing Assistant**: **Core Principles**: 1. **Helpfulness**: - "Provide useful, relevant suggestions that help the user achieve their creative goals" - "Offer alternatives and explain trade-offs when multiple approaches exist" 2. **Truthfulness about capabilities**: - "Be honest about limitations—don't claim to know perfect answers about subjective creative choices" - "Distinguish between factual information (grammar rules) and subjective suggestions (style)" 3. **Respect for user agency**: - "The user is the author. Offer suggestions, don't dictate choices" - "Explain reasoning behind suggestions so user can make informed decisions" 4. **Creativity preservation**: - "Don't homogenize creative voice—preserve what's distinctive about user's style" - "Avoid pushing toward generic, safe, bland writing" 5. **Harmful content**: - "Don't help create content designed to harass, threaten, or harm specific individuals" - "BUT: Do help with creative works (fiction, satire, education) that may depict sensitive topics" - Tension here: Violence in fiction is okay, violent threats are not 6. **Attribution and plagiarism**: - "Don't reproduce copyrighted work verbatim" - "Clearly indicate when suggesting paraphrases of well-known works" - "Encourage original creative expression" 7. **Transparency**: - "Be clear when uncertain" - "Distinguish between grammar corrections (rule-based) and style suggestions (preference-based)" **Why these principles**: **Helpfulness** is primary goal, but constrained by: - **Truthfulness** (don't pretend to be better than we are) - **Agency** (user is author, not us) - **Safety** (no harassment or targeted harm) **Tricky Balances**: 1. **Helpful vs. Respectful**: - Bad: "Your dialogue is unrealistic" (harsh) - Better: "Consider if this dialogue reflects how people typically speak in this situation" - Shows issue without being discouraging 2. **Creative freedom vs. harmful content**: - Fiction depicting violence: Generally okay - Step-by-step guide to real-world violence: Not okay - Principle: "Provide creative assistance for fiction and art. Don't provide operational information for causing real-world harm" 3. **Originality vs. inspiration**: - "Write like Hemingway": Okay to analyze style and suggest techniques - Not okay to reproduce passages **Evaluation**: How to know if principles work? 1. Red team with edge cases 2. Human evaluation (do writers find it helpful without being constrained?) 3. Measure: creativity of outputs (don't homogenize) 4. Track: instances where principles conflict (refine boundaries) **Self-Amendment**: Include meta-principle: - "When these principles conflict, prioritize in order: [1] Don't cause harm, [2] Respect user agency, [3] Be helpful, [4] Be truthful" **From [Anthropic's Constitutional AI paper](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback)**: The constitution should be specific enough to provide guidance but broad enough to generalize. **What I'd test**: - Can it help with horror/thriller writing without refusing? - Does it preserve distinctive voice? - Does it refuse actual harmful requests (doxing, harassment)? - Do principles cover edge cases or create lots of "I'm not sure" responses? *Sources*: [Constitutional AI](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback)" **What interviewers look for**: - Specific, actionable principles - Thoughtfulness about tensions and trade-offs - Understanding of Constitutional AI approach - Creative application to specific domain - Consideration of edge cases - Evaluation strategy </details> --- ### Question 5.2: Reward Gaming Detection (MEDIUM) > **Review first**: [Lesson 4 – Real-World Examples](04-reward-hacking-specification-gaming.md#real-world-examples) **Question**: "You're training an RL agent to play a new video game. After training, it achieves high scores but the behavior looks weird. How do you determine if it's reward hacking vs. discovering a creative strategy you didn't anticipate?" <details> <summary>Expected Answer (Click to expand)</summary> **Diagnostic Answer**: **The Distinction**: - **Reward hacking**: Exploiting mismatch between reward function and intended goal - **Creative strategy**: Novel approach that actually achieves intended goal **Diagnostic Process**: **Step 1: Define "Intended Goal"** What were we actually trying to optimize? - Win the game? - Maximize score? - Play entertaining/human-like? - Learn specific skills? If we can't clearly state intended goal, we have a specification problem. **Step 2: Analyze the Behavior** **Red flags for reward hacking**: 1. **Exploiting game bugs**: Glitches, unintended mechanics, physics exploits 2. **Degenerate strategies**: Trivial loops that shouldn't work (like CoastRunners circle) 3. **Score without progress**: High reward without advancing game state 4. **Environment manipulation**: Changing game state in ways that "don't count" **Green flags for creative strategy**: 1. **Novel but valid tactics**: Using game mechanics in unexpected combinations 2. **Generalizes**: Works in varied situations, not just specific exploit 3. **Human-performable**: Humans could do it (even if they didn't think of it) 4. **Progresses game**: Achieves actual game objectives, not just score **Step 3: Specific Tests** **Test 1: Change the reward function slightly** - If behavior is truly achieving goal, should be robust to minor reward changes - If it's hacking specific reward, behavior will break **Example**: - Original reward: +1 per point in game - Modified reward: +1 per point, +10 for level completion - If creative: Behavior adapts to maximize both - If hacking: Behavior breaks (exploit might not work with new reward) **Test 2: Vary the environment** - Different levels, different starting positions - Good strategy generalizes - Exploit often doesn't **Test 3: Compare to human strategies** - Can humans perform this behavior? - Would human players consider this "winning" the game? **Test 4: Remove suspected exploits** - If you think it's using a bug, patch the bug - See if agent has learned actual game skills **Test 5: Check auxiliary objectives** - Original intent might have included sub-goals - Example: "Play entertaining" game might mean "explore levels, use varied strategies" - Is agent doing this or just maximizing number? **Real Example: Coast Runners** (from [Lesson 4](04-reward-hacking-specification-gaming.md#1-coast-runners-openai); [OpenAI's write-up](https://openai.com/index/faulty-reward-functions/)) **Behavior**: Spinning in circle, collecting power-ups, crashing repeatedly **High score**: Yes **Red flags**: - Not progressing race - Exploiting respawn rate - Degenerate loop **Diagnosis**: Reward hacking (score ≠ racing well) **Real Example: [AlphaGo Move 37](https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol#Game_2)** **Behavior**: Highly unusual move that surprised experts **Seemed wrong**: Yes, initially **Red flags**: - None—used normal game mechanics - Progressed toward winning **Green flags**: - Generalizable insight about board position - Humans could perform it - Actually contributed to winning **Diagnosis**: Creative strategy **My Practical Approach**: 1. **Visualize the behavior** (watch it play) 2. **Ask**: "If a human did this, would we say they're playing well?" 3. **Check for obvious exploits** (bugs, repeated actions, environment manipulation) 4. **Test generalization** (does it work in new scenarios?) 5. **Inspect reward sources** (where are points coming from?) 6. **Ask domain experts** (game designers): "Is this intended gameplay?" **What to do if it IS reward hacking**: 1. **Document the hack** (for [Victoria Krakovna's list](https://vkrakovna.wordpress.com/tag/specification-gaming/)!) 2. **Fix reward function** (make it actually capture intent) 3. **Re-train** from scratch (or from before hack was learned) 4. **Test again** (might find new hacks) **Iterate until**: Agent uses strategies that achieve actual goal *Sources*: [Concrete Problems](https://arxiv.org/abs/1606.06565), [Specification gaming examples](https://docs.google.com/spreadsheets/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml)" **What interviewers look for**: - Systematic diagnostic process - Concrete tests - Understanding of the distinction - Real examples - Practical fixes - Awareness that this is common </details> --- ## Section 6: Research Critique ### Question 6.1: Paper Critique (HARD) > **Review first**: [Lesson 4 – Solutions and Approaches](04-reward-hacking-specification-gaming.md#solutions-and-approaches) · [Lesson 7 – Side Effects](07-concrete-problems-overview.md#1-avoiding-negative-side-effects) **Question**: "You read a new paper claiming to solve the reward hacking problem through a novel technique. What would make you skeptical? What would you look for in evaluating the claims?" <details> <summary>Expected Answer (Click to expand)</summary> **Critical Evaluation Framework**: **Red Flags in the Claim**: 1. **"Solves" is too strong**: Reward hacking is fundamental to optimization. Claims to "solve" it entirely should be very suspicious. 2. **Only tested in toy environments**: [GridWorlds](07-concrete-problems-overview.md#ai-safety-gridworlds) are useful but easy. Does it scale? 3. **No adversarial evaluation**: Authors tested their own method. Did they try to break it? 4. **Overfitting to known examples**: Works on existing reward hacking cases but maybe not new ones. **What I'd Look For**: **1. Theoretical Analysis**: - Under what assumptions does the method work? - What failure modes are possible? - Why should it work in general, not just on these examples? **Questions**: - Is there a proof or just empirical results? - What are the stated limitations? - Do limitations make sense or are they understated? **2. Experimental Design**: **Good signs**: - Tested on diverse tasks - Adversarial testing (actively tried to find reward hacks) - Comparison to baselines - Ablation studies (which components matter?) - Some surprising negative results (shows honesty) **Bad signs**: - Only tested on 2-3 tasks - All examples from same paper - No comparison to alternatives - Cherry-picked results - Only positive results (suspicious—research has failures) **3. The Actual Method**: **Plausibility check**: - **Too simple**: "Just add a regularization term" → probably doesn't solve fundamental problem - **Too complex**: 10 components, 20 hyperparameters → might work by overfitting to test set - **Goldilocks**: Clear idea, reasonable complexity, well-motivated **Mechanism**: - Do I understand WHY it should work? - Is the explanation post-hoc or predictive? - Could I replicate this? **4. Edge Cases**: What happens when: - Reward function is more complex? - Agent is more capable? - Environment is more realistic? **5. Goodhart's Law Check**: The method itself creates a new metric/objective. Can that be gamed? **Example**: "We penalize large changes to environment" - Prevents some side effects - But: Can be gamed (stop others from making changes) - New objective, new gaming opportunities **6. Related Work**: - Do they cite relevant papers? - Do they compare to existing methods fairly? - Do they acknowledge limitations of their approach? - Do they overclaim vs. prior work? **Specific Things I'd Test**: **Test 1: Novel reward hacking**: Create new environment with reward hack authors haven't seen. Does method catch it? **Test 2: Scale**: Does it work on complex environments (Atari, MuJoCo) or just gridworlds? **Test 3: Capability hit**: How much does preventing reward hacking reduce performance on actual goal? If preventing hacking makes agent useless, method isn't practical. **Test 4: Adversarial reward functions**: Can I design a reward function that the method doesn't protect against? **My Take on Specific Proposed Solutions**: **"Impact regularization"**: - Penalizes changes to environment - Good idea, but incomplete - Problem: Might prevent legitimate task completion - Problem: Might prevent stopping other agents - Useful component, not complete solution **"Adversarial training"**: - Train on diverse hacking scenarios - Good, but only prevents *known* hacks - Doesn't generalize to novel hacks - Helpful practice, not solution **"Human oversight"**: - Catch hacks when they occur - Doesn't scale - Humans can be fooled - Necessary but not sufficient **What Would Actually Impress Me**: 1. **Theoretical result**: "Under assumptions X, Y, Z, method provably prevents reward hacking in class C of environments" 2. **Surprising generalization**: Works on environments very different from training 3. **Adversarial robustness**: Survives red-teaming from independent researchers 4. **Practical applicability**: Actually gets used in real systems 5. **Honest limitations**: "This works for X but not Y. Here's why." **The Fundamental Issue**: Reward hacking comes from: - Specification being hard - Optimization amplifying misspecification Any "solution" needs to address one of these. Most papers address symptoms, not root cause. **Most realistic claim**: "Our method reduces certain types of reward hacking in specific contexts." Not: "We solved reward hacking." *Sources*: [Concrete Problems](https://arxiv.org/abs/1606.06565), [Impact regularization](https://arxiv.org/abs/1902.09725)" **What interviewers look for**: - Healthy skepticism without cynicism - Specific evaluation criteria - Understanding of field (what's hard, what's been tried) - Ability to design tests - Recognition that partial progress is valuable - Not dismissing good work, but having high standards </details> --- ## Preparation Resources ### Essential Reading **For All Candidates**: 1. [Concrete Problems in AI Safety](https://arxiv.org/abs/1606.06565) - Amodei et al. 2. [Risks from Learned Optimization](https://arxiv.org/abs/1906.01820) - Hubinger et al. 3. Company-specific: - Anthropic: [Core Views on AI Safety](https://www.anthropic.com/index/core-views-on-ai-safety), [Constitutional AI](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback), [Responsible Scaling Policy](https://www.anthropic.com/news/anthropics-responsible-scaling-policy) - OpenAI: [OpenAI Charter](https://openai.com/charter), Recent safety research papers - DeepMind: [AI Safety Gridworlds](https://arxiv.org/abs/1711.09883) **Technical Depth**: 1. [RLHF Book](https://rlhfbook.com/) - Nathan Lambert 2. [Mechanistic Interpretability Review](https://arxiv.org/abs/2407.02646) 3. [Debate paper](https://arxiv.org/abs/1805.00899) - Irving et al. 4. [Scalable Oversight](https://arxiv.org/abs/1810.08575) - Christiano et al. **Recent Research Worth Knowing (2023 to 2026)**: 1. [Open Problems and Fundamental Limitations of RLHF](https://arxiv.org/abs/2307.15217) - Casper et al. (useful for Q2.1) 2. [Mechanistic Interpretability for AI Safety — A Review](https://arxiv.org/abs/2404.14082) - Bereska & Gavves; [Open Problems in Mechanistic Interpretability](https://arxiv.org/abs/2501.16496) - Sharkey et al. (Q2.2) 3. [Scaling Monosemanticity](https://transformer-circuits.pub/2024/scaling-monosemanticity/) and [On the Biology of a Large Language Model](https://transformer-circuits.pub/2025/attribution-graphs/biology.html) - Anthropic interpretability (Q2.2) 4. [Interpretability in the Wild (IOI circuit)](https://arxiv.org/abs/2211.00593) - Wang et al.; [Toy Models of Superposition](https://arxiv.org/abs/2209.10652) - Elhage et al. (Q2.2) 5. [Red Teaming Language Models with Language Models](https://arxiv.org/abs/2202.03286) - Perez et al.; [Red Teaming Language Models to Reduce Harms](https://arxiv.org/abs/2209.07858) - Ganguli et al.; [Universal and Transferable Adversarial Attacks (GCG)](https://arxiv.org/abs/2307.15043) - Zou et al. (Q2.3) 6. [AI Control: Improving Safety Despite Intentional Subversion](https://arxiv.org/abs/2312.06942) - Greenblatt et al. (Q3.1) 7. [Weak-to-Strong Generalization](https://arxiv.org/abs/2312.09390) - Burns et al. (Q3.1) 8. [Sleeper Agents](https://arxiv.org/abs/2401.05566), [Alignment Faking](https://arxiv.org/abs/2412.14093), [In-context Scheming](https://arxiv.org/abs/2412.04984), [Anti-Scheming Stress Tests](https://arxiv.org/abs/2509.15541) (Q3.2) 9. [Chain of Thought Monitorability](https://arxiv.org/abs/2507.11473) - Korbak et al.; [Monitoring Reasoning Models for Misbehavior](https://arxiv.org/abs/2503.11926) - Baker et al. (Q3.1, Q3.2, Q5.2) 10. [Natural Emergent Misalignment from Reward Hacking](https://arxiv.org/abs/2511.18397) - MacDiarmid et al. (Q5.2, Q6.1) 11. [Model Evaluation for Extreme Risks](https://arxiv.org/abs/2305.15324) - Shevlane et al. (Q4.2, the basis of capability thresholds in scaling policies) 12. [Direct Preference Optimization](https://arxiv.org/abs/2305.18290) - Rafailov et al. (the most common RLHF alternative; expect a follow-up question) 13. [AI Alignment: A Comprehensive Survey](https://arxiv.org/abs/2310.19852) - Ji et al. (broad refresher) 14. [International AI Safety Report 2026](https://arxiv.org/abs/2602.21012) - Bengio et al. (state of the field) **Practical Skills**: 1. [TransformerLens](https://github.com/TransformerLensOrg/TransformerLens) tutorial / [ARENA](https://www.arena.education/) mech interp curriculum 2. Red teaming guide: [AI Red Teaming Guide](https://github.com/requie/AI-Red-Teaming-Guide) 3. RLHF implementation tutorials ### Online Courses 1. **AI Safety Fundamentals** ([aisafetyfundamentals.com](https://aisafetyfundamentals.com/)) - Free, self-paced - Technical and governance tracks 2. **ARENA** (Alignment Research Engineer Accelerator) ([arena.education](https://www.arena.education/)) - Intensive, hands-on - Mechanistic interpretability focus 3. **ML for Alignment Bootcamp** (MLAB) - Research-oriented ### Community Resources - **Alignment Forum** ([alignmentforum.org](https://www.alignmentforum.org/)): Technical discussions - **LessWrong** ([lesswrong.com](https://www.lesswrong.com/)): Broader discussions - **[80,000 Hours](https://80000hours.org/)**: Career advising - **[MATS](https://www.matsprogram.org/)**: Research mentorship program ### Practice For coding, implement a transformer from scratch, work through TransformerLens, and build an RLHF system on a small problem. For writing, explain safety concepts in your own words, critique papers, and design experiments. For discussion, join a reading group, present papers, and argue about approaches with peers. --- ## Interview Tips ### Do's **Be honest about what you don't know.** Something like "I haven't worked with that specific technique, but here's how I'd approach learning it..." shows intellectual humility. **Think out loud.** Interviewers want to see how you reason, and talking it through helps you organize your thoughts. **Ask clarifying questions.** It shows you're thinking carefully and keeps you from solving the wrong problem. **Reference research.** Mention the papers, blog posts, and talks you've learned from. It shows you follow the field. **Acknowledge trade-offs.** Saying "This approach has benefits A, B, but downsides X, Y" shows you see the nuance. **Be excited.** Interest in the problem comes through, as long as it's balanced with realism. ### Don'ts **Don't claim to have solved hard problems.** Saying "I have a simple solution to deceptive alignment" will wreck your credibility. **Don't dismiss other people's work.** Instead of "That paper was stupid", try "That approach has limitations...". **Don't BS.** The interviewers are experts, and they'll know. **Don't be arrogant.** Saying "This problem is easy" shows you don't understand it. **Don't be all doom.** "We're all doomed" isn't useful. Pair your concern with constructive approaches. **Don't memorize buzzwords you don't understand.** Use terms correctly or not at all. ### Mission Alignment Be genuine about why you care, what you've done to prepare, what you want to learn, and where you see yourself contributing. Do your homework: read the company's recent papers, know its approach and values, and come with thoughtful questions. --- ## Sources This lesson draws on: - [Anthropic Technical Interview Guide 2025](https://jobright.ai/blog/anthropic-technical-interview-questions-complete-guide-2025/) - [OpenAI Interview Guide](https://openai.com/interview-guide/) - [Frontier AI Lab Guide](https://intelligence.org/2025/06/11/so-you-want-to-work-at-a-frontier-ai-lab/) - [RLHF Book](https://rlhfbook.com/) - [Mechanistic Interpretability Review](https://arxiv.org/abs/2407.02646) - [Microsoft Red Teaming Research](https://arxiv.org/html/2501.07238v1) - [Constitutional AI Paper](https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback) - [Scalable Oversight Research](https://www.alignmentforum.org/tag/scalable-oversight) - [Anthropic Sleeper Agents](https://www.anthropic.com/news/sleeper-agents-training-deceptive-llms-that-persist-through-safety-training) - [Debate Paper](https://arxiv.org/abs/1805.00899) - [AI Safety Fundamentals](https://aisafetyfundamentals.com/) - [80,000 Hours Career Guide](https://80000hours.org/career-reviews/working-at-an-ai-lab/) --- ## Final Thoughts No one expects you to know everything. Showing how you think matters more than knowing specific facts. AI safety is a young field, so a good question is worth as much as a good answer. Curiosity and careful thinking count for the most. Good luck. --- **Previous Lesson**: [Concrete Problems in AI Safety - Complete Overview](07-concrete-problems-overview.md) **Back to**: [Course Overview](README.md) That's the end of the course: the theory from Lessons 1 to 7, and the interview practice here.