18 quick cards, about 12 minutes each: say your answers out loud before you open them.
First · 5 min
Goldman Sachs: what to know walking in
Using AI in the interview: No outside tools, ChatGPT or search included, in HireVue or live interviews. The one sanctioned exception is the engineering HackerRank 'Software Diagnostics' challenge, which has a built-in, limited AI assistant you're expected to use with your own judgment.
The loop
- HireVue video interview: Motivation, communication
- HackerRank assessment: DSA, debugging, math
- Live technical screen: Working code, explanation
- ML and quantitative rounds: ML theory, probability, stats
- Superday: Consistency, team fit
The full Goldman Sachs page1·Coding·Must know·12 min
Python for ML interviews
How would you compute pairwise distances between two sets of vectors without a Python loop?
Show the key points
- Broadcast: reshape A to (n, 1, d) and B to (1, m, d), subtract, square, sum over the last axis, then take the square root.
- Broadcasting aligns shapes from the right, and each dimension must be equal or 1, so (3, 2) plus (2,) gives (3, 2).
- For large inputs use ‖a‖² + ‖b‖² − 2a·b with one matrix multiply, avoiding the n by m by d intermediate array.
Which built-in data structures do you reach for under pressure, and why?
Show the key points
- Dicts and sets give average O(1) lookups, so membership checks and counting avoid scanning a list each time.
- Use collections.deque for queues and BFS, since popleft is O(1) while list.pop(0) shifts every remaining element.
- Counter, defaultdict and heapq cover counting, grouping and top-k, and comprehensions read better than index loops.
With more time (45 min): NumPy: the absolute basics for beginners (NumPy documentation). Read “Array fundamentals” through “Broadcasting”, then “More useful array operations”, “Transposing and reshaping a matrix” and “Working with mathematical formulas”.
2·Coding·Must know·12 min
Calling LLM APIs in Python: async, streaming, retries
How would you call an LLM for 500 prompts concurrently while respecting a rate limit?
Show the key points
- Wrap each call in a coroutine, fan out with asyncio.TaskGroup or gather, and cap in-flight requests with an asyncio.Semaphore.
- Put asyncio.timeout around each call, and retry 429s and 5xx errors with exponential backoff plus jitter so clients don’t retry in lockstep.
- TaskGroup cancels the remaining tasks when one raises; gather with return_exceptions=True returns failures alongside results instead.
Why can a single blocking call inside a coroutine ruin your throughput?
Show the key points
- asyncio runs cooperative tasks on one thread, and a task only yields at an await, so a blocking call freezes every other task.
- Common culprits are time.sleep, synchronous HTTP clients and heavy CPU work; use async libraries or move the work out with asyncio.to_thread.
- asyncio beats threads for many I/O-bound calls because tasks are cheap, but it gives no speedup for CPU-bound work.
With more time (40 min): Coroutines and tasks (Python documentation). Read “Coroutines”, “Awaitables”, “Creating tasks”, “Task groups”, “Running tasks concurrently”, “Timeouts” and “Running in threads”.
3·Coding·Should know·12 min
DSA patterns AI roles still ask
Answer out loud:
- Name the six High-priority topics and say which pattern each usually tests
- List the corner cases you check before coding any array or string problem
- Explain why a hash map is the first data structure to consider for an O(n) solution
- State a problem's time and space complexity out loud before writing code
Show the key points
Most Applied AI and LLM engineer loops still include at least one general coding round, but it is rarely a hard dynamic-programming puzzle; it leans on a small set of patterns: array and string scans, hash maps for counting and lookup, two pointers and sliding windows, BFS and DFS over grids and graphs, and heaps for top-k. Interviewers want to see you name the pattern, state the complexity, handle empty and single-element inputs, and write clean Python without hesitation. This topic is about recognising which pattern a prompt maps to and having a template for each one ready, not grinding hundreds of problems.
With more time (15 min): Data structures and algorithms study cheatsheets for coding interviews (Tech Interview Handbook). Read “Contents of each study guide”, “Study guides list” (note the High / Mid / Low priority tags) and “General interview tips”. Then open the High-priority Array, String and Graph guides plus the Mid-priority Hash Table and Heap guides from the list.
4·Coding·Should know·12 min
ML coding: implementing an LLM block in PyTorch
Answer out loud:
- Writes RMSNorm, RoPE, GQA and SwiGLU from memory with correct shapes
- Builds the causal mask and shows a future token can’t change earlier outputs
- Treats heads as a batch dimension instead of looping over them
- Finishes a full pre-norm block with residuals in about 30 minutes
Show the key points
The ML-coding round asks you to write a working piece of a transformer in PyTorch in about 30 minutes, usually a Llama-style decoder block: RMSNorm, rotary position embeddings, grouped-query attention, a SwiGLU feed-forward layer and pre-norm residuals. Interviewers want to see that you think in tensor shapes, can apply a causal mask correctly and can prove your code works with a quick test. A strong answer states the shape at every step, keeps heads as a batch dimension rather than looping, checks that changing a future token leaves earlier outputs unchanged, and explains why each Llama change replaced its GPT-2 counterpart.
With more time (90 min): TorchCode (Hugging Face Spaces (GitHub: duoan/TorchCode)). Open the Space (it starts JupyterLab in the browser, or use the Colab badge on any problem) and do these in order, timing each: #8 RMSNorm, #9 Causal Self-Attention, #10 Grouped Query Attention, #24 RoPE, #15 SwiGLU MLP, then #13 GPT-2 Block as the full-block rehearsal. Try every one without hints first.
5·Coding·Good to know·12 min
SQL and pandas for data rounds
Answer out loud:
- Write an inner, left and full join in both SQL and pd.merge, and say what happens to rows with duplicate or null keys in each
- Explain the difference between size() and count() in a groupby, and when each matches SQL COUNT
- Produce the top-n rows per group in both SQL (window function) and pandas (groupby + rank or head)
- Reproduce UNION vs UNION ALL with concat and drop_duplicates
Show the key points
Many Applied AI loops include a data round: a CSV or two tables, and you have to join, group, rank and filter them live, either in SQL or in pandas. The questions are predictable: pick the right join and explain what happens to unmatched or duplicate keys, aggregate per group, and use window functions (row_number, rank, lag/lead, running totals) for top-n-per-group or change-over-time questions. Interviewers also check that you can translate between the two worlds, since eval pipelines and dataset curation are usually pandas while analytics lives in SQL. This topic covers the join/groupby/window core in both and the gotchas (null keys, cartesian blow-ups, rank ties) that trip people up.
With more time (35 min): Comparison with SQL (pandas documentation). Read “GROUP BY”, “JOIN” (all four join subsections) and “pandas equivalents for some SQL analytic and aggregate functions” (both “Top n rows” subsections).
6·Coding·Good to know·12 min
Classic ML from scratch: logistic regression and k-means
Answer out loud:
- Writes a sigmoid that doesn’t overflow for large negative inputs
- Clips probabilities before taking the log in the loss
- Gradient is Xᵀ(p − y)/n with no Python loop over samples
- Can derive the gradient on a whiteboard
Show the key points
A NumPy drill: write logistic regression and k-means without a library. Interviewers use it to check that you can turn the maths into vectorised code and spot numerical traps. For logistic regression, a strong answer writes a sigmoid that won’t overflow, derives the binary cross-entropy gradient as Xᵀ(p − y)/n, and runs the update loop. For k-means, it picks a sensible initialisation (random points or k-means++), alternates assignment and mean updates, handles an empty cluster, and stops when the centroids move less than a tolerance.
With more time (40 min): Train Logistic Regression with Gradient Descent (Deep-ML). Solve it in NumPy in under 30 minutes without the solution: sigmoid, loss per iteration, gradient, update. Then solve #17 (K-Means Clustering) the same way.
7·System design·Must know·12 min
RAG vs fine-tuning vs prompting: choosing the approach
Would you use RAG or fine-tuning here, and how do you decide?
Show the key points
- Ask what’s missing: if the model lacks facts or the facts change, use retrieval; if it behaves wrongly in format, tone or niche syntax, fine-tune.
- Escalate in order: prompt with an eval set first, add retrieval next, and fine-tune only once cheaper options are shown to fall short.
- Weigh data volume, freshness, the need for citations, latency and cost; retrieval gives citations and updates without retraining.
When would you combine RAG and fine-tuning, and what does each part contribute?
Show the key points
- Retrieval supplies current, citable facts at query time, while fine-tuning teaches steady behaviour such as format, tone or how to use retrieved context.
- A typical case is a domain assistant tuned on specialised format or jargon that still retrieves the latest documents for each answer.
- Fine-tuning is an unreliable way to add facts and needs retraining when they change, so keep knowledge in retrieval and behaviour in the weights.
With more time (45 min): Optimizing LLM Accuracy (OpenAI). Read “LLM optimization context”, “Prompt engineering”, “Evaluation” and “Understanding the tools” (the RAG, Fine-tuning and “All of the above” parts). Then read “How much accuracy is ‘good enough’ for production”.
8·System design·Must know·12 min
GenAI system design
How would you design an LLM-powered product end to end, from request to response?
Show the key points
- Start from requirements and a bare model call, then add context construction, guardrails, routing, caching and tools only as each solves a stated problem.
- Put numbers on latency, cost per request and quality early, and first ask whether the feature needs an LLM at all.
- Check inputs and outputs, retry or fall back to a human on failure, and log traces and user feedback from day one to drive iteration.
When does semantic caching help, and when does it do more harm than good?
Show the key points
- Prompt caching reuses a computed prefix and exact caching reuses answers to identical queries; semantic caching reuses answers for queries with similar embeddings.
- It pays off for high-volume, repetitive questions with stable answers, like FAQs, turning a model call into a fast lookup.
- It hurts when small wording changes meaning or answers depend on the user or fresh data, so tune the similarity threshold and scope caches per user.
With more time (50 min): Building A Generative AI Platform (huyenchip.com). Read “Step 1. Enhance Context” through “Traces” (Steps 1–5 plus “Observability”), skimming “RAGs” since you've covered retrieval already.
9·System design·Must know·12 min
Design enterprise document search / a knowledge assistant
How do you make sure the assistant never shows a user a document they can’t access?
Show the key points
- Mirror each source’s permissions into the index alongside every chunk, using connectors that sync access lists as well as content.
- Filter by the user’s identity and groups at retrieval time, before anything reaches the model, so it never sees what the user can’t open.
- Keep permissions fresh with webhooks or frequent syncs, since a revoked share must vanish from results quickly, and test with users of differing access.
How would you design retrieval and evaluation for search across wikis, drives, tickets and chat?
Show the key points
- Use hybrid retrieval, lexical plus vector search, then a reranker, because company data is full of names, IDs and jargon that embeddings can miss.
- Return clickable citations to source passages so users can check answers, and set a latency target that limits how heavy reranking can be.
- Evaluate answer quality plus source precision and recall on a labelled set of real employee questions, and watch freshness as documents change.
With more time (35 min): Building Dash: How RAG and AI agents help us meet the needs of businesses (Dropbox Tech Blog). Read “Challenges in making an AI product that’s ready for businesses” (data diversity, fragmentation, modalities), then “Leveraging retrieval-augmented generation” including “Choosing the right retrieval system” and “Choosing the right model”, and “Lessons learned and future direction”.
10·ML fundamentals·Must know·12 min
Embeddings basics: word2vec, contrastive learning, what embeddings capture
How does word2vec with negative sampling learn word embeddings?
Show the key points
- Skipgram pairs each centre word with words in its window, training the model to predict which words appear nearby.
- Negative sampling swaps the full-vocabulary softmax for binary classification: real neighbour pairs versus a few randomly sampled words.
- It learns a word matrix and a context matrix and usually keeps the word one; window size shifts what similarity the vectors capture.
How are modern sentence embedders trained, and why do negatives and batch size matter?
Show the key points
- Contrastive objectives like InfoNCE pull a query and its matching passage together and push other passages in the batch apart.
- Larger batches give more in-batch negatives, and hard negatives that look similar but are wrong teach finer distinctions than random ones.
- Unlike static word vectors, which give an ambiguous word like “bank” one vector, these models encode meaning from the whole context.
With more time (40 min): The Illustrated Word2vec (jalammar.github.io). Read “Word Embeddings” through “Window Size and Number of Negative Samples”; the sections “Skipgram”, “Negative Sampling” and “Skipgram with Negative Sampling (SGNS)” are the core.
11·AI coding round·Should know·12 min
The AI-enabled coding round
Answer out loud:
- Name the four areas this round is evaluated on
- Describe how you'd orient yourself in an unfamiliar multi-file codebase in 5 minutes
- Explain how you verify AI-written code before moving on
Show the key points
Some companies now let you use an AI assistant during a coding interview, usually in a multi-file codebase rather than a single function. You're judged on how you orient yourself in unfamiliar code, plan, steer the AI, verify its output and keep talking to the interviewer. Policies differ: some companies require AI in this round, others ban it in every live interview.
With more time (15 min): Introduction to AI-Enabled Coding Interviews (Hello Interview). Read the introduction and the Fundamentals list: codebase orientation, planning, driving the AI, verification, communication.
12·Transformers·Must know·12 min
Self-attention from scratch
Why do we divide attention scores by the square root of the key dimension?
Show the key points
- With unit-variance entries, a query-key dot product has variance d_k, so raw scores spread wider as the dimension grows.
- Large scores push softmax towards a near one-hot output, where gradients become tiny and training stalls.
- Dividing by √d_k brings score variance back to about one, keeping softmax in a range with useful gradients.
How would you implement multi-head causal self-attention, and what does it cost in memory?
Show the key points
- Project the input into Q, K and V, split into h heads, compute softmax(QKᵀ/√d_k)V per head, then concatenate and project back.
- For the causal mask, set scores above the diagonal to minus infinity before softmax, and subtract the row max for numerical stability.
- Each head builds a seq_len by seq_len score matrix, so memory and compute grow as O(n²) in sequence length.
With more time (30 min): Implement Self-Attention Mechanism (Deep-ML). Solve it in NumPy in under 30 minutes without looking at the solution. Then try #107 (masked self-attention).
13·Evals·Must know·12 min
Evals and error analysis
Why should you read real traces before writing any LLM judge?
Show the key points
- Reading traces shows what actually fails; categories you imagine up front rarely match the failures users actually hit.
- Group the failures into categories and count them, so you fix the most frequent or costly ones first.
- Each category then gets a targeted check, often a cheap code assertion, so you only build judges for what code can’t test.
How would you structure evals for an LLM feature, from unit tests to A/B tests?
Show the key points
- Level one is assertion-style unit tests run on every change, such as valid JSON, required fields present or no leaked internal IDs.
- Level two is human and model evaluation of logged traces on a regular cadence, with the LLM judge kept aligned to human labels.
- Level three is A/B testing with real users once the product is mature, checking whether changes move the outcomes that matter.
With more time (25 min): Your AI Product Needs Evals (hamel.dev). Read “The Types Of Evaluation” (unit tests, human & model eval, A/B testing) using the Lucy case study.
14·RAG·Must know·12 min
Embeddings and vector search
When would you use cosine similarity versus dot product for vector search?
Show the key points
- Cosine compares direction only and ignores vector length; dot product also rewards magnitude, which some models are trained to use.
- On L2-normalised vectors cosine and dot product give identical rankings, so you can normalise once and use the cheaper dot product.
- Use the metric the embedding model was trained with; mismatching it quietly hurts retrieval quality without raising any error.
How does HNSW find nearest neighbours fast, and which knobs trade recall for latency?
Show the key points
- HNSW builds a layered graph: sparse upper layers for long hops, dense lower layers for fine search, walked greedily from the top.
- M sets links per node, trading memory and build time for recall; ef_construction sets how carefully the graph is built.
- ef_search sets the candidate list size at query time: higher gives better recall but slower queries, tuned against exact search results.
With more time (40 min): Embeddings: What they are and why they matter (simonwillison.net). Read “What are embeddings?” through “Vibes-based search”, then “Answering questions with Retrieval-Augmented Generation”.
15·Agents·Must know·12 min
Tool use and function calling
Walk me through what happens, step by step, when a model calls a tool?
Show the key points
- You send tool definitions with a name, description and JSON schema; the model replies with a structured call naming the tool and arguments, not prose.
- Your application, not the model, runs the function, then sends the output back as a tool result linked to that call’s ID.
- The loop repeats until the model answers in text; tool definitions are extra input tokens on every request, so they cost context and steer behaviour.
How do you design tools so the model uses them correctly and recovers when they fail?
Show the key points
- Keep tools few and clearly named, with descriptions saying when to use each, and shape inputs to resist misuse, such as enums or absolute paths.
- Use strict mode so arguments always match the schema, and tool_choice to let the model decide, require a call, force one tool, or disable tools.
- Return errors as a tool result with a clear message the model can act on, and cap retries so a broken tool can’t loop forever.
With more time (40 min): Function calling (OpenAI API docs). Read “How it works” (including “The tool calling flow”), “Function tool example”, “Defining functions”, “Best practices for defining functions”, “Token Usage”, “Handling function calls”, and “Additional configurations” (“Tool choice”, “Parallel function calling”, “Strict mode”).
16·Inference·Must know·12 min
Decoding: greedy, beam, temperature, top-k/top-p
What do temperature and top-p actually do to the next-token distribution?
Show the key points
- Temperature divides the logits before softmax: below 1 sharpens towards the top token, above 1 flattens it, and near 0 approaches greedy.
- Top-k keeps a fixed number of the likeliest tokens; top-p keeps the smallest set whose probabilities sum to p, then renormalises.
- Top-p adapts to confidence: when the model is sure the pool is tiny, and when it’s unsure the pool widens.
Which decoding settings would you pick for extraction versus creative writing, and why?
Show the key points
- For extraction, use greedy or a temperature near 0 so the output stays consistent and close to the most likely answer.
- For creative text, sample with moderate temperature and top-p, since greedy and beam search tend to repeat themselves on open-ended text.
- Even at temperature 0 outputs can vary, because GPU floating-point sums depend on batching and kernels, so near-tied tokens can flip.
With more time (30 min): How to generate text: using different decoding methods for language generation with Transformers (Hugging Face blog). Read “Greedy Search” through “Top-p (nucleus) sampling”, then the “Conclusion”.
17·Behavioral·Must know·12 min
Behavioral interviews: STAR stories for AI engineers
Tell me about a time an LLM feature you shipped went wrong?
Show the key points
- Use STAR in about two minutes: brief situation and task, most of the time on your actions, then a measurable result and what you learned.
- Say “I” for what you personally did, like spotting the regression, building the eval or rolling back; save “we” for team context.
- Own the mistake plainly, show how you found and fixed it, and end with the process change that stops it happening again.
Tell me about a time you disagreed with a stakeholder about a technical decision?
Show the key points
- Pick a real disagreement with stakes, such as pushing back on a model choice or a feature the model couldn’t yet do reliably.
- Show how you used evidence, like an eval result or a cost estimate, to make your case while taking their goals seriously.
- End with the outcome and the working relationship afterwards; avoid stories where you were simply right and the other person looks foolish.
With more time (40 min): Behavioral interviews for Software Engineers: How to prepare (Tech Interview Handbook). Read “Learn the STAR answer format” through “Prepare experiences to showcase fit to the company's culture / core values”, then draft your story list as you go.
18·Behavioral·Must know·12 min
Story bank
Tell me about a time you had to work through an ambiguous problem?
Show the key points
- Draw on a prepared story that covers scope, your contribution, a measured impact and the main difficulty, rather than improvising one.
- Explain how you reduced the ambiguity: the questions you asked, a small experiment or eval you ran, and the decision it led to.
- Keep your actions in “I” terms and attach a number, such as latency, an eval score, users reached or time saved.
What alternatives did you consider there, and why did you reject them?
Show the key points
- Interviewers often ask this after a couple of follow-ups, so write each story’s trade-offs and rejected options down in advance.
- Name a specific alternative and the evidence or constraint that ruled it out, such as cost, latency, eval results or the deadline.
- Admit what you’d do differently now; a candid answer holds up under probing better than a story polished to sound flawless.
With more time (60 min): Learnings from conducting ~1,000 interviews at Amazon (The Pragmatic Engineer). Read “Learning #1” to “Learning #3”, then the book excerpt, especially “The Four Dimensions That Determine Your Level” (scope, contribution, impact, difficulty). Then pick six to eight real projects, and for each write one paragraph covering those four dimensions, with your own part in “I” and at least one number. Tag each story with the classic prompts it can answer: a failure, a conflict, disagreeing with someone senior, a mistake you owned, a tight deadline, ambiguity. Record yourself telling two of them aloud.
Last · 15 min
The last 15 minutes
- Stop studying. Eat something, drink water, and go to the bathroom.
- Open the interview link, and check your camera, microphone and internet. Close everything else.
- Say your 60–90 second “tell me about yourself” out loud once.
- Have two or three questions ready for the interviewer.
- In the interview: restate each question before answering, and think out loud when you’re stuck.
- Follow Goldman Sachs’s rule on AI tools, above.