AI Fundamentals Interview Questions

How LLMs Work

Q: Explain how Large Language Models work.

LLMs are transformer neural networks trained to predict the next token. Components:

Inference is autoregressive: generate one token, append to input, repeat.

Self-attention intuition:

Each token creates Query (Q), Key (K), Value (V) vectors. Attention score = Q*K^T / sqrt(d). Softmax -> weights -> weighted sum of V. Allows "bank" to mean financial or geographical based on surrounding context.

Q: Key parameters — temperature, top-p, context window.

Temperature (0-2): Controls randomness.

Top-p (0-1): Only consider tokens comprising top p% of probability mass.

Context window: Max tokens (input + output). GPT-4o = 128K (~95K words). Claude 3 = 200K.

Q: What is RAG and why use it?

RAG (Retrieval Augmented Generation) grounds LLM answers in retrieved documents.

Solves:

Pipeline: Query -> embed -> vector DB similarity search -> top-k chunks -> LLM + context -> answer

Q: Fine-tuning vs prompting vs RAG — when to use each?

|---|---|---|

Always try prompting first, then RAG, then fine-tuning as a last resort.

Revision Notes

LLM: Transformer trained to predict next token (autoregressive)
Self-attention: Q*K^T/sqrt(d) -> weights -> weighted sum of V
Captures context: same word, different meaning based on surrounding tokens

PARAMETERS:
Temperature: 0=deterministic, 1=balanced, 2=creative
Top-p: nucleus sampling (consider top p% probability tokens)
Context window: total tokens in/out (GPT-4o=128K)

RAG: vector DB retrieval + LLM generation
Solves: cutoff, hallucination, private data
Pipeline: query -> embed -> search -> chunks -> LLM

DECISION ORDER:
1. Prompting (free) -> 2. RAG (cheap) -> 3. Fine-tuning (expensive)