Generative AI Interview Questions and Answers
Explain the model, then defend the application: practical questions with examples, tradeoffs and failure cases.
A strong Generative AI interview answer connects a concept to a design decision and a way to test it. Explain what the model can do, where it can fail and which parts of the application enforce correctness and access control.
Model foundations
1. How is Generative AI different from a classifier?
A classifier assigns a label or score, such as a support-ticket category. A generative model produces content, such as a reply. An application may combine both. Choose the output the task needs rather than assuming a text-generating model is always the right tool.
2. What does a token mean?
A token is a unit used by a model's tokenizer. It may be a word fragment, punctuation or another unit; it is not always a whole word. Token counts affect input limits and often cost. Count with the relevant tokenizer when the limit matters.
3. What does attention contribute to a transformer?
Attention computes relationships between representations so the model can combine relevant information. The original Transformer used attention rather than recurrence for sequence modelling. In an interview, explain the mechanism without treating attention weights as a complete explanation of a model's reasoning.
4. Is an embedding a summary?
An embedding is a numerical representation used for tasks such as similarity search. It is not a readable summary or a guarantee of semantic equivalence. Retrieval quality depends on the embedding model, the documents, chunking and the queries used to evaluate it.
Sources: Vaswani et al.: Attention Is All You Need. Checked 23 October 2026.
Retrieval and adaptation
5. How does retrieval-augmented generation work?
Retrieve relevant material, supply it as context and generate an answer using that evidence. Track the source of each passage so the answer can be checked. Retrieval does not guarantee a correct answer: the documents can be wrong, the search can miss evidence and generation can misinterpret it.
6. When would you choose RAG over fine-tuning?
Start with retrieval when answers need accessible, changing documents. Consider fine-tuning when examples can teach a recurring behaviour or format. They solve different problems and can be combined. Evaluate a simpler baseline before accepting the cost and maintenance of either.
7. How would you choose a chunk size?
Use the structure of the documents and the questions people ask. Compare candidate chunk sizes on retrieval relevance and final answer quality. Keep useful context such as headings or table labels. A fixed number copied from another project is a starting hypothesis, not evidence.
8. How do you prevent access-control leaks in retrieval?
Enforce document permissions before unauthorised content enters model context. Test with users who have different access. Do not rely on a prompt telling the model not to reveal a document it has already received. Recheck permissions when content or user roles change.
Sources: Lewis et al.: Retrieval-Augmented Generation. Checked 23 October 2026.
Quality, safety and operating constraints
9. What is a hallucination, and how do you reduce it?
An answer can be fluent while containing unsupported or false information. Use appropriate evidence, explicit task limits, evaluation and a refusal or escalation path when information is missing. Measure unsupported claims. None of these controls justifies promising that hallucinations are impossible.
10. How do you evaluate an LLM application?
Build representative cases with clear success criteria. Include ambiguous inputs, missing evidence and harmful instructions in documents. Assess correctness, evidence support and task completion alongside latency and cost. Compare versions on held-out cases and review disagreements rather than reporting only a single aggregate score.
11. What is prompt injection?
It is an attempt to make the system follow untrusted instructions, for example instructions embedded in a retrieved document. Treat retrieved text as data. Restrict tool permissions, validate arguments and require approval for consequential actions. A delimiter helps structure a prompt but is not an authorisation boundary.
12. What would you do when an AI API becomes unavailable?
Define a timeout and a user-visible failure state. Retry only where appropriate, with bounded backoff, and avoid duplicating actions. A fallback might be search results or a manual workflow rather than another model. Test recovery and state what the degraded service can still do.
Practise a document-answering design question
Suppose an internal assistant answers questions about travel policy. Begin by asking who may read which documents, how policy versions are approved, and what happens when two sources disagree. Propose a retrieval baseline before adding agents or fine-tuning.
Explain how you would test ordinary questions, outdated documents, inaccessible files and questions the policy cannot answer. Describe the data you would retain for debugging and how you would avoid recording unnecessary personal information. Finish with the limits of your design, not a claim that the model is always correct.
Common questions
Should interview answers include code?
Use code when it helps demonstrate an API, evaluation or retrieval decision. Be ready to explain the data flow and failure cases without relying on a particular framework.
Are these actual company interview questions?
These are original practice questions covering common engineering topics. They are not a leaked or verified question bank from an employer.
