Lesson 10 Quiz — Evaluating Foundation Model Performance (10 easy + 10 hard)
20 questions.
Q1easymcq
Why don't precision/recall/F1 work well for evaluating a chatbot's free-form generated reply?
Q2easymcq
Which metric is commonly used for summarization quality, measuring overlap with a reference text?
Q3easymcq
Which metric was originally built for machine translation quality, measuring n-gram overlap?
Q4easymcq
Which metric uses embeddings to measure semantic similarity rather than exact word overlap?
Q5easymcq
What is 'LLM-as-a-judge'?
Q6easymcq
Which AWS service runs automatic and human evaluation workflows directly against Bedrock models?
Q7easymcq
Which evaluation approach involves real people directly reviewing model outputs?
Q8easymcq
Which business-objective metric measures whether a user actually accomplished what they came to do?
Q9easymcq
Standardized test sets with known reference answers used to score/compare models are called:
Q10easymcq
Cost per interaction as an evaluation dimension connects most directly back to which earlier lesson's concept?
Q11hardmcq
A generated summary uses entirely different wording than the reference summary but conveys the same meaning. Which metric is most likely to still score it well?
Q12hardmulti
Which of these are automatic generation-quality metrics discussed in this lesson?
Select all that apply.
Q13hardmcq
A model scores excellently on ROUGE/BLEU but users report low engagement and rarely complete their intended task. What does this scenario illustrate?
Q14hardordering
Order these evaluation approaches from most automated/scalable to most manual/nuanced.
1.Human-in-the-loop evaluation
2.LLM-as-a-judge
3.Automatic metric (e.g. ROUGE/BLEU/BERTScore)
Q15hardmatching
Match each metric/approach to what it best captures.
ROUGE/BLEU
BERTScore
LLM-as-a-judge
Human-in-the-loop
Q16hardmcq
A team needs to compare two candidate Bedrock models on the same benchmark task with minimal custom tooling. Best fit?
Q17hardmcq
Why is precision/recall a poor fit for scoring a summarization model's output quality?
Q18hardmulti
Which of these are legitimate business-objective evaluation dimensions from this lesson?
Select all that apply.
Q19hardmcq
A company wants a much cheaper alternative to full human review, but more nuanced than raw n-gram overlap metrics. Best fit?
Q20hardmcq
Which statement correctly distinguishes technical vs. business evaluation layers for GenAI applications?