Training loss (smoothed)
–
nats per answer token · – average probability of the correct next word
How to read the loss. It measures how surprised the model is by each correct word of the reference
answers (cross-entropy, natural log ⇒ "nats"); lower is better. e−loss = the average probability it gives the
correct next word: 3.0 ≈ 5 % · 2.0 ≈ 14 % · 1.5 ≈ 22 % · 1.0 ≈ 37 % · 0.5 ≈ 61 % · 0.2 ≈ 82 %.
Working zones of this lab (SP answers): above 2.0 the model does not know SP yet · 1.0–2.0 it knows SP's language,
its facts are unreliable · target: 1.0 or lower, with train ≈ eval · train below 0.2, or far below eval, = it is
memorising the wording (over-fitting). The model passes on the fact-recall probe and the test set, not on the loss alone.