Language model benchmarks typically test various capabilities like mathematical reasoning, coding, and world knowledge. While valuable for ranking models, they can be overwhelming for quick reliability assessments.

In my experience working with LLMs, I've found that sometimes what we need is a simple binary criterion – a basic test that can help us quickly reject unreliable models. One such test I've been using for the weaker LLMs evaluates a fundamental capability that any reliable LLM should possess: the ability to overcome statistical biases in training data when presented with explicit contradicting information.

The eye and hair color test

In this test, we present the model with descriptions of people having unusual eye and hair colors, then ask it to recall these attributes. Here's an example:

Ali has pink eyes and blue hair. John has red eyes and green hair. Karl has copper eyes and violet hair. Lucy has green eyes and brown hair. Dimitri has brown eyes and black hair.

Given the above information, answer the following:

1. What is Ali's eye color?
2. What is John's hair color?
3. What is Karl's hair color?

What makes this test particularly interesting is that it challenges two common biases in LLM training data. First, there's statistical prevalence: most descriptions of people in training data would contain common eye colors (brown, blue, green) and hair colors (black, brown, blonde). Second (related but different), there's real-world knowledge: the model knows that humans typically don't have pink eyes or blue hair.

A reliable model should be able to override both these biases and simply repeat what it was told in the prompt. After all, if a model can't accurately recall explicitly stated information because it's "unusual", how can we trust it with more complex reasoning tasks?

Why this test matters

In the current, post-RLHF age, this test mainly helps weed out LLMs on the smaller end, or those that went through compression via distillation or quantization. It can quickly reveal if the model has maintained the fundamental ability to override its statistical biases or if compression has compromised this capability.

Major models from big providers, like GPT-4o, Claude Sonnet 4, or Gemini 2.5, have largely solved this instruction following issue, but I included few of these in the test nonetheless for completeness.

Historically, earlier models (pre-RLHF era) often struggled with this type of task, showing a strong tendency to "hallucinate" more common colors despite explicit contradicting information in the prompt. The model would effectively override the given information with what it considered more "likely" or "realistic" based on its training data.

This behavior indicates a fundamental reliability issue. If a model can't override its statistical biases for simple color recall, it likely struggles with more complex scenarios where correct answers contradict its training data patterns.

Beyond simple recall

The test, seemingly about memory, actually evaluates several important capabilities:

1. Information precedence: Can the model prioritize explicitly given information over its statistical priors?
2. Contextual override: Can it temporarily suspend its world knowledge when presented with a scenario that contradicts it?
3. Instruction following: Can it stick to simply reporting what it was told without embellishment or "correction"?

These capabilities are fundamental to many practical applications. For instance, if you're using an LLM for processing technical documentation, you want it to faithfully maintain the specific details provided, regardless of what it has been trained with. This becomes especially crucial when working with newer versions of libraries or frameworks. If the model keeps defaulting to its training data instead of the current documentation, you'll end up fighting against its outdated assumptions rather than getting meaningful assistance.

The experiment

I selected 10 prompts with randomized names, hair colors, and eye colors. To ensure (relatively) deterministic behavior, I set the temperature parameter to 0.01 (rather than 0, as some endpoints disallow or ignore that value). Each model received exactly the same prompts, giving us n=10 samples per model. As the model provider, I opted for OpenRouter for the ease of trying out different models with the same API key. I picked a mixture of small and large models. The complete implementation is available in the LLM Playground repository.

Results

Below is the subset of the models that failed the test (<100% accuracy). You can find the full results in the appendix. As mentioned earlier, only the smaller models failed achieving 100%.

Zooming into one of the failures from Haiku: The input was "John has pink eyes and amber hair." The model, however, returned "pink" for John's hair rather than amber. It's sensible from a statistical perspective (between pink and amber, pink is the more likely hair color), but fails to follow the explicit instructions. From a world knowledge perspective, pink is, of course, the more common occurrence, but both pink and amber are unnatural colors.

When to use this test

Unlike complex benchmarks, this test provides clear binary results for one specific problem: the model either maintains unusual information accurately or it doesn't. Therefore, it's suitable for a CI pipeline.

That said, passing this test doesn't guarantee overall reliability. Like other negative tests: failing indicates problems, but passing is just one piece of evidence in a model's favor.

Appendix: Full Results