The article discusses the intrinsic representation of errors, or hallucinations, in large language models (LLMs). It highlights that LLMs' internal states encode truthfulness information, which can be leveraged for error detection. The study reveals that error detectors may not generalize across datasets, implying that truthfulness encoding is multifaceted. Additionally, the research shows that internal representations can predict the types of errors the model is likely to make, and that there can be discrepancies between LLMs' internal encoding and external behavior.
