Angela Shi writes about reconstructing missing Tables of Contents in PDFs by analyzing body typography to facilitate better Retrieval-Augmented Generation. When native outlines are absent, the approach uses six deterministic signals—including font size ratios and numeric prefixes—to identify potential headings. This method then employs an LLM validation loop to filter false positives from these candidates before integrating them into a structured data frame for RAG pipelines.
- Scoring is based on variables like boldness, alignment, text length, and blank lines above segments.
- The approach includes modes for extending partial outlines or reconciling composite documents with re-starting numbering.
- LLM validation can improve precision from as low as 7% to over 90% in certain document sets.
- A secondary "tagging" layer allows thematic retrieval by mapping specific business concepts onto the structural hierarchy.
- Scoring is based on variables like boldness, alignment, text length, and blank lines above segments.
- The approach includes modes for extending partial outlines or reconciling composite documents with re-starting numbering.
- LLM validation can improve precision from as low as 7% to over 90% in certain document sets.
- A secondary "tagging" layer allows thematic retrieval by mapping specific business concepts onto the structural hierarchy.
