LLM Wikis And Hierarchical Structure - Source Excerpt 05 - Prioritized references and limitations

Back to LLM Wikis And Hierarchical Structure

Summary

This source excerpt begins near Prioritized references and limitations and preserves the surrounding evidence from Wiki.FFTAC.org/agent-file-handoff/Archive/2026-05-11-improvement-concept-layer/LLM Wikis and Hierarchical Structure.md.

**Source path:** Wiki.FFTAC.org/agent-file-handoff/Archive/2026-05-11-improvement-concept-layer/LLM Wikis and Hierarchical Structure.md

For a **flat wiki** with page-level search but little structure, the shortest safe path is to: inventory page titles, redirects, categories, templates, and assets; mint immutable IDs for pages and sections; preserve heading paths; convert categories into a governed concept vocabulary rather than loose tags; and add embeddings only after the page/section model is stable. If the source resembles MediaWiki, reuse categories, templates, infoboxes, and section anchors rather than discarding them during ingestion. citeturn24view5turn24view7turn24view8turn24view9turn31view0turn31view4

For a **vector-only store**, the main challenge is that much of the lost authorial structure must be reconstructed. The safest order is to group chunks back into documents by source metadata, infer section boundaries where possible, attach parent-child relationships, then enrich with aliases, concepts, entities, and ACLs. Only after the hierarchy exists should you add graph edges or summary layers. LangChain’s retrieval guidance is helpful here because it explicitly notes that if you already have a knowledge base, you do not need to rebuild it from scratch; rather, you layer retrieval capabilities over existing data contracts. citeturn27view1turn25view13turn25view12turn23view11

A pragmatic rollout pattern is dual-running: keep the old wiki or old vector index live, build the new hierarchy and indexes in parallel, compare retrieval traces and citation quality, then cut over query paths gradually. Permission-aware retrievers or ACL filters should be wired in before user-facing cutover, not after, because retrofitting security into search results is much harder than indexing with security metadata from the start. citeturn27view5turn23view3

' ' ' mermaid
timeline
    title Suggested migration phases
    Inventory and identity : Crawl pages, assets, redirects, categories, and existing vector metadata
    Structural normalization : Mint immutable IDs; recover page, section, and chunk hierarchy
    Metadata enrichment : Add aliases, concepts, entities, provenance, and ACL attributes
    Dual indexing : Build lexical, vector, and optional graph indexes in parallel
    Shadow evaluation : Compare recall, latency, citations, and access-control behavior
    Progressive cutover : Route selected query classes to the new stack
    Cleanup and governance : Deprecate old indexes; add vocabulary and review workflows
' ' ' 

| Starting point | Minimal viable target | Higher-maturity target |
|---|---|---|
| Flat wiki | Document tree with section-aware chunking, aliases, ACLs, lexical + vector retrieval | Add concept lattice, claim sidecar, and summary/community layer |
| Vector-only store | Recovered document tree with parent-child chunks and source-level provenance | Add concepts, entity graph, citation-aware answer generation, and policy-aware retrieval |
| Notes graph or PKM vault | Keep note graph plus normalize properties, tags, and links into explicit metadata | Add collections, concept governance, and shared editorial workflows |

## Prioritized references and limitations

The sources below are the ones I would prioritize if you were actually implementing or reviewing an LLM Wiki architecture. They are weighted toward official documentation and original papers.

| Priority | Reference | Why it matters |
|---|---|---|
| High | **Lewis et al., _Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks_** | The canonical statement of why external non-parametric memory matters for LLMs. citeturn25view0 |
| High | **Sarthi et al., _RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval_** | Best single paper on hierarchical summary trees for long-context retrieval. citeturn25view1 |
| High | **Microsoft GraphRAG docs and paper** | Best-known practical graph-and-summary retrieval stack for global corpus questions. citeturn25view2turn25view3 |
| High | **Gruber, _A Translation Approach to Portable Ontology Specifications_** | Still the cleanest conceptual grounding for ontology as a shared representational vocabulary. citeturn33view0 |
| High | **W3C OWL 2 Primer** | Primary source for formal typed ontology design. citeturn25view4 |
| High | **W3C SKOS Reference** | Primary source for lightweight taxonomies, thesauri, and migration from older vocabularies. citeturn25view5 |
| High | **Wikidata Help: Statements, Qualifiers, Sources, Items** | Most concrete production pattern for typed claims, qualifiers, references, aliases, and stable IDs. citeturn24view3turn24view4turn31view2turn31view3turn31view4 |
| High | **Wikipedia / MediaWiki help on categories, infoboxes, TOC, and transclusion** | Best real-world evidence for polyhierarchy, structural summaries, and reusable template-driven knowledge. citeturn24view5turn24view6turn24view7turn24view8turn24view9turn31view0 |
| High | **Faiss paper and HNSW paper** | Core references for vector-index latency and ANN trade-offs. citeturn25view6turn25view7 |
| Medium | **Notion docs on data sources, relations, backlinks, permissions** | Strong modern model for combining pages, structured tables, and workspace permissions. citeturn24view12turn23view1turn23view2turn23view3 |
| Medium | **Obsidian help on links, properties, backlinks, and Bases** | Strong evidence for local-first linked notes plus structured metadata and database-like views. citeturn23view0turn23view4turn23view5turn23view6turn23view7 |
| Medium | **LangChain docs on retrieval, loaders, vector stores, and structure-aware chunking** | Useful framework-level reference for ingestion and retrieval layering. citeturn27view1turn25view13turn27view0turn25view12 |
| Medium | **LlamaIndex framework docs** | Useful framework-level reference for indexes, nodes, graph stores, citations, and multi-modal context augmentation. citeturn26view0turn19search14turn18search3turn26view6 |
| Medium | **Weaviate hybrid search docs; Neo4j vector index docs; PostgreSQL JSONB and pgvector docs** | Strong practical references for storage and hybrid retrieval implementation choices. citeturn25view10turn25view9turn25view11turn29view0 |
| Medium | **Perplexity docs on citations, contextualized embeddings, and media attachments; Kagi docs on references, Lenses, and document Q&A** | Useful retrieval-UX references for modern source-aware knowledge interfaces. citeturn23view10turn23view11turn29view2turn23view9turn29view4turn29view6 |

There are a few limitations worth stating plainly. Roam’s publicly available formal schema documentation is sparse, so its treatment here is intentionally high-level. Kagi and Perplexity are better understood as **retrieval interfaces and answer engines** than as canonical collaborative wiki backends, so the comparison focuses on their retrieval and citation patterns rather than their internal storage models. And because product documentation changes faster than academic papers, the platform portions of this survey should be treated as current snapshots rather than permanent architectural guarantees. citeturn15search0turn23view8turn29view3turn23view13turn26view0