Building A Prophecy Tracking Knowledge Graph Entities, Claims, Sources, Contradictions, And Review Status 2 - Source Excerpt 02 - Data Ingestion and Extraction
Summary
This source excerpt begins near Data Ingestion and Extraction and preserves the surrounding evidence from Antichrist.net/agent-file-handoff/Archive/2026-05-12-content-reports/Building a prophecy tracking knowledge graph entities, claims, sources, contradictions, and review status 2.md.
**Source path:** Antichrist.net/agent-file-handoff/Archive/2026-05-12-content-reports/Building a prophecy tracking knowledge graph entities, claims, sources, contradictions, and review status 2.md
- **authored_by**: `(Prophecy)-[:AUTHORED_BY]->(Prophet)` (each prophecy has exactly one prophet). Similarly, a Source can have `authored_by` links to authors (people/orgs).
- **predicts**: `(Prophecy)-[:PREDICTS]->(Event)`. One prophecy can predict multiple events (e.g. “two kings will come” = two events) and an event might be predicted by multiple prophecies (many-to-many).
- **claims_about**: `(Claim)-[:CLAIMS_ABOUT]->(Event/Location/Topic)`. A claim often refers to one or more target events or entities.
- **cites**: `(Claim)-[:CITES]->(Source)` or `(Source)-[:CITES]->(Source)` (document citations). This links claims to their source documents, or links a prophecy to earlier texts it references.
- **contradicts** / **corroborates**: Binary relations between Claim or Prophecy nodes. For example, `(ClaimA)-[:CONTRADICTS]->(ClaimB)` means ClaimA negates ClaimB. These should be mutual or directional as needed, and can carry a `confidence` or `certainty` property. Likewise, `(ClaimA)-[:CORROBORATES]->(ClaimB)` for supporting claims. Cardinality: many-to-many (a claim can contradict many others).
- **reviewed_by**: `(Entity)-[:REVIEWED_BY]->(Reviewer)` where Entity could be a Claim or Prophecy. Each review action might create a node (Review) or just a timestamped property on the relationship. Status taxonomy (below) can be a property on the entity or review edge.
- **updated_to**: `(EntityOld)-[:UPDATED_TO]->(EntityNew)` linking one version of a Claim/Prophecy to its next version. This is typically one-to-one (version chain).
**Schema considerations:** The ontology may reuse Schema.org classes (e.g. `ClaimReview`, `Claim`, `Rating`) as in fact-checking datasets【20†L126-L134】. For instance, one can model a Claim as `schema:Claim` with `author`, `datePublished`, and link it to a `ClaimReview`. We should define which fields are required (e.g. every Claim must have text and a source, every Prophecy must have a prophet and date) and which are optional. Versioning can be handled by adding `version` numbers or by the `updated_to` edges. In RDF/OWL, we would declare classes (Prophecy, Claim, etc.) and properties (e.g. `ex:predicts`, `ex:contradicts`), with cardinality constraints. In a property graph (Neo4j), we enforce constraints through application logic or database schema (unique constraints on IDs, etc.).
## Data Ingestion and Extraction
Data sources include **primary prophetic texts** (e.g. scriptures, books, recordings), **media reports** (news articles, transcripts, blog posts), **social media content** (tweets, videos), and **academic papers**. Ingestion methods: web scraping (RSS feeds, news APIs), crawling archives, social media APIs, and document processing (PDF/text ingestion). For physical or image-only documents, apply OCR (e.g. Tesseract) to extract text【54†L300-L309】.
Once text is obtained, an NLP pipeline extracts structured data:
- **Preprocessing:** Clean and normalize text (lowercasing, removing boilerplate).
- **Coreference Resolution:** Link pronouns to entities (using AllenNLP or spaCy coref models) so that references (“he”, “this”) correctly map to prophets or subjects【64†L279-L286】【58†L53-L62】.
- **Named Entity Recognition (NER):** Identify names of prophets, places, events, dates with spaCy or a transformer model. E.g. “Nostradamus” → Prophet, “Paris” → Location.
- **Entity Linking:** Map entities to knowledge bases (Wikidata/DBpedia) for disambiguation and external context (as in ClaimsKG linking to DBpedia)【16†L25-L30】.
- **Temporal normalization:** Convert relative date phrases (“next month”, “Year of the Dragon”) to absolute dates. Use libraries (HeidelTime, SUTime) or custom rules.
- **Claim/Relation Extraction:** Identify factual assertions. Tools like Hugging Face BERT models or spaCy rule-based extractors can isolate candidate claims【37†L48-L57】【58†L53-L62】. The “Claim Extraction” benchmark (FEVERFact dataset) uses generation or NER approaches to produce atomic claims【58†L53-L62】. Each extracted claim is turned into a `Claim` node with properties (text, date, source link) and connected to relevant entities.
- **Interpretation tagging:** If human analysts are involved, they may add Interpretation notes to Prophecy nodes, stored as text with confidence scores.
- **Provenance capture:** For every ingested item, record source metadata (source ID, fetch date, extractor version). This is stored as properties on the node or via a `provenance` edge to a Source node【16†L25-L30】【54†L300-L309】.
Finally, the extracted entities/claims are upserted into the graph: existing nodes are merged by unique ID or name, new nodes/edges created accordingly. This pipeline can be automated and scheduled (e.g. daily news ingestion), and retrained as new data arrives.
【66†embed_image】 *Figure: Example information extraction pipeline*. Text input goes through coreference resolution, entity linking, and relation extraction steps to populate the knowledge graph【64†L279-L286】【37†L48-L57】. In practice, we may use spaCy for NER, Hugging Face transformers for claim extraction, and Tesseract for OCR【37†L39-L45】【54†L300-L309】.
## Contradiction and Corroboration Detection
Detecting contradictions among claims is crucial. We define a contradiction as one claim logically negating another. Algorithms include:
- **Rule-based checks:** Compare structured facts (dates, numbers). E.g., if Claim A predicts “2025” and Claim B says “2025 will not occur”, flag a contradiction.
- **Embedding/NLI:** Use sentence embeddings or Natural Language Inference models (e.g. a fine-tuned BERT) to classify claim pairs as *contradictory*, *neutral*, or *entailment*. For instance, models like “all-MiniLM-L6-v2-nli” on HuggingFace can score semantic contradiction【38†L0-L4】.
- **Negation generation:** As in one case study, generate explicit negations of assertions (rule-based or via prompting) and check if any actual statement matches it【37†L48-L57】.
- **Evidence lookup:** Retrieve external evidence (other claims or factual knowledge bases). If two claims conflict with authoritative data (e.g. timeline facts), mark as contradiction. This hybrid approach is advised by practitioners【40†L149-L157】【37†L60-L68】.
- **Temporal consistency:** Ensure that events referenced by multiple claims have coherent timelines (e.g. not a prophecy predicting an event before the prophecy was made).
Thresholds and uncertainty: Because these methods can err, we assign each `contradicts` or `corroborates` edge a confidence score. The system uses a high threshold for auto-flagging contradictions (favoring precision) and routes ambiguous pairs for human review【40†L147-L155】. For example, a contradiction classifier might have high precision but lower recall (so flagged contradictions are reliable but some conflicts may be missed)【56†L1-L4】. We explicitly record that uncertainty (e.g. an edge property `confidence=0.92`). Corroboration (supporting evidence) is similarly tracked.
Graph representation: Contradiction/corroboration edges link two Claim (or Prophecy) nodes. We might also log *which pieces of text* led to the conflict, as metadata or via separate “Explanation” nodes, to make the graph explainable. As RedBuffer’s pipeline noted, structuring negations and linking evidence provides an audit trail for reviewers【37†L48-L57】【37†L60-L68】.
## Review Workflow and Status
Each claim or prophecy node carries a **review status** property. Proposed statuses:
- `unreviewed` (newly ingested),
- `under_review` (assigned to a reviewer),
- `verified` (confirmed true or likely true),
- `debunked` (determined false),
- `disputed` (conflicting evaluations exist).
A *Review* action is modeled by linking the entity to a Reviewer node: `(Claim)-[:REVIEWED_BY {date, status}]->(Reviewer)`. The Reviewer (a Person) node has `role` = “fact-checker” or similar. All status changes are timestamped. This audit trail ensures transparency: we know *who* reviewed *what* and *when*.
Reviewer roles can be tiered (e.g. volunteer, editor, expert). Workflows should support delegation: an `under_review` claim may be picked up by any qualified reviewer. UI considerations: interfaces should allow searching/filtering by status, displaying provenance text and any contradictory edges, and capturing reviewer notes. We should offer explainable views (highlighting terms that triggered contradictions) and history logs. High-severity items (e.g. possibly defamatory claims) could be auto-flagged for senior review.
This mirrors fact-check systems where each claim has a “claim review” entry (e.g. Schema.org ClaimReview with a `reviewRating`【20†L126-L134】). We adapt that idea to our internal workflow taxonomy, ensuring consistency and traceability at every step.
## Storage and Query