Lawful Intelligence Collection And Reporting - Source Excerpt 03 - 8. Data Retention and Deletion Policies
Back to Lawful Intelligence Collection And Reporting
Summary
This source excerpt begins near 8. Data Retention and Deletion Policies and preserves the surrounding evidence from 2IA.org/agent-file-handoff/Archive/2026-05-17-civil-liberties-overhaul/Content/Lawful Intelligence Collection and Reporting.md.
**Source path:** 2IA.org/agent-file-handoff/Archive/2026-05-17-civil-liberties-overhaul/Content/Lawful Intelligence Collection and Reporting.md
## 8. Data Retention and Deletion Policies
Organizations must define how long intelligence data will be kept and when it will be securely deleted. Data retention policies should align with legal requirements and business needs. For example, the EU GDPR’s storage limitation principle requires that personal data “be kept… no longer than is necessary” for its processing purpose【45†L15-L22】. Similarly, the UK Information Commissioner advises establishing documented retention schedules that list each data category, its use, and retention period【44†L159-L168】. In practice, intelligence units often archive resolved cases for a defined period (e.g. 3–7 years) to meet legal or audit needs, then delete or archive data irreversibly. Sensitive data that is no longer needed should be expunged using secure deletion methods (e.g. cryptographic erasure, secure overwrite). All retention policies should include periodic reviews to purge outdated or unnecessary data.
## 9. Reporting Workflows (Tips and Contributions)
- **Intake and Validation:** Tips or raw intelligence contributions (from employees, citizens, third parties) should be collected via a standardized intake process. An intake form should capture essential fields (see Template). Upon receipt, an analyst or officer performs an initial triage: verify the tip’s authenticity, check for any immediate legal issues (e.g. is it disallowed information?), and log it into the system (assigning a tracking number).
- **Legal and Ethical Review:** Before further action, each new piece of intelligence should be screened for compliance. For example, if the tip includes personal data, confirm that collection/processing follows applicable laws. If needed, consult a legal or privacy officer at this stage. Only data from lawful sources (or sources with valid consent) should proceed.
- **Classification and Storage:** Once validated, the information is classified using the labeling scheme (e.g. assign TLP level, confidentiality tag) and stored securely (encrypted and with access controls). A detailed record is kept (who entered it, when, with a summary of the data). If forensic value is high, the chain-of-custody is continued.
- **Analysis and Sharing:** Authorized analysts are granted access to the stored intelligence for analysis. Depending on the content, they may produce formal reports. Dissemination follows the assigned labels: for instance, RED info only to specific individuals; GREEN info could be shared with partner organizations. When sharing externally (e.g. with law enforcement or other agencies), use secure channels (e.g. encrypted email or secure portals) and include only needed details.
- **Follow-up and Closure:** Outcomes of analyses (investigations, actions taken) should be documented in a reporting template. Contributors may be notified (with consent) about the result if appropriate. Finally, ensure all reports and tips are archived per the retention schedule.
## 10. Comparison of Recommended Tools
| Tool | Purpose | License | Security/Privacy | Cost | Platform |
|-----------------|---------------------------------|------------------|------------------------|------------------|-------------------|
| **OSINT Framework** (web) | Directory of OSINT resources | Open (web-based) | Depends on chosen tools; it’s just a portal | Free | Web |
| **Google / Advanced Search** | General web search (incl. Google Dorks) | Proprietary | Good security; complies with privacy policies| Free | Web |
| **Maltego** | Link analysis, entity mapping | Proprietary (free limited) | Cloud/desktop; vendor-managed; consider data residency | Free basic; paid from ~$€3000/year for advanced | Windows/Linux |
| **SpiderFoot** | Automated OSINT scans (domains, IPs) | Open-source (GPL) | Self-hosted (secure by design) | Free (Python); Enterprise paid| Windows/Linux |
| **Shodan** | Internet device search engine | Proprietary | Data from internet scans; HTTPS API | Free tier; paid plans ($49+/mo) | Web/API |
| **WHOIS / DNS Tools** | Domain registration/IP lookup | N/A (ICANN data) | Public data; generally privacy-respecting | Free | Web/CLI |
| **theHarvester**| Harvest emails, subdomains | Open-source (GPL) | Self-run tool; outputs public data | Free | Linux/Windows |
| **Signal** | Secure messaging / tips submission | Open-source (GPL) | End-to-end encryption; high privacy | Free | iOS/Android/Desktop |
| **VeraCrypt** | Disk encryption (storage) | Open-source (custom license) | Strong on-disk encryption (AES, etc.) | Free | Windows/Mac/Linux |
| **ProtonMail** | Secure email (reporting) | Proprietary (with open-core) | End-to-end encrypted mail | Free limited; paid plans | Web/iOS/Android |
| **TensorFlow / Python** | Custom data analysis (OSINT AI) | Open-source (Apache) | Depends on implementation; use responsibly| Free | Multi-platform |
| **Tableau/Power BI**| Visualization of intelligence data | Proprietary | Depends on deployment; internal servers recommended | Commercial | Windows/Cloud |
*Note:* “Security/Privacy” notes indicate trustworthiness and deployment considerations. Open-source and self-hosted tools (e.g. SpiderFoot, VeraCrypt) allow maximum control. Cloud services (Maltego Cloud, Shodan API) require checking provider security. Costs and platforms are current estimates (2026).
## 11. Process Flowcharts
The following flowcharts illustrate key workflows:
' ' ' mermaid
flowchart LR
subgraph Intake
A[[Tip or Data Submission]]
A --> B{Initial Review}
end
subgraph Validation
B --> C[Verify Source / Content]
C --> D[Legal/Privacy Check]
end
subgraph Storage
D --> E[Assign Classification/Label (e.g. TLP)]
E --> F[Encrypt and Store Data]
F --> G[Log Access & Track Chain-of-Custody]
end
subgraph Sharing
G --> H{Authorized Sharing?}
H -->|Yes| I[Share with Recipients (per label)]
H -->|No| J[Restrict Access]
end
' ' '
' ' ' mermaid
flowchart LR
subgraph Collection
A[Intelligence Collection] --> B{Is Data Public?}
B -- Yes --> C[Proceed under OSINT protocols]
B -- No --> D{Legal Authority?}
D -- No --> Z[Stop/Discard Data]
D -- Yes --> E[Obtain Warrants/Consent]
end
subgraph Compliance
C & E --> F[Data Privacy and Ethics Review]
F --> G{Meets Regulatory Requirements?}
G -- Yes --> H[Continue Processing]
G -- No --> Z[Halt & Remediate]
end
' ' '
These diagrams show that all intake undergoes review and labeling, with legal compliance checks before any analysis or sharing.
## 12. Templates for Intake and Reporting
**Intake Form (Tips and Leads):** A structured form (electronic or paper) to capture incoming intelligence. Example fields include:
| Field | Description |
|----------------------|-------------------------------------------------|
| Submission Date (UTC)| Date/time the tip was received (UTC). |
| Source Name / Org | Who provided the tip (individual or agency). |
| Contact Info | Phone/email (optional, for follow-up). |
| Tip Category | (E.g. cybersecurity, physical security, fraud). |
| Description of Info | Brief narrative of the information/tip. |
| Attachments/Links | URLs or files supporting the tip. |
| Suggested Classification | (Analyst fills, e.g. TLP level or “Confidential”). |
| Action Taken | (Internal use: investigator assigned, etc.) |
This form ensures uniform data capture. It should include a consent clause if personal data is collected.
**Reporting Template (Intelligence Report):** When an analyst produces findings, use a report format such as:
- **Report ID:** Unique identifier (auto-generated).
- **Date (UTC):** Report creation timestamp.
- **Summary:** One-paragraph overview of the intel.
- **Source(s):** References (e.g. tip IDs, public websites, archives).
- **Detail/Analysis:** Full description of findings, methods, and conclusions.
- **Attachments/Evidence:** Linked files or exhibits.
- **Classification:** E.g. “TLP:AMBER – Confidential”.
- **Recommended Actions:** Suggested next steps or alerts.
- **Prepared by:** Analyst name, team.
**Consent Notice:** Before accepting personal data, provide a clear notice. For example:
> *“By submitting this information, you consent to [Org Name] collecting and processing your data for [purpose, e.g. threat analysis] in accordance with our Privacy Policy. We will use your data only for legitimate security purposes. You have the right to access or request deletion of your data at any time.”*