Core51 Evidence Verifier — Documentation
=======================================

Last updated: 2026-04-22

This document describes the Core51 Evidence Verifier (Python) and the Core51 export formats.
It is written for: engineers, auditors, and legal/forensic workflows.

0) What the verifier is
-----------------------
The verifier is an offline-first tool that:

- validates the integrity of a chat export package (ZIP)
- checks internal consistency (hash-chain)
- checks message signatures (best-effort; strict where possible)
- decrypts v2 messages locally if you provide your encryption private key
- prints a readable transcript to stdout OR exports a readable package
- can download and decrypt attachments on demand (does not download everything)

Important: Core51 is end-to-end encrypted. The server stores ciphertext. Decryption happens locally.

1) Download
-----------

Verifier source code (Python):
- /downloads/core51-evidence-verifier-src.zip

After downloading:
- unzip the archive
- install requirements

2) Requirements
---------------

Python:
- Python 3.10+ recommended.

Dependencies:
- PyNaCl (for Ed25519 signature verification + NaCl box open)
- cryptography (for AES-GCM)

Install:

Windows:
  python -m pip install -r requirements.txt

Linux/macOS:
  python3 -m venv .venv
  . .venv/bin/activate
  pip install -r requirements.txt

3) Quick start
--------------

3.1 Verify only (no decryption)

  python verify_evidence_zip.py chat.zip

This checks:
- manifest.json parses
- sha256 matches for users_keys.json and wire_messages.jsonl
- wire_messages.jsonl parses
- per-conversation hash-chain consistency

3.2 Verify + decrypt v2 messages

  python verify_evidence_zip.py --decrypt chat_v2.zip

The tool will ask:
  Encryption private key (base64, 32 bytes; seed32 or sk32):

You can paste the key (TEST mode; visible input).

Notes:
- Decryption is best-effort.
- Only v2 messages that include stored key envelopes can be decrypted.

3.3 Print decrypted transcript (stdout)

  python verify_evidence_zip.py --decrypt --print-decrypted chat_v2.zip

Output:
- prints readable lines:
  [timestamp] SENDER_CORE_ID -> RECIPIENT_CORE_ID msg_id=... sig=PASS|WARN
  plaintext

If plaintext is an attachment message, the verifier prints:
  ATTACHMENT_ID: <uuid>

4) Tamper-evident readable export
---------------------------------

  python verify_evidence_zip.py --decrypt --export-readable out_readable.zip chat_v2.zip

Creates out_readable.zip with:
- report.json
  - verifier_version
  - input_zip_sha256
  - summary (PASS/WARN/FAIL counts)
  - checks[] list

- transcript.jsonl
  - one JSON object per decrypted message

- transcript.txt
  - human-readable transcript

- transcript.sha256
  - sha256 of transcript.jsonl

- input.zip
  - a copy of the original evidence ZIP (reproducibility)

Why this is tamper-evident:
- report.json includes sha256 of input.zip
- transcript.sha256 binds the transcript.jsonl content
- anyone can rerun the verifier on input.zip and compare transcript.jsonl

5) On-demand attachment download + decryption (Bearer)
------------------------------------------------------

Core51 stores attachment ciphertext in Cloudflare R2 (S3-compatible). The verifier does not download all attachments.
Instead, you can download one attachment by id when needed.

Command:

  python verify_evidence_zip.py --decrypt chat_v2.zip \
    --bearer-token "<JWT>" \
    --fetch-attachment "<ATTACHMENT_ID>"

Behavior:
- verifier calls:
    GET {api_base}/api/attachments/{attachment_id}
  with Authorization: Bearer <JWT>

- backend returns:
  - download_url (presigned GET)
  - ciphertext_sha256_b64
  - nonce_b64
  - aad_json
  - wrapped_keys_json
  - sender_id
  - filename

- verifier downloads ciphertext from download_url
- verifier checks sha256(ciphertext) == ciphertext_sha256_b64
- verifier unwraps file_key using your enc private key
- verifier decrypts AES-256-GCM with:
  - nonce = nonce_b64
  - AAD = aad_json (UTF-8)
- verifier writes decrypted file to disk as filename

Security note:
- for tests, passing JWT via CLI is acceptable.
- for production, do NOT pass tokens on the command line. Prefer stdin or environment variables.

6) Export formats
-----------------

6.1 v1 export (legacy)
Schema:
- core51.wire_export.v1

Sources:
- wire_messages table (legacy)

Properties:
- can be verified for integrity and chain
- cannot be decrypted with v2 envelopes
- signature verification may WARN after key rotation (historical pubkeys not included)

Endpoint:
- GET /api/chats/{chat_id}/wire-export.zip

6.2 v2 export (recommended)
Schema:
- core51.wire_export.v2

Sources:
- messages table (algo=v2)
- message_key_envelopes

Endpoint:
- GET /api/chats/{chat_id}/wire-export.v2.zip
- GET /api/chats/{chat_id}/wire-export.v2 (NDJSON)

v2 message fields include:
- msg_id, conversation_id, sender_id, recipient_id
- created_at, server_received_at
- gcm_nonce_b64
- ciphertext_b64
- ciphertext_hash_b64
- envelopes_hash_b64
- envelopes[] (recipient_user_id, wrap_algo, wrap_nonce_b64, wrapped_key_b64)
- payload_hash_b64 and prev_payload_hash_b64 (hash-chain)
- sender_signature_b64 (Ed25519)

Audit fields for stronger signature verification:
- sender_sign_pubkey_b64 (rotation-proof)
- sender_key_version_at_send
- signed_prev_msg_hash_b64 (the prev value that was part of the signature)

Interpreting PASS/WARN/FAIL:
- PASS: checks succeeded
- WARN: evidence is consistent, but some strict verification is not possible
  (e.g., legacy messages without audit fields, key rotation without historical keys)
- FAIL: integrity broken (sha256 mismatch, chain mismatch, invalid ZIP structure)

7) Practical guidance
---------------------

- If you need maximum credibility, always export v2 (wire-export.v2.zip).
- If decrypted count is smaller than message count, check whether old messages were stored without envelopes.
- Keep the original ZIP unchanged; always re-run verifier from the ZIP.