Command Palette

Search for a command to run...

GitHub

What Evidence Do AI Agent Audit Logs Need After Redaction?

A Matched-Pair Pilot on Verifying Consent from Masked and Aliased Logs

Anish Prashun · Independent researcher · Technical report ·

Read the paper (PDF)Code and data (GitHub)DOI 10.5281/zenodo.24163831

Summary

AI agent audit logs often have to be redacted before a reviewer sees them, yet the reviewer must still decide whether each recorded action was authorised. This matched-pair pilot measures which evidence a redacted log must keep for that decision. It ships redaction_audit, a standard-library toolkit that tests your own redaction schemes against your own authorisation policy.

Key findings

  • Uniform masking made 6 of 10 matched pairs byte-identical, and the reviewer could confirm none of the 10 authorised actions.
  • Consistent within-run aliases restored all 20 verdicts while exposing none of the 66 planted raw values.
  • Removing expiry or revocation evidence left 12 of 20 decisions unresolved, including every authorised action.
  • Hashing identifiers with a fresh salt per event led a reviewer unaware of the scheme to flag all 10 authorised actions as violations, so redacted logs should declare how they were redacted.

Reproduce it

Python 3 standard library only, synthetic data, no network access. Clone the repository and run:

python3 run_pilot.py

Cite

Prashun, A. (2026). What Evidence Do AI Agent Audit Logs Need After Redaction? A Matched-Pair Pilot on Verifying Consent from Masked and Aliased Logs [Technical report]. Zenodo. https://doi.org/10.5281/zenodo.24163831

@misc{prashun2026redaction,
  author    = {Prashun, Anish},
  title     = {What Evidence Do AI Agent Audit Logs Need After Redaction? A Matched-Pair Pilot on Verifying Consent from Masked and Aliased Logs},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.24163831},
  url       = {https://doi.org/10.5281/zenodo.24163831},
  note      = {Technical report, code and data}
}

Paper and data: CC BY 4.0. Code: MIT. All data in the study is synthetic.

  • AI agents
  • Audit logs
  • Redaction
  • Pseudonymisation
  • Privacy engineering

More research