Summary
The self-assessment: adversarially probe a system you built. Ten findings, five fixes, one honest report. The exercise that ships production readiness.
Objectives
- 01Systematically probe a system for failure modes.
- 02Categorize findings by severity and class.
- 03Ship at least five fixes.
- 04Report findings and residual risk honestly.
Key Ideas
- The team that will not red-team its own system will ship one that is red-teamed by users.
- Severity is not the same as ease of exploit — track both.
Lab
Choose a system you or a peer built. Spend at least two days adversarially probing it: prompt injection, jailbreaks, data exfiltration, denial of wallet, PII leakage, unsafe outputs, tool misuse. Document at least ten distinct findings with reproduction steps and severity ratings (low/medium/high). Ship fixes for at least five, prioritizing the high-severity ones. Write a report with findings, fixes, and residual risk. Include a decision rule for when a finding should block deployment.
Deliverables
- — Ten findings with reproduction steps and severity.
- — Five shipped fixes with tests that catch regression.
- — Written report with residual risk and blocking criteria.
- — Presentation to at least two peers.
Rubric
- — Coverage — findings span at least four distinct attack classes.
- — Reproducibility — every finding has runnable steps.
- — Prioritization — high-severity items are addressed first.
- — Honesty — residual risk is named, not hidden.