Monthly research note. Theme: Cryptographic Infrastructure.
TL;DR
A focused memo on TLS Beyond Defaults: Ciphersuites, ALPN, and Operational Reality: define the model, state the properties, then design the system so those properties remain true under failure and adversaries.
Most failures are boundary failures: parsing, persistence, concurrency, retries, and upgrades.
Key takeaways
- Treat key IDs as capabilities; never pass raw private key material across boundaries.
- Rotation and rollback are core features—design them before you ship.
- Bind purpose and context (domain separation) so keys can’t be misused accidentally.
- Measure correctness signals, not only latency/throughput.
- Write assumptions down; treat them as interfaces.
Why this matters
- Most organizations don’t know where their keys live—until an incident.
- Side channels turn performance details into security boundaries.
- Cryptographic agility is useless if rollout and rollback are unsafe.
- Key management failures are systemic: the breach is “a workflow,” not a bug.
Key questions
- Which operations must be constant-time and how do you validate that?
- What is the blast radius of compromise (tenant, service, region, environment)?
- How do you prove usage (who signed what, when, and why) without leaking secrets?
- What is your disaster recovery story for KMS/HSM outages?
- How do you separate duties (operators vs developers vs security responders)?
- How do keys rotate safely (overlap windows, dual-sign, staged rollout)?
Assumptions
- Key usage is high-volume; audit pipelines must scale without sampling away truth.
- Secrets leak through logs, metrics, crash dumps, and backups unless prevented.
- Rotation must occur under incident pressure; automation must be safe.
- Some environments are hostile (CI, ephemeral runners, shared build agents).
Non-goals
- Designing audit trails that expose sensitive plaintext or identifiers.
- Assuming “HSM = secure” without defining the threat model.
Parsing is an attacker-controlled interface—validate early and fail fast.
Model & invariants
A practical safety statement for key usage is least authority:
Treat key identifiers as capabilities with purpose constraints—enforce in code and policy.
Assume compromise and design for recovery: rotation, revocation, and forensics.
Monotonicity beats timestamps: counters and epochs survive clock skew.
Security properties
- Integrity: invalid transitions are rejected (and detectable).
- Downgrade resistance: negotiation can’t silently weaken security posture.
- Replay resistance: duplicated inputs do not change outcomes.
- Least authority: privileges are scoped by purpose and time.
Failure modes
- Recovery paths that only work when nothing is broken.
- Resource exhaustion (CPU/bandwidth/storage) turning into correctness failures.
- Timeout ambiguity causing double-apply or partial state transitions.
- Mixed-version behavior that violates assumptions silently.
Sampling hides the rare schedule that breaks your invariants.
Design sketch
flowchart TD
gen["KeyGen (HSM/KMS)"] --> use["Use (TLS/VPN/Signing)"]
use --> rot["Rotate (policy + automation)"]
rot --> revoke["Revoke (incident)"]
revoke --> audit["Audit/Forensics"]
audit --> genImplementation notes
Never pass secrets around; pass handles with purpose constraints.
Acknowledge only after durability (or make “ack” explicitly best-effort).
#[derive(Clone, Copy, Debug)]
pub enum Purpose { Tls, Jwt, Firmware, Ledger }
pub struct KeyHandle { id: String, purpose: Purpose }
// Enforce purpose and algorithm policy at the boundary, not in the caller.Verification strategy
- Constant-time validation: microbenchmarks + side-channel tooling where feasible.
- Config drift detection: policy-as-code with diffs treated as security events.
- Chaos for KMS: inject throttling, partial outages, and latency spikes.
- Misuse resistance tests: wrong purpose, wrong context, wrong key type must fail.
- Rotation drills: staged rollout, dual-sign windows, and rollback.
Operational notes
- Automate rotation with safety rails (canary, dual-sign, fast rollback).
- Alert on policy drift: cipher suites, key sizes, algorithm toggles, TTL changes.
- Make audit streams append-only and queryable during incidents.
- Test backup/restore for crypto material with the same rigor as databases.
- Separate duties and restrict production key access paths.
Keep audit and config history queryable during incidents—evidence beats intuition.
What to monitor
- Invariant violation rate (should be ~0).
- Authz failures and policy denials (unexpected spikes).
- Rollback events and the conditions that triggered them.
- Error budget burn + tail latency under load.
- Admission-control / rate-limit rejections (by reason).
Rollback plan
- Use canaries and staged rollout; stop early when signals degrade.
- Define an explicit rollback trigger (metrics + thresholds).
- Keep dual-write / dual-verify windows where appropriate.
- Prefer backward-compatible changes; avoid “flag day” upgrades.
- Preserve evidence (configs, artifacts, audit logs) to reconstruct what changed.
Evidence
- RFC 8446: TLS 1.3 (1) — Modern handshake design, key schedule, and downgrade resistance patterns.
- Evidence: Handshake transcript binding and downgrade resistance patterns; monitor negotiation paths and failure reasons.
- Learn TLA+ (2) — Practical entry point for specification and model checking.
- Evidence: Model the smallest thing that can break; use model checking to validate invariants before optimizing.
Open questions
- Which secrets must remain confidential for 10+ years and where are they stored today?
- What would a KMS compromise look like in your telemetry?
- How do you guarantee that audit does not become a data exfiltration channel?
- What is your plan for emergency revocation at global scale?
Checklist
- Rollback plan rehearsed and automated.
- Assumptions listed and reviewed.
- Safety properties stated as invariants.
- Telemetry captures correctness signals.
- Costs bounded (CPU/memory/bandwidth) under adversarial inputs.
- Failure modes enumerated with mitigations.
Further reading
- RFC 5869: HKDF — Domain separation and key derivation done sanely.
- Let's Encrypt Incident Reports — Real-world PKI incidents and operational lessons.
- NIST SP 800-57 Part 1 Rev. 5 — Key management guidance: lifecycle, strength, and policy.
- RFC 8446: TLS 1.3 — Modern handshake design, key schedule, and downgrade resistance patterns.
- Site Reliability Engineering (Google) — Error budgets, incident response, and reliability as an engineering discipline.
- Learn TLA+ — Practical entry point for specification and model checking.