Skip to content

Frozen agent recovery evaluation: full96-v1

Latest

Choose a tag to compare

@niejingchuan niejingchuan released this 10 Oct 01:44

Frozen agent recovery evaluation: full96-v1

The first public reproduction package includes 24 synthetic recovery cases, a deterministic simulator and strict scorer, 49 offline calibration checks, and all 96 recorded external-provider episodes from the frozen October 9, 2026 evaluation.

Both model identifiers completed all 34 progress-required episodes each. Strict passes were 15/48 and 0/48. Final JSON format, attempted permission/binding violations, and actual business effects are reported separately. No duplicate-write attempts or duplicate effects were observed in this fixed batch.

Results preserve the actual Tencent TokenHub model identifiers. These are local external-provider research observations, not Kaggle leaderboard entries. Offline calibration requires no model API calls; new provider evaluation requires private credentials and incurs usage.

The benchmark code and accompanying authored materials use Apache-2.0. Imported third-party implementations are not bundled and retain their own licenses. The release contains full source hashes, result records and audit evidence.

ZIP SHA-256: 0254098eb055e94e5d11fc7f8041626df1541ba4ff15c2d051dc2c71641ebc03.

See README and RESULTS before interpreting the scores. The fixed, instruction-rich simulator and two identifiers from one family through one provider do not establish production safety or general model rankings.