Testing platform for ZShot
| Template | Total Runs | Pass | Fail | Pass Rate | Avg Tokens In | Avg Tokens Out | Tokens In per Pass |
|---|---|---|---|---|---|---|---|
| doom_destroy_a_barrel | 8 | 4 | 4 | 50.0% | 200,028 | 5,984 | 400,056 |
| parcel_redelivery | 218 | 80 | 138 | 36.7% | 169,996 | 16,895 | 451,889 |
Token figures count runs recorded with --llm-usage. Tokens in per pass is every metered run's input over the metered runs that passed.
| ID | Template | Label | Result | Furthest | Tokens | Est. Cost | Took | Created |
|---|---|---|---|---|---|---|---|---|
| 79 | parcel_redelivery | scout-deepseek | fail | confirm | 201,365 in / 87,506 out | $0.162 | 7m 33s | 2026-09-28 11:49 |
| 78 | parcel_redelivery | scout-deepseek | fail | done | 119,996 in / 18,440 out | $0.058 | 2m 4s | 2026-09-28 11:47 |
| 77 | parcel_redelivery | scout-qwen | fail | preferences | 346,850 in / 6,319 out | $0.023 | 7m 22s | 2026-09-28 11:45 |
| 76 | parcel_redelivery | scout-sonnet | fail | done | 248,899 in / 10,790 out | $0.433 | 2m 55s | 2026-09-28 11:44 |
| 75 | parcel_redelivery | scout-gemini | fail | done | 85,981 in / 10,077 out | $0.070 | 1m 13s | 2026-09-28 11:44 |
| 74 | parcel_redelivery | scout-gemini | fail | done | 81,444 in / 15,939 out | $0.089 | 1m 36s | 2026-09-28 11:42 |
| 73 | parcel_redelivery | scout-sonnet | fail | done | 177,088 in / 7,698 out | $0.302 | 2m 5s | 2026-09-28 11:42 |
| 72 | parcel_redelivery | scout-sonnet-opus | pass | done | 171,985 in / 5,384 out | $0.476 | 1m 49s | 2026-09-28 11:39 |
| 71 | parcel_redelivery | scout-sonnet-opus | pass | done | 241,606 in / 8,203 out | $0.689 | 2m 21s | 2026-09-28 11:37 |
| 70 | parcel_redelivery | scout-sonnet | fail | done | 149,784 in / 5,209 out | $0.225 | 1m 23s | 2026-09-28 11:35 |
| 69 | parcel_redelivery | scout-sonnet | fail | done | 265,073 in / 10,550 out | $0.485 | 2m 57s | 2026-09-28 11:32 |
| 68 | parcel_redelivery | scout-opus | pass | done | 192,639 in / 6,065 out | $0.600 | 2m 13s | 2026-09-28 11:30 |
| 67 | parcel_redelivery | scout-opus | pass | done | 232,613 in / 8,199 out | $0.792 | 2m 55s | 2026-09-28 11:27 |
| 66 | parcel_redelivery | opus-thinking | fail | confirm | 212,288 in / 7,537 out | $1.000 | 4m 10s | 2026-09-28 11:23 |
| 65 | parcel_redelivery | scout-gemini | fail | confirm | 62,117 in / 8,157 out | $0.050 | 59s | 2026-09-28 11:21 |
| 64 | parcel_redelivery | opus-thinking | pass | done | 179,970 in / 4,873 out | $0.817 | 2m 52s | 2026-09-28 11:20 |
| 63 | parcel_redelivery | deepseek | fail | done | 190,480 in / 95,029 out | $0.154 | 8m 32s | 2026-09-28 11:20 |
| 62 | parcel_redelivery | scout-gemini | fail | done | 166,289 in / 32,128 out | $0.209 | 3m 36s | 2026-09-28 11:18 |
| 61 | parcel_redelivery | deepseek | fail | preferences | 93,764 in / 31,350 out | $0.059 | 3m 18s | 2026-09-28 11:16 |
| 60 | parcel_redelivery | sonnet | fail | preferences | 467,846 in / 22,981 out | $1.17 | 7m 2s | 2026-09-28 11:13 |
| 59 | parcel_redelivery | scout-qwen | fail | confirm | 110,097 in / 3,369 out | $0.023 | 1m 46s | 2026-09-28 11:11 |
| 58 | parcel_redelivery | deepseek-low | fail | done | 233,042 in / 68,628 out | $0.133 | 7m 9s | 2026-09-28 11:09 |
| 57 | parcel_redelivery | scout-qwen | fail | done | 207,231 in / 4,764 out | $0.039 | 3m 18s | 2026-09-28 11:07 |
| 56 | parcel_redelivery | gemini | fail | preferences | 300,334 in / 143,388 out | $0.763 | 13m 1s | 2026-09-28 11:05 |
| 55 | parcel_redelivery | sonnet | fail | preferences | 537,973 in / 26,686 out | $1.34 | 10m 13s | 2026-09-28 11:03 |
| 54 | parcel_redelivery | deepseek-low | fail | preferences | 339,288 in / 53,147 out | $0.143 | 6m 32s | 2026-09-28 11:03 |
| 53 | parcel_redelivery | qwen-noreasoning | fail | preferences | 260,715 in / 2,361 out | $0.000 | 6m 15s | 2026-09-28 11:01 |
| 52 | parcel_redelivery | opus | pass | done | 202,780 in / 5,898 out | $0.929 | 3m 27s | 2026-09-28 10:59 |
| 51 | parcel_redelivery | scout-deepseek | fail | confirm | 203,700 in / 38,398 out | $0.112 | 3m 56s | 2026-09-28 10:59 |
| 50 | parcel_redelivery | scout-deepseek | fail | done | 120,121 in / 29,604 out | $0.071 | 2m 56s | 2026-09-28 10:56 |
| 49 | parcel_redelivery | deepseek-noreasoning | fail | done | 79,364 in / 1,108 out | $0.018 | 1m 1s | 2026-09-28 10:55 |
| 48 | parcel_redelivery | opus | fail | slot | 373,988 in / 23,506 out | $1.97 | 7m 18s | 2026-09-28 10:52 |
| 47 | parcel_redelivery | deepseek-noreasoning | fail | done | 264,069 in / 3,700 out | $0.068 | 2m 36s | 2026-09-28 10:52 |
| 46 | parcel_redelivery | qwen-noreasoning | fail | preferences | 384,036 in / 4,240 out | $0.000 | 9m 6s | 2026-09-28 10:52 |
| 45 | parcel_redelivery | gemini | fail | preferences | 275,572 in / 145,978 out | $0.754 | 12m 26s | 2026-09-28 10:52 |
| 44 | parcel_redelivery | deepseek | fail | preferences | 235,861 in / 179,575 out | $0.263 | 15m 5s | 2026-09-28 03:51 |
| 43 | parcel_redelivery | scout-sonnet-opus | fail | done | 59,126 in / 3,177 out | $0.033 | 31s | 2026-09-28 03:40 |
| 42 | parcel_redelivery | scout-sonnet-opus | fail | done | 83,382 in / 4,192 out | $0.043 | 37s | 2026-09-28 03:39 |
| 41 | parcel_redelivery | scout-sonnet | fail | done | 60,473 in / 2,835 out | $0.058 | 38s | 2026-09-28 03:38 |
| 40 | parcel_redelivery | scout-sonnet | fail | done | 80,560 in / 3,646 out | $0.042 | 38s | 2026-09-28 03:38 |
| 39 | parcel_redelivery | scout-opus | pass | done | 148,140 in / 4,430 out | $0.424 | 1m 33s | 2026-09-28 03:36 |
| 38 | parcel_redelivery | deepseek | fail | preferences | 264,375 in / 155,574 out | $0.249 | 15m 6s | 2026-09-28 03:35 |
| 37 | parcel_redelivery | scout-opus | pass | done | 164,501 in / 4,978 out | $0.437 | 1m 36s | 2026-09-28 03:35 |
| 36 | parcel_redelivery | opus-thinking | fail | confirm | 173,136 in / 5,492 out | $0.802 | 2m 35s | 2026-09-28 03:32 |
| 35 | parcel_redelivery | scout-qwen | fail | done | 58,573 in / 2,837 out | $0.022 | 36s | 2026-09-28 03:26 |
| 34 | parcel_redelivery | scout-qwen | fail | done | 64,855 in / 2,807 out | $0.021 | 50s | 2026-09-28 03:25 |
| 33 | parcel_redelivery | opus-thinking | fail | confirm | 422,288 in / 24,969 out | $2.19 | 7m 37s | 2026-09-28 03:24 |
| 32 | parcel_redelivery | deepseek-low | fail | preferences | 324,309 in / 129,276 out | $0.229 | 12m 31s | 2026-09-28 03:23 |
| 31 | parcel_redelivery | sonnet | fail | preferences | 152,833 in / 15,462 out | $0.460 | 3m 57s | 2026-09-28 03:21 |
| 30 | parcel_redelivery | scout-gemini | fail | done | 89,950 in / 4,461 out | $0.044 | 44s | 2026-09-28 03:20 |