Curated Terminal-Bench 4.0
Our agent Vis vs. 5 other coding agents on 12 CPU-only Terminal-Bench 4.0 tasks, all on GPT-6.1-sol with medium reasoning.
- Tested
- Vis* (0.5.7)
- Codex (0.166.0)
- OpenCode (2.2.5)
- Pi (1.5.0)
- Kmet (v0.3.1-78-gf07fa90e-dirty)
- Kmet eval-only (v0.3.1-78-gf07fa90e-dirty)
- Kmet codemode (v0.3.1-78-gf07fa90e-dirty)
- Kmet shell-only (v0.3.1-78-gf07fa90e-dirty)
- Wisp (9cddabcd)
Run natively (arm64) on a MacBook Pro M4 Max (Podman VM, 12 CPUs, 24 GiB). * To be run again natively.
Rounds
1 to 5 per agent
300 trials
Passed per round
53.4
50% pass rate
Cost per round
$24.99
1 agent round without a value left out
Calls per round
2,126
Tool calls per round
2,184
Tool errors per round
128.2
Trials
108
52 passed
Pass rate
48%
Calls
2,141
12 h 25 min recorded
Cost
$25.57
108 of 108 trials
Tool calls
2,195
Tool errors
126
6% of the calls that mark errors
Trials
48
21 passed
Pass rate
44%
Calls
1,042
6 h 03 min recorded
Cost
$12.49
48 of 48 trials
Tool calls
1,117
Tool errors
38
3% of the calls that mark errors
Trials
48
28 passed
Pass rate
58%
Calls
1,067
5 h 42 min recorded
Cost
$12.83
48 of 48 trials
Tool calls
1,126
Tool errors
58
5% of the calls that mark errors
Trials
48
27 passed
Pass rate
56%
Calls
1,039
5 h 38 min recorded
Cost
$11.80
47 of 48 trials
Tool calls
1,101
Tool errors
54
5% of the calls that mark errors
Trials
48
23 passed
Pass rate
48%
Calls
977
5 h 13 min recorded
Cost
$11.84
48 of 48 trials
Tool calls
1,027
Tool errors
41
4% of the calls that mark errors
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Pass rate (%)" 0 --> 100
bar [50, 50, 42, 58, 33, 60, 53, 58, 40]| codex | 5 | 6.0 ± 1.6 | 50% | $3.64 ± $0.3814 | 2 h 11 min ± 7 min 23 s | 244 ± 24 | 10.93M ± 1.44M | 179.1k ± 16.4k | 232 ± 24 | 2.0 ± 1.4 | <1% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| kmet | 1 | 6.0 | 50% | $2.39 | 1 h 57 min | 261 | 4.01M | 121.6k | 309 | 34.0 | 11% |
| kmet-codemode | 1 | 5.0 | 42% | $2.38 | 1 h 54 min | 195 | 3.58M | 121.3k | 181 | 9.0 | 5% |
| kmet-eval-only | 1 | 7.0 | 58% | $2.37 | 1 h 35 min | 213 | 3.40M | 129.5k | 201 | 14.0 | 7% |
| kmet-shell-only | 1 | 4.0 | 33% | $1.69 | 1 h 21 min | 141 | 2.19M | 85.9k | 129 | 13.0 | 10% |
| opencode | 5 | 7.2 ± 0.8 | 60% | not recorded | 2 h 20 min ± 10 min 27 s | 259 ± 17 | not recorded | not recorded | 296 ± 22 | 1.6 ± 1.1 | <1% |
| pi | 5 | 6.4 ± 1.1 | 53% | $3.04 ± $0.2569 | 2 h 09 min ± 10 min 48 s | 311 ± 14 | 7.06M ± 1.16M | 160.0k ± 12.2k | 360 ± 13 | 19.2 ± 7.3 | 5% |
| vis-solo | 1 | 7.0 | 58% | $3.67 | 2 h 49 min | 281 | 7.27M | 233.0k | 269 | 11.0 | 4% |
| wisp | 5 | 4.8 ± 1.6 | 40% | $2.73 ± $0.3470 | 1 h 52 min ± 7 min 37 s | 222 ± 19 | 4.29M ± 594.5k | 159.9k ± 18.3k | 207 ± 19 | 24.4 ± 6.8 | 12% |
Passes per task
| bun-sourcemap-leak | 0/5 | 0/1 | 0/1 | 0/1 | 0/1 | 2/5 | 1/5 | 1/1 | 2/5 |
|---|---|---|---|---|---|---|---|---|---|
| html-js-filter | 3/5 | 1/1 | 1/1 | 1/1 | 0/1 | 5/5 | 3/5 | 0/1 | 2/5 |
| interleaved-vigenere | 5/5 | 1/1 | 1/1 | 1/1 | 1/1 | 4/5 | 5/5 | 1/1 | 5/5 |
| kv-live-surgery | 3/5 | 0/1 | 0/1 | 0/1 | 0/1 | 0/5 | 1/5 | 0/1 | 1/5 |
| music-harmony | 0/5 | 0/1 | 0/1 | 0/1 | 0/1 | 2/5 | 1/5 | 0/1 | 0/5 |
| mvcc-lsm-compaction | 3/5 | 0/1 | 0/1 | 1/1 | 1/1 | 5/5 | 3/5 | 1/1 | 2/5 |
| payments-pipeline-fix | 4/5 | 0/1 | 0/1 | 1/1 | 0/1 | 0/5 | 3/5 | 1/1 | 2/5 |
| photonic-waveguide-routing | 5/5 | 1/1 | 1/1 | 1/1 | 1/1 | 5/5 | 5/5 | 1/1 | 5/5 |
| react-lead-form | 2/5 | 1/1 | 0/1 | 1/1 | 1/1 | 3/5 | 2/5 | 0/1 | 2/5 |
| session-window-debug | 1/5 | 0/1 | 1/1 | 0/1 | 0/1 | 3/5 | 1/5 | 1/1 | 1/5 |
| vf2-speedup-networkx | 0/5 | 1/1 | 0/1 | 0/1 | 0/1 | 3/5 | 3/5 | 0/1 | 0/5 |
| wal-recovery-ordering | 4/5 | 1/1 | 1/1 | 1/1 | 0/1 | 4/5 | 4/5 | 1/1 | 2/5 |
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Pass rate (%)" 0 --> 100
bar [58, 50, 42, 58, 33, 50, 58, 58, 25]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Cost (USD)" 0 --> 5
bar [4.19, 2.39, 2.38, 2.37, 1.69, 2.83, 3.47, 3.67, 2.58]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Input tokens (millions)" 0 --> 20
bar [13.4, 4.0, 3.6, 3.4, 2.2, 6.4, 9.0, 7.3, 3.7]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 300
bar [189.8, 121.6, 121.3, 129.5, 85.9, 153.2, 174.8, 233.0, 153.8]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [96, 90, 88, 89, 85, 95, 95, 96, 91]| codex | 12 | 7 | 58% | 2 h 21 min | 8 min 21 s | 276 | 1 h 43 min 276/276 calls recorded | $4.19 12/12 trials recorded | 264 | 3 | 1% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| kmet | 12 | 6 | 50% | 1 h 57 min | 6 min 16 s | 261 | 1 h 30 min 261/261 calls recorded | $2.39 12/12 trials recorded | 309 | 34 | 11% |
| kmet-codemode | 12 | 5 | 42% | 1 h 54 min | 7 min 21 s | 195 | 1 h 19 min 195/195 calls recorded | $2.38 12/12 trials recorded | 181 | 9 | 5% |
| kmet-eval-only | 12 | 7 | 58% | 1 h 35 min | 6 min 48 s | 213 | 1 h 02 min 213/213 calls recorded | $2.37 12/12 trials recorded | 201 | 14 | 7% |
| kmet-shell-only | 12 | 4 | 33% | 1 h 21 min | 4 min 48 s | 141 | 53 min 56 s 141/141 calls recorded | $1.69 12/12 trials recorded | 129 | 13 | 10% |
| opencode | 12 | 6 | 50% | 2 h 05 min | 8 min 11 s | 241 | 1 h 06 min 241/241 calls recorded | $2.83 12/12 trials recorded | 277 | 0 | 0% |
| pi | 12 | 7 | 58% | 2 h 20 min | 8 min 04 s | 329 | 1 h 37 min 329/329 calls recorded | $3.47 12/12 trials recorded | 374 | 17 | 5% |
| vis-solo | 12 | 7 | 58% | 2 h 49 min | 10 min 54 s | 281 | 1 h 52 min 281/281 calls recorded | $3.67 12/12 trials recorded | 269 | 11 | 4% |
| wisp | 12 | 3 | 25% | 1 h 57 min | 8 min 05 s | 204 | 1 h 17 min 204/204 calls recorded | $2.58 12/12 trials recorded | 191 | 25 | 13% |
Tasks passed by every agent, one attempt each: photonic-waveguide-routing.
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Cost (USD)" 0 --> 1
bar [0.54, 0.35, 0.22, 0.38, 0.14, 0.49, 0.56, 0.64, 0.38]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Input tokens (millions)" 0 --> 2
bar [1.1, 0.3, 0.2, 0.4, 0.1, 0.6, 1.3, 1.1, 0.5]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 50
bar [31.6, 22.7, 14.6, 26.9, 10.2, 38.2, 33.2, 49.4, 29.4]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "kmet", "kmet-codemode", "kmet-eval-only", "kmet-shell-only", "opencode", "pi", "vis-solo", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [94, 85, 79, 89, 81, 96, 96, 98, 95]| codex | 1 | 1 | 100% | 23 min 06 s | 23 min 06 s | 23 | 20 min 10 s 23/23 calls recorded | $0.5350 1/1 trials recorded | 22 | 1 | 5% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| kmet | 1 | 1 | 100% | 14 min 18 s | 14 min 18 s | 27 | 11 min 43 s 27/27 calls recorded | $0.3501 1/1 trials recorded | 26 | 0 | 0% |
| kmet-codemode | 1 | 1 | 100% | 14 min 30 s | 14 min 30 s | 13 | 8 min 03 s 13/13 calls recorded | $0.2247 1/1 trials recorded | 12 | 0 | 0% |
| kmet-eval-only | 1 | 1 | 100% | 15 min 40 s | 15 min 40 s | 26 | 12 min 28 s 26/26 calls recorded | $0.3806 1/1 trials recorded | 25 | 1 | 4% |
| kmet-shell-only | 1 | 1 | 100% | 7 min 51 s | 7 min 51 s | 9 | 5 min 47 s 9/9 calls recorded | $0.1449 1/1 trials recorded | 8 | 0 | 0% |
| opencode | 1 | 1 | 100% | 27 min 28 s | 27 min 28 s | 17 | 22 min 35 s 17/17 calls recorded | $0.4895 1/1 trials recorded | 16 | 0 | 0% |
| pi | 1 | 1 | 100% | 29 min 43 s | 29 min 43 s | 44 | 20 min 53 s 44/44 calls recorded | $0.5585 1/1 trials recorded | 43 | 0 | 0% |
| vis-solo | 1 | 1 | 100% | 32 min 10 s | 32 min 10 s | 26 | 27 min 24 s 26/26 calls recorded | $0.6379 1/1 trials recorded | 25 | 1 | 4% |
| wisp | 1 | 1 | 100% | 20 min 10 s | 20 min 10 s | 16 | 17 min 45 s 16/16 calls recorded | $0.3793 1/1 trials recorded | 15 | 0 | 0% |
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Pass rate (%)" 0 --> 100
bar [42, 58, 50, 25]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 4
bar [3.75, 3.23, 3.08, 2.43]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 20
bar [10.9, 7.2, 6.5, 4.8]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 200
bar [193.8, 191.7, 171.6, 140.2]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [96, 96, 94, 94]| codex | 12 | 5 | 42% | 2 h 11 min | 8 min 49 s | 255 | 1 h 38 min 255/255 calls recorded | $3.75 12/12 trials recorded | 243 | 4 | 2% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 12 | 7 | 58% | 2 h 29 min | 8 min 42 s | 274 | 1 h 30 min 274/274 calls recorded | $3.23 12/12 trials recorded | 321 | 2 | <1% |
| pi | 12 | 6 | 50% | 2 h 21 min | 7 min 28 s | 304 | 1 h 42 min 304/304 calls recorded | $3.08 12/12 trials recorded | 357 | 14 | 4% |
| wisp | 12 | 3 | 25% | 1 h 42 min | 5 min 47 s | 209 | 1 h 12 min 209/209 calls recorded | $2.43 12/12 trials recorded | 196 | 18 | 9% |
Tasks passed by every agent, one attempt each: interleaved-vigenere, photonic-waveguide-routing.
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 1
bar [0.62, 0.7, 0.76, 0.47]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 2
bar [1.5, 0.8, 1.1, 0.6]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 60
bar [36.0, 53.5, 48.0, 31.3]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [96, 95, 92, 92]| codex | 2 | 2 | 100% | 27 min 12 s | 13 min 36 s | 43 | 22 min 31 s 43/43 calls recorded | $0.6157 2/2 trials recorded | 41 | 1 | 2% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 2 | 2 | 100% | 38 min 51 s | 19 min 25 s | 39 | 32 min 36 s 39/39 calls recorded | $0.7049 2/2 trials recorded | 40 | 0 | 0% |
| pi | 2 | 2 | 100% | 40 min 06 s | 20 min 03 s | 56 | 31 min 57 s 56/56 calls recorded | $0.7606 2/2 trials recorded | 54 | 0 | 0% |
| wisp | 2 | 2 | 100% | 21 min 50 s | 10 min 55 s | 34 | 16 min 57 s 34/34 calls recorded | $0.4661 2/2 trials recorded | 32 | 1 | 3% |
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Pass rate (%)" 0 --> 100
bar [67, 67, 50, 50]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 4
bar [3.58, 3.09, 2.84, 3.33]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 10
bar [10.0, 6.5, 7.1, 5.0]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 200
bar [171.0, 175.9, 150.1, 186.5]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [95, 94, 95, 90]| codex | 12 | 8 | 67% | 2 h 10 min | 9 min 20 s | 238 | 1 h 32 min 238/238 calls recorded | $3.58 12/12 trials recorded | 226 | 1 | <1% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 12 | 8 | 67% | 2 h 15 min | 8 min 18 s | 253 | 1 h 13 min 253/253 calls recorded | $3.09 12/12 trials recorded | 289 | 1 | <1% |
| pi | 12 | 6 | 50% | 2 h 01 min | 7 min 38 s | 323 | 1 h 25 min 323/323 calls recorded | $2.84 12/12 trials recorded | 373 | 21 | 6% |
| wisp | 12 | 6 | 50% | 2 h 02 min | 8 min 07 s | 253 | 1 h 30 min 252/253 calls recorded | $3.33 12/12 trials recorded | 238 | 35 | 15% |
Tasks passed by every agent, one attempt each: mvcc-lsm-compaction, photonic-waveguide-routing.
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 1
bar [0.48, 0.44, 0.42, 0.66]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 1
bar [0.8, 0.7, 1.0, 1.0]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 40
bar [25.8, 28.9, 20.8, 36.8]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [90, 94, 94, 90]| codex | 2 | 2 | 100% | 22 min 36 s | 11 min 18 s | 23 | 15 min 58 s 23/23 calls recorded | $0.4764 2/2 trials recorded | 21 | 0 | 0% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 2 | 2 | 100% | 25 min 25 s | 12 min 42 s | 30 | 17 min 07 s 30/30 calls recorded | $0.4432 2/2 trials recorded | 32 | 0 | 0% |
| pi | 2 | 2 | 100% | 22 min 00 s | 11 min 00 s | 41 | 13 min 10 s 41/41 calls recorded | $0.4215 2/2 trials recorded | 51 | 4 | 8% |
| wisp | 2 | 2 | 100% | 27 min 15 s | 13 min 38 s | 37 | 20 min 04 s 37/37 calls recorded | $0.6594 2/2 trials recorded | 35 | 2 | 6% |
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Pass rate (%)" 0 --> 100
bar [50, 58, 67, 50]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 4
bar [3.55, 2.63, 2.91, 2.71]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 20
bar [10.2, 6.0, 5.9, 3.7]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 200
bar [186.7, 152.4, 149.6, 169.7]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [97, 96, 93, 91]| codex | 12 | 6 | 50% | 2 h 14 min | 8 min 25 s | 242 | 1 h 36 min 242/242 calls recorded | $3.55 12/12 trials recorded | 230 | 1 | <1% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 12 | 7 | 58% | 2 h 19 min | 10 min 00 s | 280 | 1 h 15 min 280/280 calls recorded | $2.63 11/12 trials recorded | 317 | 3 | <1% |
| pi | 12 | 8 | 67% | 2 h 04 min | 8 min 37 s | 299 | 1 h 22 min 299/299 calls recorded | $2.91 12/12 trials recorded | 353 | 31 | 9% |
| wisp | 12 | 6 | 50% | 1 h 51 min | 6 min 35 s | 218 | 1 h 24 min 218/218 calls recorded | $2.71 12/12 trials recorded | 201 | 19 | 9% |
Tasks passed by every agent, one attempt each: interleaved-vigenere, photonic-waveguide-routing.
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 1
bar [0.78, 0.55, 0.67, 0.61]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 2
bar [2.0, 0.9, 1.8, 0.6]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 50
bar [45.3, 39.2, 32.6, 45.1]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [97, 96, 95, 91]| codex | 2 | 2 | 100% | 34 min 05 s | 17 min 03 s | 51 | 29 min 36 s 51/51 calls recorded | $0.7774 2/2 trials recorded | 49 | 1 | 2% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 2 | 2 | 100% | 29 min 57 s | 14 min 59 s | 38 | 23 min 33 s 38/38 calls recorded | $0.5514 2/2 trials recorded | 37 | 0 | 0% |
| pi | 2 | 2 | 100% | 33 min 06 s | 16 min 33 s | 62 | 22 min 10 s 62/62 calls recorded | $0.6687 2/2 trials recorded | 63 | 3 | 5% |
| wisp | 2 | 2 | 100% | 27 min 15 s | 13 min 37 s | 34 | 23 min 50 s 34/34 calls recorded | $0.6134 2/2 trials recorded | 31 | 0 | 0% |
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Pass rate (%)" 0 --> 100
bar [33, 67, 42, 50]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 4
bar [3.14, 3.19, 2.9, 2.62]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 20
bar [10.1, 6.8, 6.9, 4.2]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 200
bar [154.1, 172.1, 154.0, 149.2]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [97, 94, 95, 91]| codex | 12 | 4 | 33% | 2 h 01 min | 7 min 27 s | 210 | 1 h 20 min 210/210 calls recorded | $3.14 12/12 trials recorded | 198 | 1 | <1% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 12 | 8 | 67% | 2 h 30 min | 10 min 55 s | 245 | 1 h 13 min 245/245 calls recorded | $3.19 12/12 trials recorded | 276 | 2 | <1% |
| pi | 12 | 5 | 42% | 1 h 58 min | 9 min 35 s | 298 | 1 h 25 min 298/298 calls recorded | $2.90 12/12 trials recorded | 344 | 13 | 4% |
| wisp | 12 | 6 | 50% | 1 h 48 min | 7 min 18 s | 224 | 1 h 14 min 224/224 calls recorded | $2.62 12/12 trials recorded | 209 | 25 | 12% |
Tasks passed by every agent, one attempt each: interleaved-vigenere, photonic-waveguide-routing, wal-recovery-ordering.
---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cost (USD)" 0 --> 1
bar [0.6, 0.82, 0.79, 0.45]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Input tokens (millions)" 0 --> 2
bar [1.0, 1.2, 1.3, 0.6]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Output tokens (thousands)" 0 --> 60
bar [38.4, 52.8, 48.2, 27.7]---
config:
xyChart:
showDataLabel: true
---
xychart-beta horizontal
x-axis ["codex", "opencode", "pi", "wisp"]
y-axis "Cached share of input (%)" 0 --> 100
bar [94, 93, 92, 89]| codex | 3 | 3 | 100% | 29 min 35 s | 7 min 50 s | 36 | 20 min 06 s 36/36 calls recorded | $0.6022 3/3 trials recorded | 33 | 1 | 3% |
|---|---|---|---|---|---|---|---|---|---|---|---|
| opencode | 3 | 3 | 100% | 51 min 14 s | 11 min 45 s | 53 | 30 min 14 s 53/53 calls recorded | $0.8186 3/3 trials recorded | 52 | 0 | 0% |
| pi | 3 | 3 | 100% | 38 min 52 s | 11 min 34 s | 71 | 29 min 00 s 71/71 calls recorded | $0.7941 3/3 trials recorded | 84 | 0 | 0% |
| wisp | 3 | 3 | 100% | 22 min 19 s | 6 min 20 s | 37 | 14 min 50 s 37/37 calls recorded | $0.4521 3/3 trials recorded | 34 | 0 | 0% |
Tool errors by tool
| wisp | bash | 201.6 ± 21.5 | 24.2 ± 6.8 | 12% | 7.8 ± 0.4 |
| pi | bash | 155.8 ± 6.4 | 18.8 ± 7.4 | 12% | 8.6 ± 0.5 |
| kmet | bash | 122.0 | 18.0 | 15% | 8.0 |
| kmet-eval-only | eval | 201.0 | 14.0 | 7% | 9.0 |
| kmet-shell-only | bash | 129.0 | 13.0 | 10% | 6.0 |
| kmet | read | 88.0 | 11.0 | 12% | 5.0 |
| vis-solo | python_execution | 269.0 | 11.0 | 4% | 6.0 |
| codex | exec | 230.8 ± 23.9 | 2.0 ± 1.4 | <1% | 2.0 ± 1.4 |
| kmet-codemode | codemode | 181.0 | 9.0 | 5% | 8.0 |
| kmet | edit | 71.0 | 5.0 | 7% | 3.0 |
| opencode | read | 58.6 ± 9.9 | 0.8 ± 0.4 | 1% | 0.8 ± 0.4 |
| opencode | patch | 66.2 ± 5.4 | 0.6 ± 0.5 | <1% | 0.6 ± 0.5 |
| pi | edit | 78.2 ± 6.6 | 0.4 ± 0.5 | <1% | 0.4 ± 0.5 |
| opencode | shell | 165.2 ± 13.0 | 0.2 ± 0.4 | <1% | 0.2 ± 0.4 |
| wisp | python3 | 3.4 ± 5.0 | 0.2 ± 0.4 | 6% | 0.2 ± 0.4 |
| wisp | bash | 177 | 24 | 14% | 7 |
| kmet | bash | 122 | 18 | 15% | 8 |
| pi | bash | 162 | 17 | 10% | 9 |
| kmet-eval-only | eval | 201 | 14 | 7% | 9 |
| kmet-shell-only | bash | 129 | 13 | 10% | 6 |
| kmet | read | 88 | 11 | 12% | 5 |
| vis-solo | python_execution | 269 | 11 | 4% | 6 |
| kmet-codemode | codemode | 181 | 9 | 5% | 8 |
| kmet | edit | 71 | 5 | 7% | 3 |
| codex | exec | 262 | 3 | 1% | 3 |
| wisp | python3 | 12 | 1 | 8% | 1 |
| wisp | bash | 194 | 18 | 9% | 8 |
| pi | bash | 148 | 13 | 9% | 8 |
| codex | exec | 242 | 4 | 2% | 4 |
| opencode | patch | 70 | 1 | 1% | 1 |
| opencode | read | 69 | 1 | 1% | 1 |
| pi | edit | 77 | 1 | 1% | 1 |
| wisp | bash | 235 | 35 | 15% | 8 |
| pi | bash | 158 | 20 | 13% | 9 |
| codex | exec | 224 | 1 | <1% | 1 |
| opencode | read | 45 | 1 | 2% | 1 |
| pi | edit | 87 | 1 | 1% | 1 |
| pi | bash | 150 | 31 | 21% | 9 |
| wisp | bash | 195 | 19 | 10% | 8 |
| codex | exec | 229 | 1 | <1% | 1 |
| opencode | patch | 66 | 1 | 2% | 1 |
| opencode | read | 65 | 1 | 2% | 1 |
| opencode | shell | 181 | 1 | <1% | 1 |
| wisp | bash | 207 | 25 | 12% | 8 |
| pi | bash | 161 | 13 | 8% | 8 |
| codex | exec | 197 | 1 | <1% | 1 |
| opencode | patch | 65 | 1 | 2% | 1 |
| opencode | read | 52 | 1 | 2% | 1 |
Trials
300 trials
Clear filtersNo trials match these filters.
| Rounds | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| bun-sourcemap-leak | codex | 0/5 | 1 2 3 4 5 | 6 min 57 s ± 1 min 45 s | $0.1715 ± $0.0262 | 13.0 ± 2.8 | 5 min 07 s ± 1 min 38 s | 12.0 ± 2.8 | 0.0 ± 0.0 | 0.00 ± 0.00 | gpt-6.1-sol |
| bun-sourcemap-leak | kmet | 0/1 | 1 | 4 min 12 s | $0.1198 | 14.0 | 3 min 16 s | 22.0 | 3.0 | 0.00 | gpt-6.1-sol |
| bun-sourcemap-leak | kmet-codemode | 0/1 | 1 | 4 min 47 s | $0.1286 | 9.0 | 3 min 53 s | 8.0 | 0.0 | 0.00 | gpt-6.1-sol |
| bun-sourcemap-leak | kmet-eval-only | 0/1 | 1 | 4 min 01 s | $0.1215 | 13.0 | 3 min 05 s | 12.0 | 0.0 | 0.00 | gpt-6.1-sol |
| bun-sourcemap-leak | kmet-shell-only | 0/1 | 1 | 4 min 52 s | $0.1132 | 11.0 | 3 min 57 s | 10.0 | 1.0 | 0.00 | gpt-6.1-sol |
| bun-sourcemap-leak | opencode | 2/5 | 1 2 3 4 5 | 8 min 41 s ± 3 min 39 s | $0.2042 ± $0.0736 | 22.0 ± 7.9 | 5 min 26 s ± 3 min 07 s | 25.6 ± 10.6 | 0.0 ± 0.0 | 0.40 ± 0.55 | gpt-6.1-sol |
| bun-sourcemap-leak | pi | 1/5 | 1 2 3 4 5 | 6 min 03 s ± 1 min 45 s | $0.1371 ± $0.0601 | 18.6 ± 4.5 | 4 min 45 s ± 1 min 46 s | 27.2 ± 4.5 | 3.8 ± 3.5 | 0.20 ± 0.45 | gpt-6.1-sol |
| bun-sourcemap-leak | vis-solo | 1/1 | 1 | 20 min 06 s | $0.4572 | 28.0 | 15 min 41 s | 27.0 | 0.0 | 1.00 | gpt-6.1-sol |
| bun-sourcemap-leak | wisp | 2/5 | 1 2 3 4 5 | 9 min 54 s ± 5 min 19 s | $0.2799 ± $0.1853 | 22.2 ± 13.1 | 8 min 14 s ± 5 min 15 s | 20.8 ± 12.6 | 5.4 ± 4.2 | 0.40 ± 0.55 | gpt-6.1-sol |
| html-js-filter | codex | 3/5 | 1 2 3 4 5 | 9 min 16 s ± 1 min 04 s | $0.1535 ± $0.0066 | 10.6 ± 1.9 | 5 min 28 s ± 57.4 s | 9.6 ± 1.9 | 0.2 ± 0.4 | 0.60 ± 0.55 | gpt-6.1-sol |
| html-js-filter | kmet | 1/1 | 1 | 5 min 24 s | $0.1039 | 11.0 | 2 min 43 s | 10.0 | 1.0 | 1.00 | gpt-6.1-sol |
| html-js-filter | kmet-codemode | 1/1 | 1 | 21 min 41 s | $0.0981 | 9.0 | 18 min 11 s | 8.0 | 1.0 | 1.00 | gpt-6.1-sol |
| html-js-filter | kmet-eval-only | 1/1 | 1 | 8 min 23 s | $0.1403 | 12.0 | 5 min 27 s | 11.0 | 0.0 | 1.00 | gpt-6.1-sol |
| html-js-filter | kmet-shell-only | 0/1 | 1 | 6 min 02 s | $0.0903 | 9.0 | 2 min 35 s | 8.0 | 0.0 | 0.00 | gpt-6.1-sol |
| html-js-filter | opencode | 5/5 | 1 2 3 4 5 | 8 min 24 s ± 46.9 s | $0.1248 ± $0.0086 | 11.0 ± 1.0 | 3 min 26 s ± 28.6 s | 10.0 ± 1.0 | 0.0 ± 0.0 | 1.00 ± 0.00 | gpt-6.1-sol |
| html-js-filter | pi | 3/5 | 1 2 3 4 5 | 8 min 47 s ± 47.0 s | $0.1203 ± $0.0106 | 12.4 ± 1.1 | 5 min 24 s ± 53.2 s | 11.2 ± 1.1 | 0.2 ± 0.4 | 0.60 ± 0.55 | gpt-6.1-sol |
| html-js-filter | vis-solo | 0/1 | 1 | 12 min 16 s | $0.1932 | 19.0 | 5 min 38 s | 18.0 | 2.0 | 0.00 | gpt-6.1-sol |
| html-js-filter | wisp | 2/5 | 1 2 3 4 5 | 8 min 10 s ± 1 min 46 s | $0.1416 ± $0.0506 | 11.8 ± 5.8 | 5 min 14 s ± 1 min 47 s | 10.6 ± 5.3 | 0.2 ± 0.4 | 0.40 ± 0.55 | gpt-6.1-sol |
| interleaved-vigenere | codex | 5/5 | 1 2 3 4 5 | 8 min 05 s ± 2 min 12 s | $0.2407 ± $0.0567 | 21.8 ± 4.0 | 5 min 57 s ± 1 min 59 s | 20.8 ± 4.0 | 0.8 ± 0.4 | 1.00 ± 0.00 | gpt-6.1-sol |
| interleaved-vigenere | kmet | 1/1 | 1 | 5 min 28 s | $0.1839 | 22.0 | 4 min 19 s | 24.0 | 0.0 | 1.00 | gpt-6.1-sol |
| interleaved-vigenere | kmet-codemode | 1/1 | 1 | 8 min 21 s | $0.2095 | 25.0 | 7 min 02 s | 24.0 | 1.0 | 1.00 | gpt-6.1-sol |
| interleaved-vigenere | kmet-eval-only | 1/1 | 1 | 6 min 24 s | $0.1646 | 20.0 | 4 min 50 s | 19.0 | 1.0 | 1.00 | gpt-6.1-sol |
| interleaved-vigenere | kmet-shell-only | 1/1 | 1 | 4 min 44 s | $0.1773 | 15.0 | 3 min 22 s | 14.0 | 0.0 | 1.00 | gpt-6.1-sol |
| interleaved-vigenere | opencode | 4/5 | 1 2 3 4 5 | 8 min 00 s ± 2 min 37 s | $0.1953 ± $0.0689 | 22.8 ± 4.8 | 4 min 33 s ± 1 min 33 s | 22.6 ± 5.7 | 0.0 ± 0.0 | 0.80 ± 0.45 | gpt-6.1-sol |
| interleaved-vigenere | pi | 5/5 | 1 2 3 4 5 | 7 min 00 s ± 2 min 44 s | $0.1780 ± $0.0926 | 22.6 ± 6.8 | 5 min 13 s ± 2 min 36 s | 23.2 ± 7.0 | 0.0 ± 0.0 | 1.00 ± 0.00 | gpt-6.1-sol |
| interleaved-vigenere | vis-solo | 1/1 | 1 | 7 min 38 s | $0.1653 | 23.0 | 4 min 42 s | 22.0 | 1.0 | 1.00 | gpt-6.1-sol |
| interleaved-vigenere | wisp | 5/5 | 1 2 3 4 5 | 7 min 13 s ± 4 min 43 s | $0.2091 ± $0.1369 | 23.2 ± 14.0 | 5 min 55 s ± 4 min 29 s | 22.0 ± 13.6 | 0.8 ± 1.3 | 1.00 ± 0.00 | gpt-6.1-sol |
| kv-live-surgery | codex | 3/5 | 1 2 3 4 5 | 21 min 51 s ± 4 min 56 s | $1.02 ± $0.1627 | 56.8 ± 7.3 | 17 min 26 s ± 4 min 33 s | 55.8 ± 7.3 | 0.0 ± 0.0 | 0.60 ± 0.55 | gpt-6.1-sol |
| kv-live-surgery | kmet | 0/1 | 1 | 38 min 39 s | $0.3874 | 36.0 | 36 min 08 s | 35.0 | 2.0 | 0.00 | gpt-6.1-sol |
| kv-live-surgery | kmet-codemode | 0/1 | 1 | 10 min 25 s | $0.5997 | 36.0 | 6 min 54 s | 35.0 | 0.0 | 0.00 | gpt-6.1-sol |
| kv-live-surgery | kmet-eval-only | 0/1 | 1 | 10 min 43 s | $0.4144 | 35.0 | 7 min 08 s | 34.0 | 3.0 | 0.00 | gpt-6.1-sol |
| kv-live-surgery | kmet-shell-only | 0/1 | 1 | 6 min 08 s | $0.2976 | 19.0 | 2 min 55 s | 18.0 | 3.0 | 0.00 | gpt-6.1-sol |
| kv-live-surgery | opencode | 0/5 | 1 2 3 4 5 | 11 min 53 s ± 2 min 06 s | $0.4483 ± $0.0603 | 33.2 ± 6.5 | 6 min 13 s ± 1 min 17 s | 32.2 ± 6.5 | 0.8 ± 0.4 | 0.00 ± 0.00 | gpt-6.1-sol |
| kv-live-surgery | pi | 1/5 | 1 2 3 4 5 | 11 min 39 s ± 4 min 58 s | $0.5556 ± $0.2694 | 39.4 ± 16.8 | 7 min 57 s ± 4 min 09 s | 38.4 ± 16.8 | 1.2 ± 0.4 | 0.20 ± 0.45 | gpt-6.1-sol |
| kv-live-surgery | vis-solo | 0/1 | 1 | 8 min 45 s | $0.2791 | 33.0 | 5 min 43 s | 32.0 | 5.0 | 0.00 | gpt-6.1-sol |
| kv-live-surgery | wisp | 1/5 | 1 2 3 4 5 | 10 min 15 s ± 4 min 12 s | $0.3589 ± $0.1227 | 29.2 ± 11.3 | 7 min 15 s ± 3 min 29 s | 28.2 ± 11.3 | 2.0 ± 1.4 | 0.20 ± 0.45 | gpt-6.1-sol |
| music-harmony | codex | 0/5 | 1 2 3 4 5 | 7 min 07 s ± 1 min 08 s | $0.1732 ± $0.0388 | 12.4 ± 3.2 | 4 min 55 s ± 48.3 s | 11.4 ± 3.2 | 0.0 ± 0.0 | 0.00 ± 0.00 | gpt-6.1-sol |
| music-harmony | kmet | 0/1 | 1 | 4 min 33 s | $0.1440 | 16.0 | 3 min 39 s | 15.0 | 2.0 | 0.00 | gpt-6.1-sol |
| music-harmony | kmet-codemode | 0/1 | 1 | 4 min 58 s | $0.1483 | 17.0 | 4 min 03 s | 16.0 | 1.0 | 0.00 | gpt-6.1-sol |
| music-harmony | kmet-eval-only | 0/1 | 1 | 6 min 16 s | $0.1517 | 12.0 | 5 min 21 s | 11.0 | 0.0 | 0.00 | gpt-6.1-sol |
| music-harmony | kmet-shell-only | 0/1 | 1 | 4 min 04 s | $0.1744 | 15.0 | 3 min 07 s | 14.0 | 3.0 | 0.00 | gpt-6.1-sol |
| music-harmony | opencode | 2/5 | 1 2 3 4 5 | 5 min 38 s ± 32.0 s | $0.1203 ± $0.0254 | 12.0 ± 3.1 | 2 min 45 s ± 29.1 s | 11.0 ± 3.1 | 0.0 ± 0.0 | 0.40 ± 0.55 | gpt-6.1-sol |
| music-harmony | pi | 1/5 | 1 2 3 4 5 | 5 min 00 s ± 40.2 s | $0.1253 ± $0.0136 | 17.8 ± 5.7 | 3 min 42 s ± 39.2 s | 16.8 ± 5.7 | 2.8 ± 1.9 | 0.20 ± 0.45 | gpt-6.1-sol |
| music-harmony | vis-solo | 0/1 | 1 | 4 min 51 s | $0.1008 | 12.0 | 2 min 45 s | 11.0 | 0.0 | 0.00 | gpt-6.1-sol |
| music-harmony | wisp | 0/5 | 1 2 3 4 5 | 3 min 58 s ± 40.1 s | $0.1036 ± $0.0246 | 11.0 ± 3.8 | 2 min 57 s ± 41.1 s | 10.0 ± 3.8 | 1.4 ± 0.5 | 0.00 ± 0.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | codex | 3/5 | 1 2 3 4 5 | 6 min 19 s ± 16.9 s | $0.0751 ± $0.0156 | 6.6 ± 0.9 | 1 min 31 s ± 8.9 s | 5.6 ± 0.9 | 0.0 ± 0.0 | 0.60 ± 0.55 | gpt-6.1-sol |
| mvcc-lsm-compaction | kmet | 0/1 | 1 | 5 min 10 s | $0.0814 | 13.0 | 1 min 43 s | 24.0 | 3.0 | 0.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | kmet-codemode | 0/1 | 1 | 4 min 44 s | $0.0730 | 8.0 | 1 min 19 s | 7.0 | 1.0 | 0.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | kmet-eval-only | 1/1 | 1 | 4 min 50 s | $0.0737 | 9.0 | 1 min 20 s | 8.0 | 1.0 | 1.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | kmet-shell-only | 1/1 | 1 | 4 min 14 s | $0.0396 | 5.0 | 48.7 s | 4.0 | 0.0 | 1.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | opencode | 5/5 | 1 2 3 4 5 | 6 min 38 s ± 43.8 s | $0.0712 ± $0.0145 | 10.6 ± 2.1 | 59.0 s ± 11.4 s | 12.4 ± 3.5 | 0.0 ± 0.0 | 1.00 ± 0.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | pi | 3/5 | 1 2 3 4 5 | 5 min 38 s ± 12.7 s | $0.0695 ± $0.0063 | 11.4 ± 0.5 | 1 min 30 s ± 8.1 s | 22.2 ± 0.4 | 1.0 ± 0.0 | 0.60 ± 0.55 | gpt-6.1-sol |
| mvcc-lsm-compaction | vis-solo | 1/1 | 1 | 7 min 31 s | $0.0786 | 9.0 | 1 min 33 s | 8.0 | 0.0 | 1.00 | gpt-6.1-sol |
| mvcc-lsm-compaction | wisp | 2/5 | 1 2 3 4 5 | 5 min 25 s ± 28.4 s | $0.0581 ± $0.0174 | 7.0 ± 1.2 | 1 min 23 s ± 8.1 s | 6.0 ± 1.2 | 1.4 ± 0.9 | 0.40 ± 0.55 | gpt-6.1-sol |
| payments-pipeline-fix | codex | 4/5 | 1 2 3 4 5 | 18 min 31 s ± 3 min 13 s | $0.4794 ± $0.1662 | 40.4 ± 12.3 | 13 min 42 s ± 4 min 07 s | 39.4 ± 12.3 | 0.0 ± 0.0 | 0.80 ± 0.45 | gpt-6.1-sol |
| payments-pipeline-fix | kmet | 0/1 | 1 | 11 min 16 s | $0.2504 | 25.0 | 7 min 33 s | 29.0 | 0.0 | 0.00 | gpt-6.1-sol |
| payments-pipeline-fix | kmet-codemode | 0/1 | 1 | 17 min 57 s | $0.2171 | 22.0 | 13 min 39 s | 19.0 | 0.0 | not recorded | gpt-6.1-sol |
| payments-pipeline-fix | kmet-eval-only | 1/1 | 1 | 11 min 24 s | $0.2550 | 25.0 | 6 min 10 s | 24.0 | 2.0 | 1.00 | gpt-6.1-sol |
| payments-pipeline-fix | kmet-shell-only | 0/1 | 1 | 26 min 51 s | $0.2302 | 18.0 | 20 min 57 s | 17.0 | 1.0 | 0.00 | gpt-6.1-sol |
| payments-pipeline-fix | opencode | 0/5 | 1 2 3 4 5 | 24 min 34 s ± 5 min 15 s | $0.6110 ± $0.1197 | 51.2 ± 10.5 | 13 min 10 s ± 3 min 48 s | 61.2 ± 12.2 | 0.2 ± 0.4 | 0.00 ± 0.00 | gpt-6.1-sol |
| payments-pipeline-fix | pi | 3/5 | 1 2 3 4 5 | 20 min 16 s ± 7 min 14 s | $0.4401 ± $0.1743 | 47.4 ± 15.9 | 14 min 51 s ± 6 min 24 s | 53.6 ± 17.0 | 3.8 ± 1.8 | 0.60 ± 0.55 | gpt-6.1-sol |
| payments-pipeline-fix | vis-solo | 1/1 | 1 | 26 min 38 s | $0.6621 | 53.0 | 18 min 45 s | 52.0 | 1.0 | 1.00 | gpt-6.1-sol |
| payments-pipeline-fix | wisp | 2/5 | 1 2 3 4 5 | 19 min 27 s ± 2 min 31 s | $0.4805 ± $0.0702 | 38.0 ± 6.9 | 13 min 43 s ± 2 min 16 s | 36.4 ± 6.4 | 4.2 ± 1.8 | 0.40 ± 0.55 | gpt-6.1-sol |
| photonic-waveguide-routing | codex | 5/5 | 1 2 3 4 5 | 19 min 25 s ± 5 min 29 s | $0.4078 ± $0.1418 | 19.2 ± 6.8 | 16 min 57 s ± 5 min 20 s | 18.2 ± 6.8 | 0.4 ± 0.5 | 1.00 ± 0.00 | gpt-6.1-sol |
| photonic-waveguide-routing | kmet | 1/1 | 1 | 14 min 18 s | $0.3501 | 27.0 | 11 min 43 s | 26.0 | 0.0 | 1.00 | gpt-6.1-sol |
| photonic-waveguide-routing | kmet-codemode | 1/1 | 1 | 14 min 30 s | $0.2247 | 13.0 | 8 min 03 s | 12.0 | 0.0 | 1.00 | gpt-6.1-sol |
| photonic-waveguide-routing | kmet-eval-only | 1/1 | 1 | 15 min 40 s | $0.3806 | 26.0 | 12 min 28 s | 25.0 | 1.0 | 1.00 | gpt-6.1-sol |
| photonic-waveguide-routing | kmet-shell-only | 1/1 | 1 | 7 min 51 s | $0.1449 | 9.0 | 5 min 47 s | 8.0 | 0.0 | 1.00 | gpt-6.1-sol |
| photonic-waveguide-routing | opencode | 5/5 | 1 2 3 4 5 | 26 min 02 s ± 4 min 25 s | $0.4336 ± $0.0666 | 16.2 ± 2.4 | 21 min 33 s ± 4 min 08 s | 15.6 ± 2.1 | 0.0 ± 0.0 | 1.00 ± 0.00 | gpt-6.1-sol |
| photonic-waveguide-routing | pi | 5/5 | 1 2 3 4 5 | 25 min 29 s ± 7 min 02 s | $0.4800 ± $0.1254 | 34.4 ± 10.6 | 18 min 56 s ± 5 min 38 s | 33.4 ± 10.6 | 1.2 ± 1.6 | 1.00 ± 0.00 | gpt-6.1-sol |
| photonic-waveguide-routing | vis-solo | 1/1 | 1 | 32 min 10 s | $0.6379 | 26.0 | 27 min 24 s | 25.0 | 1.0 | 1.00 | gpt-6.1-sol |
| photonic-waveguide-routing | wisp | 5/5 | 1 2 3 4 5 | 18 min 44 s ± 4 min 19 s | $0.4048 ± $0.1315 | 20.4 ± 6.2 | 16 min 03 s ± 4 min 17 s | 19.2 ± 6.3 | 0.2 ± 0.4 | 1.00 ± 0.00 | gpt-6.1-sol |
| react-lead-form | codex | 2/5 | 1 2 3 4 5 | 8 min 24 s ± 56.6 s | $0.2591 ± $0.0335 | 15.8 ± 2.6 | 6 min 42 s ± 58.5 s | 14.8 ± 2.6 | 0.0 ± 0.0 | 0.40 ± 0.55 | gpt-6.1-sol |
| react-lead-form | kmet | 1/1 | 1 | 7 min 05 s | $0.2585 | 35.0 | 5 min 50 s | 44.0 | 8.0 | 1.00 | gpt-6.1-sol |
| react-lead-form | kmet-codemode | 0/1 | 1 | 5 min 58 s | $0.2040 | 13.0 | 4 min 36 s | 12.0 | 1.0 | 0.00 | gpt-6.1-sol |
| react-lead-form | kmet-eval-only | 1/1 | 1 | 5 min 32 s | $0.1818 | 15.0 | 4 min 17 s | 14.0 | 1.0 | 1.00 | gpt-6.1-sol |
| react-lead-form | kmet-shell-only | 1/1 | 1 | 4 min 33 s | $0.1319 | 10.0 | 3 min 20 s | 9.0 | 0.0 | 1.00 | gpt-6.1-sol |
| react-lead-form | opencode | 3/5 | 1 2 3 4 5 | 10 min 12 s ± 1 min 41 s | $0.2807 ± $0.0568 | 22.2 ± 3.3 | 4 min 29 s ± 49.8 s | 38.6 ± 3.6 | 0.4 ± 0.5 | 0.60 ± 0.55 | gpt-6.1-sol |
| react-lead-form | pi | 2/5 | 1 2 3 4 5 | 11 min 02 s ± 1 min 36 s | $0.3131 ± $0.0427 | 35.6 ± 5.0 | 9 min 19 s ± 1 min 42 s | 45.6 ± 4.9 | 2.2 ± 0.8 | 0.40 ± 0.55 | gpt-6.1-sol |
| react-lead-form | vis-solo | 0/1 | 1 | 17 min 22 s | $0.4947 | 26.0 | 13 min 29 s | 25.0 | 1.0 | 0.00 | gpt-6.1-sol |
| react-lead-form | wisp | 2/5 | 1 2 3 4 5 | 7 min 24 s ± 1 min 08 s | $0.2176 ± $0.0455 | 14.6 ± 2.4 | 5 min 58 s ± 1 min 07 s | 13.6 ± 2.4 | 3.2 ± 1.1 | 0.40 ± 0.55 | gpt-6.1-sol |
| session-window-debug | codex | 1/5 | 1 2 3 4 5 | 6 min 03 s ± 19.9 s | $0.1611 ± $0.0275 | 11.6 ± 1.5 | 4 min 10 s ± 24.0 s | 10.6 ± 1.5 | 0.2 ± 0.4 | 0.20 ± 0.45 | gpt-6.1-sol |
| session-window-debug | kmet | 0/1 | 1 | 3 min 44 s | $0.0988 | 16.0 | 2 min 51 s | 21.0 | 6.0 | 0.00 | gpt-6.1-sol |
| session-window-debug | kmet-codemode | 1/1 | 1 | 3 min 44 s | $0.0984 | 11.0 | 2 min 53 s | 10.0 | 2.0 | 1.00 | gpt-6.1-sol |
| session-window-debug | kmet-eval-only | 0/1 | 1 | 3 min 27 s | $0.0968 | 11.0 | 2 min 34 s | 10.0 | 1.0 | 0.00 | gpt-6.1-sol |
| session-window-debug | kmet-shell-only | 0/1 | 1 | 2 min 11 s | $0.0545 | 5.0 | 1 min 18 s | 4.0 | 0.0 | 0.00 | gpt-6.1-sol |
| session-window-debug | opencode | 3/5 | 1 2 3 4 5 | 4 min 51 s ± 1 min 30 s | $0.0960 ± $0.0460 | 10.6 ± 2.6 | 1 min 57 s ± 56.5 s | 16.6 ± 4.3 | 0.0 ± 0.0 | 0.60 ± 0.55 | gpt-6.1-sol |
| session-window-debug | pi | 1/5 | 1 2 3 4 5 | 5 min 06 s ± 35.4 s | $0.1201 ± $0.0281 | 16.2 ± 1.6 | 3 min 53 s ± 37.2 s | 21.8 ± 2.8 | 0.8 ± 0.4 | 0.20 ± 0.45 | gpt-6.1-sol |
| session-window-debug | vis-solo | 1/1 | 1 | 7 min 08 s | $0.1949 | 16.0 | 5 min 10 s | 15.0 | 0.0 | 1.00 | gpt-6.1-sol |
| session-window-debug | wisp | 1/5 | 1 2 3 4 5 | 3 min 55 s ± 46.0 s | $0.1134 ± $0.0315 | 10.8 ± 3.0 | 3 min 03 s ± 46.7 s | 9.2 ± 2.5 | 0.6 ± 0.9 | 0.20 ± 0.45 | gpt-6.1-sol |
| vf2-speedup-networkx | codex | 0/5 | 1 2 3 4 5 | 11 min 44 s ± 2 min 43 s | $0.3575 ± $0.1076 | 27.8 ± 5.2 | 9 min 14 s ± 2 min 29 s | 26.8 ± 5.2 | 0.2 ± 0.4 | 0.00 ± 0.00 | gpt-6.1-sol |
| vf2-speedup-networkx | kmet | 1/1 | 1 | 10 min 24 s | $0.2983 | 32.0 | 8 min 10 s | 31.0 | 7.0 | 1.00 | gpt-6.1-sol |
| vf2-speedup-networkx | kmet-codemode | 0/1 | 1 | 10 min 44 s | $0.2733 | 24.0 | 7 min 19 s | 23.0 | 1.0 | 0.00 | gpt-6.1-sol |
| vf2-speedup-networkx | kmet-eval-only | 0/1 | 1 | 11 min 49 s | $0.2833 | 24.0 | 6 min 32 s | 23.0 | 3.0 | 0.00 | gpt-6.1-sol |
| vf2-speedup-networkx | kmet-shell-only | 0/1 | 1 | 7 min 31 s | $0.1708 | 19.0 | 4 min 18 s | 18.0 | 4.0 | 0.00 | gpt-6.1-sol |
| vf2-speedup-networkx | opencode | 3/5 | 1 2 3 4 5 | 17 min 25 s ± 5 min 06 s | not recorded | 38.0 ± 14.3 | 9 min 52 s ± 4 min 21 s | 42.2 ± 16.3 | 0.2 ± 0.4 | 0.60 ± 0.55 | gpt-6.1-sol |
| vf2-speedup-networkx | pi | 3/5 | 1 2 3 4 5 | 16 min 35 s ± 1 min 51 s | $0.3871 ± $0.0644 | 37.8 ± 3.0 | 12 min 12 s ± 1 min 19 s | 36.8 ± 3.0 | 2.0 ± 1.7 | 0.60 ± 0.55 | gpt-6.1-sol |
| vf2-speedup-networkx | vis-solo | 0/1 | 1 | 15 min 29 s | $0.2496 | 23.0 | 7 min 45 s | 22.0 | 0.0 | 0.00 | gpt-6.1-sol |
| vf2-speedup-networkx | wisp | 0/5 | 1 2 3 4 5 | 13 min 48 s ± 5 min 04 s | $0.2819 ± $0.1120 | 27.2 ± 10.9 | 8 min 05 s ± 3 min 22 s | 25.8 ± 10.1 | 5.0 ± 2.9 | 0.00 ± 0.00 | gpt-6.1-sol |
| wal-recovery-ordering | codex | 4/5 | 1 2 3 4 5 | 8 min 01 s ± 1 min 41 s | $0.1458 ± $0.0222 | 8.2 ± 1.1 | 3 min 11 s ± 34.1 s | 7.2 ± 1.1 | 0.2 ± 0.4 | 0.80 ± 0.45 | gpt-6.1-sol |
| wal-recovery-ordering | kmet | 1/1 | 1 | 7 min 19 s | $0.1111 | 14.0 | 2 min 59 s | 28.0 | 2.0 | 1.00 | gpt-6.1-sol |
| wal-recovery-ordering | kmet-codemode | 1/1 | 1 | 6 min 21 s | $0.1042 | 8.0 | 2 min 03 s | 7.0 | 1.0 | 1.00 | gpt-6.1-sol |
| wal-recovery-ordering | kmet-eval-only | 1/1 | 1 | 7 min 12 s | $0.1038 | 11.0 | 2 min 56 s | 10.0 | 1.0 | 1.00 | gpt-6.1-sol |
| wal-recovery-ordering | kmet-shell-only | 0/1 | 1 | 2 min 53 s | $0.0661 | 6.0 | 1 min 30 s | 5.0 | 1.0 | 0.00 | gpt-6.1-sol |
| wal-recovery-ordering | opencode | 4/5 | 1 2 3 4 5 | 7 min 51 s ± 1 min 49 s | $0.0988 ± $0.0099 | 8.8 ± 0.4 | 1 min 34 s ± 17.3 s | 8.0 ± 0.7 | 0.0 ± 0.0 | 0.80 ± 0.45 | gpt-6.1-sol |
| wal-recovery-ordering | pi | 4/5 | 1 2 3 4 5 | 6 min 47 s ± 1 min 34 s | $0.1155 ± $0.0186 | 17.0 ± 2.5 | 3 min 06 s ± 24.3 s | 30.0 ± 1.2 | 0.2 ± 0.4 | 0.80 ± 0.45 | gpt-6.1-sol |
| wal-recovery-ordering | vis-solo | 1/1 | 1 | 9 min 33 s | $0.1594 | 13.0 | 3 min 47 s | 12.0 | 0.0 | 1.00 | gpt-6.1-sol |
| wal-recovery-ordering | wisp | 2/5 | 1 2 3 4 5 | 4 min 12 s ± 1 min 24 s | $0.0843 ± $0.0102 | 6.2 ± 0.8 | 1 min 54 s ± 14.1 s | 5.2 ± 0.8 | 0.0 ± 0.0 | 0.40 ± 0.55 | gpt-6.1-sol |
| bun-sourcemap-leak | codex | Failed | 5 min 32 s | $0.1596 | 16 | 3 min 48 s | 15 | 0 | 0 | 3 min 53 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:47 UTC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| html-js-filter | codex | Passed | 8 min 47 s | $0.1482 | 10 | 5 min 14 s | 9 | 0 | 1 | 5 min 21 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 19:41 UTC |
| interleaved-vigenere | codex | Passed | 7 min 47 s | $0.2562 | 22 | 5 min 29 s | 21 | 1 | 1 | 5 min 58 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 22:04 UTC |
| kv-live-surgery | codex | Passed | 26 min 24 s | $1.18 | 64 | 21 min 48 s | 63 | 0 | 1 | 24 min 07 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 23:47 UTC |
| music-harmony | codex | Failed | 8 min 27 s | $0.2270 | 17 | 6 min 02 s | 16 | 0 | 0 | 6 min 11 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:28 UTC |
| mvcc-lsm-compaction | codex | Failed | 6 min 08 s | $0.0566 | 6 | 1 min 21 s | 5 | 0 | 0 | 1 min 34 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 23:29 UTC |
| payments-pipeline-fix | codex | Passed | 19 min 46 s | $0.6376 | 51 | 15 min 07 s | 50 | 0 | 1 | 16 min 06 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 22:22 UTC |
| photonic-waveguide-routing | codex | Passed | 23 min 06 s | $0.5350 | 23 | 20 min 10 s | 22 | 1 | 1 | 20 min 48 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 20:08 UTC |
| react-lead-form | codex | Passed | 7 min 43 s | $0.2711 | 17 | 6 min 07 s | 16 | 0 | 1 | 6 min 25 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 01:24 UTC |
| session-window-debug | codex | Failed | 5 min 53 s | $0.1498 | 13 | 4 min 05 s | 12 | 1 | 0 | 4 min 10 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 02:14 UTC |
| vf2-speedup-networkx | codex | Failed | 13 min 41 s | $0.4179 | 29 | 11 min 00 s | 28 | 0 | 0 | 11 min 52 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 00:45 UTC |
| wal-recovery-ordering | codex | Passed | 8 min 14 s | $0.1529 | 8 | 3 min 10 s | 7 | 0 | 1 | 3 min 16 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 01:50 UTC |
| bun-sourcemap-leak | kmet | Failed | 4 min 12 s | $0.1198 | 14 | 3 min 16 s | 22 | 3 | 0 | 3 min 20 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:35 UTC |
| html-js-filter | kmet | Passed | 5 min 24 s | $0.1039 | 11 | 2 min 43 s | 10 | 1 | 1 | 2 min 46 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 22:41 UTC |
| interleaved-vigenere | kmet | Passed | 5 min 28 s | $0.1839 | 22 | 4 min 19 s | 24 | 0 | 1 | 4 min 29 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:53 UTC |
| kv-live-surgery | kmet | Failed | 38 min 39 s | $0.3874 | 36 | 36 min 08 s | 35 | 2 | 0 | 37 min 04 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 02:44 UTC |
| music-harmony | kmet | Failed | 4 min 33 s | $0.1440 | 16 | 3 min 39 s | 15 | 2 | 0 | 3 min 44 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:15 UTC |
| mvcc-lsm-compaction | kmet | Failed | 5 min 10 s | $0.0814 | 13 | 1 min 43 s | 24 | 3 | 0 | 1 min 54 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 02:25 UTC |
| payments-pipeline-fix | kmet | Failed | 11 min 16 s | $0.2504 | 25 | 7 min 33 s | 29 | 0 | 0 | 8 min 56 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 01:18 UTC |
| photonic-waveguide-routing | kmet | Passed | 14 min 18 s | $0.3501 | 27 | 11 min 43 s | 26 | 0 | 1 | 13 min 04 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 23:23 UTC |
| react-lead-form | kmet | Passed | 7 min 05 s | $0.2585 | 35 | 5 min 50 s | 44 | 8 | 1 | 6 min 02 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:31 UTC |
| session-window-debug | kmet | Failed | 3 min 44 s | $0.0988 | 16 | 2 min 51 s | 21 | 6 | 0 | 2 min 54 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:18 UTC |
| vf2-speedup-networkx | kmet | Passed | 10 min 24 s | $0.2983 | 32 | 8 min 10 s | 31 | 7 | 1 | 9 min 18 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 03:50 UTC |
| wal-recovery-ordering | kmet | Passed | 7 min 19 s | $0.1111 | 14 | 2 min 59 s | 28 | 2 | 1 | 3 min 03 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:54 UTC |
| bun-sourcemap-leak | kmet-codemode | Failed | 4 min 47 s | $0.1286 | 9 | 3 min 53 s | 8 | 0 | 0 | 3 min 56 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:43 UTC |
| html-js-filter | kmet-codemode | Passed | 21 min 41 s | $0.0981 | 9 | 18 min 11 s | 8 | 1 | 1 | 18 min 14 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 22:55 UTC |
| interleaved-vigenere | kmet-codemode | Passed | 8 min 21 s | $0.2095 | 25 | 7 min 02 s | 24 | 1 | 1 | 7 min 27 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 01:05 UTC |
| kv-live-surgery | kmet-codemode | Failed | 10 min 25 s | $0.5997 | 36 | 6 min 54 s | 35 | 0 | 0 | 8 min 50 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 03:34 UTC |
| music-harmony | kmet-codemode | Failed | 4 min 58 s | $0.1483 | 17 | 4 min 03 s | 16 | 1 | 0 | 4 min 10 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:26 UTC |
| mvcc-lsm-compaction | kmet-codemode | Failed | 4 min 44 s | $0.0730 | 8 | 1 min 19 s | 7 | 1 | 0 | 1 min 30 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 02:35 UTC |
| payments-pipeline-fix | kmet-codemode | Error | 17 min 57 s | $0.2171 | 22 | 13 min 39 s | 19 | 0 | not recorded | 14 min 56 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 01:41 UTC |
| photonic-waveguide-routing | kmet-codemode | Passed | 14 min 30 s | $0.2247 | 13 | 8 min 03 s | 12 | 0 | 1 | 13 min 10 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 23:53 UTC |
| react-lead-form | kmet-codemode | Failed | 5 min 58 s | $0.2040 | 13 | 4 min 36 s | 12 | 1 | 0 | 4 min 57 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:44 UTC |
| session-window-debug | kmet-codemode | Passed | 3 min 44 s | $0.0984 | 11 | 2 min 53 s | 10 | 2 | 1 | 2 min 55 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:25 UTC |
| vf2-speedup-networkx | kmet-codemode | Failed | 10 min 44 s | $0.2733 | 24 | 7 min 19 s | 23 | 1 | 0 | 9 min 00 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:13 UTC |
| wal-recovery-ordering | kmet-codemode | Passed | 6 min 21 s | $0.1042 | 8 | 2 min 03 s | 7 | 1 | 1 | 2 min 06 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:09 UTC |
| bun-sourcemap-leak | kmet-eval-only | Failed | 4 min 01 s | $0.1215 | 13 | 3 min 05 s | 12 | 0 | 0 | 3 min 09 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:39 UTC |
| html-js-filter | kmet-eval-only | Passed | 8 min 23 s | $0.1403 | 12 | 5 min 27 s | 11 | 0 | 1 | 5 min 30 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 22:46 UTC |
| interleaved-vigenere | kmet-eval-only | Passed | 6 min 24 s | $0.1646 | 20 | 4 min 50 s | 19 | 1 | 1 | 5 min 14 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:58 UTC |
| kv-live-surgery | kmet-eval-only | Failed | 10 min 43 s | $0.4144 | 35 | 7 min 08 s | 34 | 3 | 0 | 9 min 08 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 03:23 UTC |
| music-harmony | kmet-eval-only | Failed | 6 min 16 s | $0.1517 | 12 | 5 min 21 s | 11 | 0 | 0 | 5 min 28 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:20 UTC |
| mvcc-lsm-compaction | kmet-eval-only | Passed | 4 min 50 s | $0.0737 | 9 | 1 min 20 s | 8 | 1 | 1 | 1 min 32 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 02:31 UTC |
| payments-pipeline-fix | kmet-eval-only | Passed | 11 min 24 s | $0.2550 | 25 | 6 min 10 s | 24 | 2 | 1 | 8 min 13 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 01:29 UTC |
| photonic-waveguide-routing | kmet-eval-only | Passed | 15 min 40 s | $0.3806 | 26 | 12 min 28 s | 25 | 1 | 1 | 14 min 29 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 23:37 UTC |
| react-lead-form | kmet-eval-only | Passed | 5 min 32 s | $0.1818 | 15 | 4 min 17 s | 14 | 1 | 1 | 4 min 31 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:38 UTC |
| session-window-debug | kmet-eval-only | Failed | 3 min 27 s | $0.0968 | 11 | 2 min 34 s | 10 | 1 | 0 | 2 min 38 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:22 UTC |
| vf2-speedup-networkx | kmet-eval-only | Failed | 11 min 49 s | $0.2833 | 24 | 6 min 32 s | 23 | 3 | 0 | 10 min 30 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:01 UTC |
| wal-recovery-ordering | kmet-eval-only | Passed | 7 min 12 s | $0.1038 | 11 | 2 min 56 s | 10 | 1 | 1 | 2 min 59 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:02 UTC |
| bun-sourcemap-leak | kmet-shell-only | Failed | 4 min 52 s | $0.1132 | 11 | 3 min 57 s | 10 | 1 | 0 | 4 min 01 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:48 UTC |
| html-js-filter | kmet-shell-only | Failed | 6 min 02 s | $0.0903 | 9 | 2 min 35 s | 8 | 0 | 0 | 2 min 38 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-19 23:17 UTC |
| interleaved-vigenere | kmet-shell-only | Passed | 4 min 44 s | $0.1773 | 15 | 3 min 22 s | 14 | 0 | 1 | 3 min 48 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 01:13 UTC |
| kv-live-surgery | kmet-shell-only | Failed | 6 min 08 s | $0.2976 | 19 | 2 min 55 s | 18 | 3 | 0 | 4 min 33 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 03:44 UTC |
| music-harmony | kmet-shell-only | Failed | 4 min 04 s | $0.1744 | 15 | 3 min 07 s | 14 | 3 | 0 | 3 min 15 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:31 UTC |
| mvcc-lsm-compaction | kmet-shell-only | Passed | 4 min 14 s | $0.0396 | 5 | 48.7 s | 4 | 0 | 1 | 57.3 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 02:40 UTC |
| payments-pipeline-fix | kmet-shell-only | Failed | 26 min 51 s | $0.2302 | 18 | 20 min 57 s | 17 | 1 | 0 | 22 min 13 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 01:59 UTC |
| photonic-waveguide-routing | kmet-shell-only | Passed | 7 min 51 s | $0.1449 | 9 | 5 min 47 s | 8 | 0 | 1 | 6 min 37 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 00:07 UTC |
| react-lead-form | kmet-shell-only | Passed | 4 min 33 s | $0.1319 | 10 | 3 min 20 s | 9 | 0 | 1 | 3 min 33 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:50 UTC |
| session-window-debug | kmet-shell-only | Failed | 2 min 11 s | $0.0545 | 5 | 1 min 18 s | 4 | 0 | 0 | 1 min 20 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:29 UTC |
| vf2-speedup-networkx | kmet-shell-only | Failed | 7 min 31 s | $0.1708 | 19 | 4 min 18 s | 18 | 4 | 0 | 6 min 26 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 04:23 UTC |
| wal-recovery-ordering | kmet-shell-only | Failed | 2 min 53 s | $0.0661 | 6 | 1 min 30 s | 5 | 1 | 0 | 1 min 33 s | gpt-6.1-sol | kmet-89fa382/r1 · round 1 | 2026-11-20 05:15 UTC |
| bun-sourcemap-leak | opencode | Failed | 5 min 42 s | $0.1663 | 15 | 3 min 08 s | 18 | 0 | 0 | 4 min 06 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:58 UTC |
| html-js-filter | opencode | Passed | 8 min 57 s | $0.1288 | 12 | 3 min 23 s | 11 | 0 | 1 | 4 min 39 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 19:59 UTC |
| interleaved-vigenere | opencode | Passed | 5 min 03 s | $0.1178 | 16 | 2 min 45 s | 15 | 0 | 1 | 3 min 25 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 22:17 UTC |
| kv-live-surgery | opencode | Failed | 12 min 03 s | $0.4927 | 39 | 7 min 13 s | 38 | 0 | 0 | 9 min 47 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 00:33 UTC |
| music-harmony | opencode | Passed | 5 min 50 s | $0.1590 | 15 | 3 min 04 s | 14 | 0 | 1 | 4 min 07 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:41 UTC |
| mvcc-lsm-compaction | opencode | Passed | 6 min 23 s | $0.0875 | 14 | 1 min 08 s | 18 | 0 | 1 | 2 min 04 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 23:41 UTC |
| payments-pipeline-fix | opencode | Failed | 24 min 21 s | $0.6161 | 57 | 12 min 34 s | 67 | 0 | 0 | 19 min 30 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 23:05 UTC |
| photonic-waveguide-routing | opencode | Passed | 27 min 28 s | $0.4895 | 17 | 22 min 35 s | 16 | 0 | 1 | 24 min 01 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:00 UTC |
| react-lead-form | opencode | Failed | 8 min 30 s | $0.2360 | 21 | 4 min 14 s | 40 | 0 | 0 | 7 min 10 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 01:41 UTC |
| session-window-debug | opencode | Failed | 3 min 57 s | $0.0739 | 10 | 1 min 25 s | 15 | 0 | 0 | 2 min 15 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 02:25 UTC |
| vf2-speedup-networkx | opencode | Failed | 9 min 33 s | $0.1633 | 16 | 3 min 30 s | 16 | 0 | 0 | 7 min 47 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 01:15 UTC |
| wal-recovery-ordering | opencode | Passed | 7 min 52 s | $0.0951 | 9 | 2 min 00 s | 9 | 0 | 1 | 2 min 52 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 02:06 UTC |
| bun-sourcemap-leak | pi | Failed | 5 min 44 s | $0.1192 | 18 | 4 min 28 s | 26 | 3 | 0 | 4 min 33 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:52 UTC |
| html-js-filter | pi | Failed | 8 min 35 s | $0.1052 | 11 | 4 min 40 s | 10 | 1 | 0 | 4 min 52 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 19:50 UTC |
| interleaved-vigenere | pi | Passed | 5 min 36 s | $0.1357 | 19 | 4 min 18 s | 18 | 0 | 1 | 4 min 28 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 22:12 UTC |
| kv-live-surgery | pi | Failed | 19 min 34 s | $0.9758 | 62 | 14 min 38 s | 61 | 1 | 0 | 17 min 43 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 00:14 UTC |
| music-harmony | pi | Failed | 4 min 32 s | $0.1244 | 15 | 3 min 13 s | 14 | 3 | 0 | 3 min 20 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 21:36 UTC |
| mvcc-lsm-compaction | pi | Passed | 5 min 24 s | $0.0745 | 12 | 1 min 26 s | 23 | 1 | 1 | 1 min 39 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 23:36 UTC |
| payments-pipeline-fix | pi | Passed | 22 min 54 s | $0.5240 | 49 | 17 min 10 s | 54 | 5 | 1 | 19 min 50 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 22:42 UTC |
| photonic-waveguide-routing | pi | Passed | 29 min 43 s | $0.5585 | 44 | 20 min 53 s | 43 | 0 | 1 | 27 min 12 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-13 20:31 UTC |
| react-lead-form | pi | Passed | 9 min 08 s | $0.2638 | 30 | 7 min 37 s | 40 | 1 | 1 | 7 min 52 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 01:32 UTC |
| session-window-debug | pi | Failed | 5 min 46 s | $0.1416 | 18 | 4 min 35 s | 23 | 1 | 0 | 4 min 40 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 02:19 UTC |
| vf2-speedup-networkx | pi | Passed | 15 min 37 s | $0.3482 | 34 | 11 min 55 s | 33 | 1 | 1 | 14 min 14 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 00:59 UTC |
| wal-recovery-ordering | pi | Passed | 7 min 33 s | $0.1011 | 17 | 3 min 06 s | 29 | 0 | 1 | 3 min 11 s | gpt-6.1-sol | gpt-23a3a92/r1 · round 1 | 2026-11-14 01:58 UTC |
| bun-sourcemap-leak | vis-solo | Passed | 20 min 06 s | $0.4572 | 28 | 15 min 41 s | 27 | 0 | 1 | 18 min 24 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 21:13 UTC |
| html-js-filter | vis-solo | Failed | 12 min 16 s | $0.1932 | 19 | 5 min 38 s | 18 | 2 | 0 | 6 min 34 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 20:24 UTC |
| interleaved-vigenere | vis-solo | Passed | 7 min 38 s | $0.1653 | 23 | 4 min 42 s | 22 | 1 | 1 | 5 min 27 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 21:33 UTC |
| kv-live-surgery | vis-solo | Error | 8 min 45 s | $0.2791 | 33 | 5 min 43 s | 32 | 5 | 0 | 6 min 26 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 22:15 UTC |
| music-harmony | vis-solo | Failed | 4 min 51 s | $0.1008 | 12 | 2 min 45 s | 11 | 0 | 0 | 3 min 02 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 21:08 UTC |
| mvcc-lsm-compaction | vis-solo | Passed | 7 min 31 s | $0.0786 | 9 | 1 min 33 s | 8 | 0 | 1 | 2 min 32 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 22:08 UTC |
| payments-pipeline-fix | vis-solo | Passed | 26 min 38 s | $0.6621 | 53 | 18 min 45 s | 52 | 1 | 1 | 22 min 32 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 21:41 UTC |
| photonic-waveguide-routing | vis-solo | Passed | 32 min 10 s | $0.6379 | 26 | 27 min 24 s | 25 | 1 | 1 | 29 min 55 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 20:36 UTC |
| react-lead-form | vis-solo | Failed | 17 min 22 s | $0.4947 | 26 | 13 min 29 s | 25 | 1 | 0 | 14 min 49 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 22:39 UTC |
| session-window-debug | vis-solo | Passed | 7 min 08 s | $0.1949 | 16 | 5 min 10 s | 15 | 0 | 1 | 5 min 23 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 23:06 UTC |
| vf2-speedup-networkx | vis-solo | Failed | 15 min 29 s | $0.2496 | 23 | 7 min 45 s | 22 | 0 | 0 | 13 min 34 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 22:24 UTC |
| wal-recovery-ordering | vis-solo | Passed | 9 min 33 s | $0.1594 | 13 | 3 min 47 s | 12 | 0 | 1 | 4 min 04 s | gpt-6.1-sol | gpt-vis-solo-057-43a8048 · round 1 | 2026-11-09 22:57 UTC |
| bun-sourcemap-leak | wisp | Failed | 11 min 20 s | $0.2347 | 21 | 6 min 50 s | 20 | 7 | 0 | 7 min 14 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 13:34 UTC |
| html-js-filter | wisp | Passed | 7 min 16 s | $0.1063 | 8 | 4 min 20 s | 7 | 0 | 1 | 4 min 22 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 13:03 UTC |
| interleaved-vigenere | wisp | Passed | 8 min 15 s | $0.2336 | 27 | 6 min 22 s | 26 | 1 | 1 | 7 min 02 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 13:46 UTC |
| kv-live-surgery | wisp | Failed | 7 min 54 s | $0.2722 | 23 | 4 min 59 s | 22 | 1 | 0 | 5 min 36 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 14:15 UTC |
| music-harmony | wisp | Failed | 3 min 59 s | $0.0829 | 7 | 2 min 33 s | 6 | 1 | 0 | 2 min 36 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 13:30 UTC |
| mvcc-lsm-compaction | wisp | Failed | 5 min 11 s | $0.0491 | 6 | 1 min 21 s | 5 | 0 | 0 | 1 min 29 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 14:10 UTC |
| payments-pipeline-fix | wisp | Failed | 16 min 08 s | $0.3765 | 30 | 10 min 15 s | 29 | 6 | 0 | 12 min 55 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 13:54 UTC |
| photonic-waveguide-routing | wisp | Passed | 20 min 10 s | $0.3793 | 16 | 17 min 45 s | 15 | 0 | 1 | 18 min 38 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 13:10 UTC |
| react-lead-form | wisp | Failed | 8 min 53 s | $0.2723 | 18 | 7 min 15 s | 17 | 3 | 0 | 7 min 41 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 14:43 UTC |
| session-window-debug | wisp | Failed | 4 min 32 s | $0.1440 | 13 | 3 min 38 s | 11 | 1 | 0 | 3 min 40 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 14:55 UTC |
| vf2-speedup-networkx | wisp | Failed | 19 min 36 s | $0.3480 | 28 | 10 min 06 s | 27 | 5 | 0 | 14 min 46 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 14:23 UTC |
| wal-recovery-ordering | wisp | Failed | 3 min 50 s | $0.0816 | 7 | 2 min 13 s | 6 | 0 | 0 | 2 min 16 s | gpt-6.1-sol | gpt-wisp-5818f6d/r1 · round 1 | 2026-11-19 14:52 UTC |
| bun-sourcemap-leak | codex | Failed | 8 min 21 s | $0.1596 | 11 | 6 min 36 s | 10 | 0 | 0 | 6 min 41 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:30 UTC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| html-js-filter | codex | Passed | 8 min 06 s | $0.1470 | 13 | 4 min 41 s | 12 | 1 | 1 | 4 min 47 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 02:29 UTC |
| interleaved-vigenere | codex | Passed | 11 min 57 s | $0.3335 | 28 | 9 min 29 s | 27 | 0 | 1 | 10 min 11 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:52 UTC |
| kv-live-surgery | codex | Passed | 20 min 22 s | $0.9341 | 49 | 16 min 30 s | 48 | 0 | 1 | 18 min 03 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 06:56 UTC |
| music-harmony | codex | Failed | 6 min 24 s | $0.1705 | 10 | 4 min 38 s | 9 | 0 | 0 | 4 min 44 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:13 UTC |
| mvcc-lsm-compaction | codex | Failed | 6 min 08 s | $0.0850 | 7 | 1 min 25 s | 6 | 0 | 0 | 1 min 39 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 06:38 UTC |
| payments-pipeline-fix | codex | Passed | 20 min 59 s | $0.6295 | 54 | 16 min 38 s | 53 | 0 | 1 | 17 min 09 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 05:20 UTC |
| photonic-waveguide-routing | codex | Passed | 15 min 16 s | $0.2822 | 15 | 13 min 03 s | 14 | 1 | 1 | 13 min 27 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 02:54 UTC |
| react-lead-form | codex | Failed | 9 min 17 s | $0.2631 | 15 | 7 min 42 s | 14 | 0 | 0 | 8 min 00 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 08:30 UTC |
| session-window-debug | codex | Failed | 6 min 36 s | $0.1982 | 12 | 4 min 51 s | 11 | 0 | 0 | 4 min 57 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 09:19 UTC |
| vf2-speedup-networkx | codex | Failed | 12 min 30 s | $0.4301 | 33 | 9 min 52 s | 32 | 1 | 0 | 10 min 45 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 07:36 UTC |
| wal-recovery-ordering | codex | Failed | 5 min 32 s | $0.1198 | 8 | 3 min 11 s | 7 | 1 | 0 | 3 min 16 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 09:01 UTC |
| bun-sourcemap-leak | opencode | Passed | 9 min 38 s | $0.2056 | 23 | 6 min 34 s | 34 | 0 | 1 | 7 min 57 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:42 UTC |
| html-js-filter | opencode | Passed | 8 min 46 s | $0.1239 | 11 | 4 min 15 s | 10 | 0 | 1 | 5 min 22 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 02:45 UTC |
| interleaved-vigenere | opencode | Passed | 8 min 23 s | $0.1887 | 25 | 5 min 15 s | 26 | 0 | 1 | 6 min 27 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 05:11 UTC |
| kv-live-surgery | opencode | Failed | 9 min 58 s | $0.3780 | 29 | 5 min 19 s | 28 | 1 | 0 | 7 min 42 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 07:26 UTC |
| music-harmony | opencode | Failed | 5 min 36 s | $0.1040 | 15 | 2 min 59 s | 14 | 0 | 0 | 3 min 50 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:24 UTC |
| mvcc-lsm-compaction | opencode | Passed | 6 min 02 s | $0.0644 | 11 | 1 min 03 s | 12 | 0 | 1 | 1 min 55 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 06:49 UTC |
| payments-pipeline-fix | opencode | Failed | 29 min 56 s | $0.7502 | 62 | 15 min 40 s | 72 | 1 | 0 | 23 min 42 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 06:08 UTC |
| photonic-waveguide-routing | opencode | Passed | 30 min 28 s | $0.5162 | 14 | 27 min 21 s | 14 | 0 | 1 | 28 min 23 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 03:42 UTC |
| react-lead-form | opencode | Passed | 8 min 38 s | $0.2160 | 21 | 4 min 31 s | 35 | 0 | 1 | 7 min 21 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 08:53 UTC |
| session-window-debug | opencode | Failed | 3 min 46 s | $0.0687 | 8 | 1 min 19 s | 13 | 0 | 0 | 1 min 58 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 09:30 UTC |
| vf2-speedup-networkx | opencode | Passed | 23 min 20 s | $0.5106 | 47 | 15 min 18 s | 56 | 0 | 1 | 21 min 32 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 08:07 UTC |
| wal-recovery-ordering | opencode | Failed | 5 min 25 s | $0.0992 | 8 | 1 min 22 s | 7 | 0 | 0 | 2 min 18 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 09:13 UTC |
| bun-sourcemap-leak | pi | Failed | 4 min 04 s | $0.0859 | 15 | 2 min 43 s | 24 | 2 | 0 | 2 min 48 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:38 UTC |
| html-js-filter | pi | Passed | 7 min 41 s | $0.1183 | 12 | 4 min 24 s | 11 | 0 | 1 | 4 min 29 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 02:37 UTC |
| interleaved-vigenere | pi | Passed | 7 min 14 s | $0.1748 | 27 | 5 min 10 s | 26 | 0 | 1 | 5 min 48 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 05:04 UTC |
| kv-live-surgery | pi | Failed | 9 min 55 s | $0.4176 | 30 | 6 min 03 s | 29 | 1 | 0 | 7 min 57 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 07:16 UTC |
| music-harmony | pi | Passed | 5 min 20 s | $0.1185 | 12 | 4 min 03 s | 11 | 2 | 1 | 4 min 09 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 04:19 UTC |
| mvcc-lsm-compaction | pi | Failed | 5 min 25 s | $0.0693 | 11 | 1 min 18 s | 22 | 1 | 0 | 1 min 32 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 06:44 UTC |
| payments-pipeline-fix | pi | Failed | 27 min 04 s | $0.5633 | 58 | 21 min 33 s | 66 | 2 | 0 | 24 min 01 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 05:41 UTC |
| photonic-waveguide-routing | pi | Passed | 32 min 52 s | $0.5858 | 29 | 26 min 47 s | 28 | 0 | 1 | 31 min 28 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 03:09 UTC |
| react-lead-form | pi | Passed | 13 min 05 s | $0.3290 | 40 | 11 min 30 s | 49 | 3 | 1 | 11 min 46 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 08:40 UTC |
| session-window-debug | pi | Failed | 4 min 30 s | $0.0838 | 15 | 3 min 15 s | 20 | 1 | 0 | 3 min 19 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 09:25 UTC |
| vf2-speedup-networkx | pi | Failed | 18 min 38 s | $0.4264 | 42 | 13 min 02 s | 41 | 2 | 0 | 16 min 53 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 07:48 UTC |
| wal-recovery-ordering | pi | Passed | 6 min 04 s | $0.1107 | 13 | 2 min 41 s | 30 | 0 | 1 | 2 min 46 s | gpt-6.1-sol | gpt-23a3a92/r2 · round 2 | 2026-11-14 09:07 UTC |
| bun-sourcemap-leak | wisp | Failed | 3 min 52 s | $0.0971 | 9 | 2 min 58 s | 8 | 2 | 0 | 3 min 00 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:30 UTC |
| html-js-filter | wisp | Failed | 7 min 09 s | $0.1151 | 10 | 4 min 23 s | 9 | 0 | 0 | 4 min 24 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:00 UTC |
| interleaved-vigenere | wisp | Passed | 3 min 16 s | $0.0945 | 11 | 2 min 13 s | 10 | 0 | 1 | 2 min 19 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:34 UTC |
| kv-live-surgery | wisp | Failed | 17 min 05 s | $0.5488 | 48 | 12 min 58 s | 47 | 3 | 0 | 15 min 31 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:59 UTC |
| music-harmony | wisp | Failed | 3 min 57 s | $0.0864 | 12 | 3 min 05 s | 11 | 2 | 0 | 3 min 09 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:26 UTC |
| mvcc-lsm-compaction | wisp | Failed | 4 min 55 s | $0.0482 | 7 | 1 min 24 s | 6 | 2 | 0 | 1 min 33 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:54 UTC |
| payments-pipeline-fix | wisp | Failed | 17 min 28 s | $0.4609 | 31 | 12 min 51 s | 30 | 2 | 0 | 14 min 46 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:37 UTC |
| photonic-waveguide-routing | wisp | Passed | 18 min 34 s | $0.3716 | 23 | 14 min 44 s | 22 | 1 | 1 | 17 min 26 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 15:07 UTC |
| react-lead-form | wisp | Failed | 6 min 38 s | $0.1669 | 14 | 4 min 57 s | 13 | 2 | 0 | 5 min 18 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 16:27 UTC |
| session-window-debug | wisp | Failed | 4 min 10 s | $0.1068 | 13 | 3 min 17 s | 11 | 0 | 0 | 3 min 20 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 16:38 UTC |
| vf2-speedup-networkx | wisp | Failed | 10 min 35 s | $0.2397 | 25 | 7 min 21 s | 24 | 4 | 0 | 9 min 23 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 16:16 UTC |
| wal-recovery-ordering | wisp | Passed | 4 min 47 s | $0.0929 | 6 | 1 min 50 s | 5 | 0 | 1 | 1 min 52 s | gpt-6.1-sol | gpt-wisp-5818f6d/r2 · round 2 | 2026-11-19 16:34 UTC |
| bun-sourcemap-leak | codex | Failed | 6 min 32 s | $0.1747 | 12 | 4 min 39 s | 11 | 0 | 0 | 4 min 45 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:13 UTC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| html-js-filter | codex | Passed | 10 min 38 s | $0.1635 | 12 | 7 min 05 s | 11 | 0 | 1 | 7 min 15 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 09:33 UTC |
| interleaved-vigenere | codex | Passed | 7 min 04 s | $0.1965 | 20 | 5 min 04 s | 19 | 1 | 1 | 5 min 23 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:31 UTC |
| kv-live-surgery | codex | Passed | 19 min 00 s | $0.9219 | 52 | 14 min 41 s | 51 | 0 | 1 | 16 min 36 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 13:29 UTC |
| music-harmony | codex | Failed | 6 min 52 s | $0.1853 | 12 | 4 min 34 s | 11 | 0 | 0 | 4 min 42 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 10:54 UTC |
| mvcc-lsm-compaction | codex | Passed | 6 min 10 s | $0.0804 | 6 | 1 min 42 s | 5 | 0 | 1 | 1 min 56 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 13:11 UTC |
| payments-pipeline-fix | codex | Passed | 21 min 11 s | $0.4939 | 40 | 17 min 08 s | 39 | 0 | 1 | 17 min 45 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:54 UTC |
| photonic-waveguide-routing | codex | Passed | 16 min 26 s | $0.3960 | 17 | 14 min 16 s | 16 | 0 | 1 | 14 min 34 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 10:02 UTC |
| react-lead-form | codex | Failed | 8 min 22 s | $0.2914 | 19 | 6 min 18 s | 18 | 0 | 0 | 6 min 57 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 15:55 UTC |
| session-window-debug | codex | Passed | 5 min 45 s | $0.1684 | 9 | 3 min 51 s | 8 | 0 | 1 | 3 min 56 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:46 UTC |
| vf2-speedup-networkx | codex | Failed | 11 min 53 s | $0.3266 | 29 | 9 min 11 s | 28 | 0 | 0 | 10 min 13 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 15:08 UTC |
| wal-recovery-ordering | codex | Passed | 10 min 17 s | $0.1794 | 10 | 4 min 07 s | 9 | 0 | 1 | 4 min 14 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:23 UTC |
| bun-sourcemap-leak | opencode | Failed | 5 min 56 s | $0.1359 | 20 | 3 min 18 s | 19 | 0 | 0 | 4 min 20 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:25 UTC |
| html-js-filter | opencode | Passed | 7 min 37 s | $0.1326 | 10 | 3 min 15 s | 9 | 0 | 1 | 4 min 16 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 09:54 UTC |
| interleaved-vigenere | opencode | Failed | 8 min 47 s | $0.2189 | 21 | 4 min 36 s | 20 | 0 | 0 | 5 min 31 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:45 UTC |
| kv-live-surgery | opencode | Failed | 9 min 54 s | $0.3987 | 24 | 4 min 31 s | 23 | 1 | 0 | 6 min 08 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-17 20:14 UTC |
| music-harmony | opencode | Passed | 6 min 26 s | $0.1309 | 12 | 3 min 12 s | 11 | 0 | 1 | 4 min 19 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:07 UTC |
| mvcc-lsm-compaction | opencode | Passed | 5 min 59 s | $0.0858 | 10 | 1 min 11 s | 13 | 0 | 1 | 1 min 59 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 13:23 UTC |
| payments-pipeline-fix | opencode | Failed | 29 min 45 s | $0.7066 | 55 | 18 min 19 s | 67 | 0 | 0 | 25 min 39 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 12:41 UTC |
| photonic-waveguide-routing | opencode | Passed | 19 min 25 s | $0.3574 | 20 | 15 min 56 s | 19 | 0 | 1 | 17 min 05 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 10:34 UTC |
| react-lead-form | opencode | Passed | 10 min 09 s | $0.2806 | 25 | 5 min 36 s | 38 | 0 | 1 | 8 min 54 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:13 UTC |
| session-window-debug | opencode | Passed | 7 min 19 s | $0.1777 | 15 | 3 min 33 s | 24 | 0 | 1 | 5 min 01 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:56 UTC |
| vf2-speedup-networkx | opencode | Passed | 16 min 20 s | $0.3763 | 32 | 8 min 20 s | 38 | 0 | 1 | 14 min 09 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 15:38 UTC |
| wal-recovery-ordering | opencode | Passed | 7 min 49 s | $0.0864 | 9 | 1 min 28 s | 8 | 0 | 1 | 2 min 31 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:38 UTC |
| bun-sourcemap-leak | pi | Failed | 5 min 41 s | $0.1023 | 19 | 4 min 21 s | 27 | 2 | 0 | 4 min 26 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:20 UTC |
| html-js-filter | pi | Passed | 9 min 51 s | $0.1174 | 12 | 6 min 30 s | 11 | 0 | 1 | 6 min 34 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 09:44 UTC |
| interleaved-vigenere | pi | Passed | 6 min 11 s | $0.1438 | 20 | 3 min 54 s | 21 | 0 | 1 | 4 min 13 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:38 UTC |
| kv-live-surgery | pi | Failed | 9 min 04 s | $0.4034 | 31 | 5 min 34 s | 30 | 1 | 0 | 7 min 13 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 13:48 UTC |
| music-harmony | pi | Failed | 5 min 55 s | $0.1422 | 27 | 4 min 37 s | 26 | 6 | 0 | 4 min 45 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 11:01 UTC |
| mvcc-lsm-compaction | pi | Passed | 5 min 44 s | $0.0722 | 11 | 1 min 37 s | 22 | 1 | 1 | 1 min 51 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 13:17 UTC |
| payments-pipeline-fix | pi | Passed | 25 min 47 s | $0.6014 | 66 | 18 min 58 s | 73 | 4 | 1 | 22 min 49 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 12:15 UTC |
| photonic-waveguide-routing | pi | Passed | 16 min 16 s | $0.3493 | 30 | 11 min 33 s | 29 | 3 | 1 | 14 min 45 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 10:18 UTC |
| react-lead-form | pi | Failed | 10 min 09 s | $0.3041 | 36 | 8 min 35 s | 47 | 2 | 0 | 8 min 48 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:03 UTC |
| session-window-debug | pi | Failed | 4 min 32 s | $0.1030 | 14 | 3 min 17 s | 19 | 0 | 0 | 3 min 22 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:52 UTC |
| vf2-speedup-networkx | pi | Passed | 18 min 20 s | $0.4021 | 39 | 14 min 02 s | 38 | 1 | 1 | 16 min 55 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 15:20 UTC |
| wal-recovery-ordering | pi | Failed | 4 min 26 s | $0.1000 | 18 | 2 min 59 s | 30 | 1 | 0 | 3 min 03 s | gpt-6.1-sol | gpt-23a3a92/r3 · round 3 | 2026-11-14 16:34 UTC |
| bun-sourcemap-leak | wisp | Passed | 16 min 18 s | $0.5532 | 38 | 15 min 14 s | 36 | 12 | 1 | 15 min 21 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 17:14 UTC |
| html-js-filter | wisp | Failed | 6 min 52 s | $0.1264 | 9 | 3 min 57 s | 8 | 0 | 0 | 4 min 00 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 16:43 UTC |
| interleaved-vigenere | wisp | Passed | 14 min 59 s | $0.4364 | 46 | 13 min 28 s | 44 | 3 | 1 | 14 min 01 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 17:30 UTC |
| kv-live-surgery | wisp | Failed | 9 min 22 s | $0.3560 | 26 | 6 min 46 s | 25 | 4 | 0 | 7 min 42 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 18:12 UTC |
| music-harmony | wisp | Failed | 2 min 53 s | $0.0898 | 7 | 2 min 00 s | 6 | 2 | 0 | 2 min 03 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 17:11 UTC |
| mvcc-lsm-compaction | wisp | Passed | 5 min 58 s | $0.0883 | 7 | 1 min 24 s | 6 | 2 | 1 | 1 min 38 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 18:06 UTC |
| payments-pipeline-fix | wisp | Passed | 20 min 54 s | $0.5605 | 44 | 15 min 16 s | 42 | 6 | 1 | 17 min 55 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 17:45 UTC |
| photonic-waveguide-routing | wisp | Passed | 21 min 17 s | $0.5711 | 30 | 18 min 40 s | 29 | 0 | 1 | 20 min 06 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 16:50 UTC |
| react-lead-form | wisp | Passed | 6 min 13 s | $0.1918 | 12 | 4 min 52 s | 11 | 3 | 1 | 5 min 11 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 18:33 UTC |
| session-window-debug | wisp | Failed | 2 min 46 s | $0.0704 | 7 | 1 min 52 s | 6 | 0 | 0 | 1 min 54 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 18:42 UTC |
| vf2-speedup-networkx | wisp | Failed | 11 min 24 s | $0.2133 | 22 | 5 min 35 s | 21 | 3 | 0 | 10 min 18 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 18:21 UTC |
| wal-recovery-ordering | wisp | Failed | 3 min 16 s | $0.0695 | 5 | 1 min 41 s | 4 | 0 | 0 | 1 min 43 s | gpt-6.1-sol | gpt-wisp-5818f6d/r3 · round 3 | 2026-11-19 18:39 UTC |
| bun-sourcemap-leak | codex | Failed | 9 min 10 s | $0.2153 | 16 | 7 min 05 s | 15 | 0 | 0 | 7 min 21 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:07 UTC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| html-js-filter | codex | Failed | 8 min 40 s | $0.1544 | 8 | 4 min 52 s | 7 | 0 | 0 | 4 min 58 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 17:04 UTC |
| interleaved-vigenere | codex | Passed | 6 min 51 s | $0.2091 | 22 | 4 min 49 s | 21 | 1 | 1 | 5 min 02 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:40 UTC |
| kv-live-surgery | codex | Failed | 15 min 59 s | $0.8429 | 54 | 11 min 52 s | 53 | 0 | 0 | 12 min 58 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 21:08 UTC |
| music-harmony | codex | Failed | 8 min 05 s | $0.1638 | 14 | 5 min 24 s | 13 | 0 | 0 | 5 min 33 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 18:49 UTC |
| mvcc-lsm-compaction | codex | Passed | 6 min 20 s | $0.0609 | 6 | 1 min 37 s | 5 | 0 | 1 | 1 min 53 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 20:48 UTC |
| payments-pipeline-fix | codex | Passed | 17 min 08 s | $0.3911 | 32 | 12 min 32 s | 31 | 0 | 1 | 13 min 21 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:57 UTC |
| photonic-waveguide-routing | codex | Passed | 27 min 14 s | $0.5683 | 29 | 24 min 47 s | 28 | 0 | 1 | 25 min 20 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 17:29 UTC |
| react-lead-form | codex | Passed | 7 min 14 s | $0.2025 | 12 | 5 min 38 s | 11 | 0 | 1 | 5 min 57 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 22:33 UTC |
| session-window-debug | codex | Failed | 5 min 55 s | $0.1662 | 12 | 4 min 10 s | 11 | 0 | 0 | 4 min 16 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 23:28 UTC |
| vf2-speedup-networkx | codex | Failed | 13 min 33 s | $0.4313 | 29 | 11 min 04 s | 28 | 0 | 0 | 11 min 40 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 21:44 UTC |
| wal-recovery-ordering | codex | Passed | 8 min 10 s | $0.1405 | 8 | 2 min 48 s | 7 | 0 | 1 | 2 min 54 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 23:04 UTC |
| bun-sourcemap-leak | opencode | Passed | 14 min 36 s | $0.3277 | 35 | 10 min 25 s | 40 | 0 | 1 | 12 min 26 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:25 UTC |
| html-js-filter | opencode | Passed | 7 min 31 s | $0.1104 | 10 | 3 min 02 s | 9 | 0 | 1 | 4 min 02 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 17:21 UTC |
| interleaved-vigenere | opencode | Passed | 6 min 02 s | $0.1530 | 23 | 3 min 27 s | 22 | 0 | 1 | 4 min 11 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:51 UTC |
| kv-live-surgery | opencode | Failed | 12 min 29 s | $0.4519 | 36 | 6 min 28 s | 35 | 1 | 0 | 9 min 26 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 21:31 UTC |
| music-harmony | opencode | Failed | 5 min 08 s | $0.1126 | 8 | 2 min 24 s | 7 | 0 | 0 | 3 min 07 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:02 UTC |
| mvcc-lsm-compaction | opencode | Passed | 7 min 10 s | $0.0558 | 9 | 46.9 s | 9 | 0 | 1 | 1 min 48 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 21:01 UTC |
| payments-pipeline-fix | opencode | Failed | 18 min 43 s | $0.4907 | 47 | 9 min 53 s | 59 | 0 | 0 | 14 min 48 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 20:30 UTC |
| photonic-waveguide-routing | opencode | Passed | 23 min 56 s | $0.3984 | 15 | 20 min 06 s | 15 | 0 | 1 | 21 min 25 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 18:25 UTC |
| react-lead-form | opencode | Failed | 12 min 25 s | $0.3544 | 26 | 4 min 48 s | 44 | 1 | 0 | 9 min 46 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 22:51 UTC |
| session-window-debug | opencode | Passed | 3 min 59 s | $0.0767 | 10 | 1 min 25 s | 15 | 0 | 1 | 2 min 10 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 23:40 UTC |
| vf2-speedup-networkx | opencode | Error | 19 min 40 s | not recorded | 52 | 11 min 24 s | 54 | 1 | 0 | 17 min 58 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 22:13 UTC |
| wal-recovery-ordering | opencode | Passed | 7 min 34 s | $0.0994 | 9 | 1 min 18 s | 8 | 0 | 1 | 2 min 32 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 23:20 UTC |
| bun-sourcemap-leak | pi | Passed | 8 min 54 s | $0.2386 | 26 | 7 min 37 s | 35 | 10 | 1 | 7 min 42 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:16 UTC |
| html-js-filter | pi | Passed | 8 min 46 s | $0.1284 | 14 | 5 min 25 s | 13 | 0 | 1 | 5 min 30 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 17:12 UTC |
| interleaved-vigenere | pi | Passed | 4 min 27 s | $0.0993 | 15 | 3 min 03 s | 17 | 0 | 1 | 3 min 14 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 19:46 UTC |
| kv-live-surgery | pi | Failed | 6 min 43 s | $0.3131 | 22 | 4 min 18 s | 21 | 2 | 0 | 4 min 35 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 21:24 UTC |
| music-harmony | pi | Failed | 4 min 58 s | $0.1342 | 19 | 3 min 40 s | 18 | 1 | 0 | 3 min 47 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 18:57 UTC |
| mvcc-lsm-compaction | pi | Failed | 5 min 50 s | $0.0726 | 12 | 1 min 37 s | 22 | 1 | 0 | 1 min 51 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 20:55 UTC |
| payments-pipeline-fix | pi | Passed | 15 min 32 s | $0.3079 | 38 | 10 min 33 s | 45 | 6 | 1 | 12 min 16 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 20:14 UTC |
| photonic-waveguide-routing | pi | Passed | 28 min 38 s | $0.5694 | 47 | 19 min 06 s | 46 | 3 | 1 | 27 min 02 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 17:56 UTC |
| react-lead-form | pi | Failed | 10 min 36 s | $0.2916 | 31 | 8 min 06 s | 41 | 2 | 0 | 8 min 21 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 22:40 UTC |
| session-window-debug | pi | Passed | 5 min 37 s | $0.1530 | 17 | 4 min 25 s | 26 | 1 | 1 | 4 min 29 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 23:34 UTC |
| vf2-speedup-networkx | pi | Passed | 16 min 02 s | $0.4605 | 38 | 11 min 01 s | 37 | 5 | 1 | 14 min 36 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 21:57 UTC |
| wal-recovery-ordering | pi | Passed | 8 min 28 s | $0.1453 | 20 | 3 min 46 s | 32 | 0 | 1 | 3 min 52 s | gpt-6.1-sol | gpt-23a3a92/r4 · round 4 | 2026-11-14 23:12 UTC |
| bun-sourcemap-leak | wisp | Passed | 12 min 58 s | $0.3717 | 33 | 12 min 01 s | 31 | 4 | 1 | 12 min 05 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 19:23 UTC |
| html-js-filter | wisp | Failed | 11 min 09 s | $0.2306 | 22 | 8 min 19 s | 20 | 0 | 0 | 8 min 23 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 18:45 UTC |
| interleaved-vigenere | wisp | Passed | 5 min 00 s | $0.1320 | 16 | 3 min 56 s | 15 | 0 | 1 | 4 min 06 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 19:36 UTC |
| kv-live-surgery | wisp | Failed | 6 min 08 s | $0.2327 | 19 | 4 min 02 s | 18 | 1 | 0 | 4 min 35 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 20:09 UTC |
| music-harmony | wisp | Failed | 4 min 34 s | $0.1206 | 15 | 3 min 38 s | 14 | 1 | 0 | 3 min 41 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 19:19 UTC |
| mvcc-lsm-compaction | wisp | Passed | 5 min 52 s | $0.0575 | 9 | 1 min 35 s | 8 | 2 | 1 | 1 min 53 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 20:03 UTC |
| payments-pipeline-fix | wisp | Passed | 22 min 00 s | $0.5262 | 42 | 15 min 58 s | 39 | 3 | 1 | 19 min 04 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 19:41 UTC |
| photonic-waveguide-routing | wisp | Passed | 22 min 14 s | $0.4814 | 18 | 19 min 54 s | 16 | 0 | 1 | 21 min 06 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 18:56 UTC |
| react-lead-form | wisp | Passed | 7 min 02 s | $0.1982 | 13 | 5 min 46 s | 12 | 3 | 1 | 6 min 04 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 20:24 UTC |
| session-window-debug | wisp | Failed | 3 min 34 s | $0.1009 | 8 | 2 min 42 s | 7 | 2 | 0 | 2 min 45 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 20:34 UTC |
| vf2-speedup-networkx | wisp | Failed | 8 min 34 s | $0.1659 | 16 | 4 min 36 s | 15 | 3 | 0 | 7 min 29 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 20:15 UTC |
| wal-recovery-ordering | wisp | Failed | 2 min 48 s | $0.0949 | 7 | 1 min 42 s | 6 | 0 | 0 | 1 min 44 s | gpt-6.1-sol | gpt-wisp-5818f6d/r4 · round 4 | 2026-11-19 20:31 UTC |
| bun-sourcemap-leak | codex | Failed | 5 min 10 s | $0.1484 | 10 | 3 min 29 s | 9 | 0 | 0 | 3 min 33 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 01:22 UTC |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| html-js-filter | codex | Failed | 10 min 07 s | $0.1545 | 10 | 5 min 28 s | 9 | 0 | 0 | 5 min 42 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-14 23:44 UTC |
| interleaved-vigenere | codex | Passed | 6 min 45 s | $0.2083 | 17 | 4 min 57 s | 16 | 1 | 1 | 5 min 05 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 01:44 UTC |
| kv-live-surgery | codex | Failed | 27 min 31 s | $1.20 | 65 | 22 min 21 s | 64 | 0 | 0 | 25 min 03 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 03:08 UTC |
| music-harmony | codex | Failed | 5 min 46 s | $0.1194 | 9 | 3 min 58 s | 8 | 0 | 0 | 4 min 04 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 01:07 UTC |
| mvcc-lsm-compaction | codex | Passed | 6 min 48 s | $0.0926 | 8 | 1 min 29 s | 7 | 0 | 1 | 1 min 49 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 02:49 UTC |
| payments-pipeline-fix | codex | Failed | 13 min 34 s | $0.2451 | 25 | 7 min 03 s | 24 | 0 | 0 | 7 min 29 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 02:08 UTC |
| photonic-waveguide-routing | codex | Passed | 15 min 01 s | $0.2576 | 12 | 12 min 29 s | 11 | 0 | 1 | 12 min 54 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 00:17 UTC |
| react-lead-form | codex | Failed | 9 min 22 s | $0.2674 | 16 | 7 min 47 s | 15 | 0 | 0 | 8 min 05 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 04:54 UTC |
| session-window-debug | codex | Failed | 6 min 08 s | $0.1230 | 12 | 3 min 54 s | 11 | 0 | 0 | 3 min 59 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 05:45 UTC |
| vf2-speedup-networkx | codex | Failed | 7 min 04 s | $0.1815 | 19 | 5 min 01 s | 18 | 0 | 0 | 5 min 28 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 04:14 UTC |
| wal-recovery-ordering | codex | Passed | 7 min 50 s | $0.1363 | 7 | 2 min 40 s | 6 | 0 | 1 | 2 min 48 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 05:26 UTC |
| bun-sourcemap-leak | opencode | Failed | 7 min 35 s | $0.1857 | 17 | 3 min 45 s | 17 | 0 | 0 | 4 min 48 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 14:16 UTC |
| html-js-filter | opencode | Passed | 9 min 11 s | $0.1285 | 12 | 3 min 15 s | 11 | 0 | 1 | 4 min 24 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 13:32 UTC |
| interleaved-vigenere | opencode | Passed | 11 min 45 s | $0.2982 | 29 | 6 min 42 s | 30 | 0 | 1 | 8 min 06 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 14:24 UTC |
| kv-live-surgery | opencode | Failed | 15 min 00 s | $0.5202 | 38 | 7 min 33 s | 37 | 1 | 0 | 11 min 06 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 15:03 UTC |
| music-harmony | opencode | Failed | 5 min 11 s | $0.0950 | 10 | 2 min 04 s | 9 | 0 | 0 | 2 min 48 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 14:11 UTC |
| mvcc-lsm-compaction | opencode | Passed | 7 min 38 s | $0.0624 | 9 | 47.0 s | 10 | 0 | 1 | 1 min 32 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 14:56 UTC |
| payments-pipeline-fix | opencode | Failed | 20 min 04 s | $0.4915 | 35 | 9 min 27 s | 41 | 0 | 0 | 14 min 29 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 14:36 UTC |
| photonic-waveguide-routing | opencode | Passed | 28 min 56 s | $0.4066 | 15 | 21 min 49 s | 14 | 0 | 1 | 24 min 19 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 13:42 UTC |
| react-lead-form | opencode | Passed | 11 min 17 s | $0.3166 | 18 | 3 min 19 s | 36 | 1 | 1 | 7 min 52 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 15:37 UTC |
| session-window-debug | opencode | Passed | 5 min 13 s | $0.0829 | 10 | 2 min 05 s | 16 | 0 | 1 | 3 min 00 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 15:59 UTC |
| vf2-speedup-networkx | opencode | Passed | 18 min 13 s | $0.4840 | 43 | 10 min 48 s | 47 | 0 | 1 | 14 min 57 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 15:19 UTC |
| wal-recovery-ordering | opencode | Passed | 10 min 33 s | $0.1138 | 9 | 1 min 43 s | 8 | 0 | 1 | 2 min 55 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-17 15:49 UTC |
| bun-sourcemap-leak | pi | Failed | 5 min 53 s | $0.1395 | 15 | 4 min 35 s | 24 | 2 | 0 | 4 min 40 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 01:27 UTC |
| html-js-filter | pi | Failed | 9 min 04 s | $0.1324 | 13 | 6 min 02 s | 11 | 0 | 0 | 6 min 08 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-14 23:54 UTC |
| interleaved-vigenere | pi | Passed | 11 min 34 s | $0.3366 | 32 | 9 min 41 s | 34 | 0 | 1 | 10 min 12 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 01:50 UTC |
| kv-live-surgery | pi | Passed | 13 min 00 s | $0.6680 | 52 | 9 min 13 s | 51 | 1 | 1 | 11 min 09 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 03:35 UTC |
| music-harmony | pi | Failed | 4 min 13 s | $0.1071 | 16 | 2 min 59 s | 15 | 2 | 0 | 3 min 06 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 01:12 UTC |
| mvcc-lsm-compaction | pi | Passed | 5 min 48 s | $0.0587 | 11 | 1 min 32 s | 22 | 1 | 1 | 1 min 45 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 02:56 UTC |
| payments-pipeline-fix | pi | Failed | 10 min 05 s | $0.2038 | 26 | 6 min 01 s | 30 | 2 | 0 | 7 min 25 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 02:22 UTC |
| photonic-waveguide-routing | pi | Passed | 19 min 56 s | $0.3369 | 22 | 16 min 20 s | 21 | 0 | 1 | 18 min 28 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 00:32 UTC |
| react-lead-form | pi | Failed | 12 min 14 s | $0.3770 | 41 | 10 min 45 s | 51 | 3 | 0 | 10 min 58 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 05:03 UTC |
| session-window-debug | pi | Failed | 5 min 08 s | $0.1193 | 17 | 3 min 56 s | 21 | 1 | 0 | 4 min 01 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 05:52 UTC |
| vf2-speedup-networkx | pi | Failed | 14 min 18 s | $0.2981 | 36 | 10 min 59 s | 35 | 1 | 0 | 12 min 55 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 04:21 UTC |
| wal-recovery-ordering | pi | Passed | 7 min 22 s | $0.1206 | 17 | 2 min 59 s | 29 | 0 | 1 | 3 min 04 s | gpt-6.1-sol | gpt-23a3a92/r5 · round 5 | 2026-11-15 05:33 UTC |
| bun-sourcemap-leak | wisp | Failed | 5 min 00 s | $0.1427 | 10 | 4 min 07 s | 9 | 2 | 0 | 4 min 09 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 21:02 UTC |
| html-js-filter | wisp | Passed | 8 min 26 s | $0.1294 | 10 | 5 min 12 s | 9 | 1 | 1 | 5 min 14 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 20:37 UTC |
| interleaved-vigenere | wisp | Passed | 4 min 34 s | $0.1488 | 16 | 3 min 34 s | 15 | 0 | 1 | 3 min 41 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 21:07 UTC |
| kv-live-surgery | wisp | Passed | 10 min 48 s | $0.3847 | 30 | 7 min 29 s | 29 | 1 | 1 | 9 min 05 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 21:37 UTC |
| music-harmony | wisp | Failed | 4 min 28 s | $0.1385 | 14 | 3 min 31 s | 13 | 1 | 0 | 3 min 35 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 20:57 UTC |
| mvcc-lsm-compaction | wisp | Failed | 5 min 07 s | $0.0472 | 6 | 1 min 12 s | 5 | 1 | 0 | 1 min 23 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 21:32 UTC |
| payments-pipeline-fix | wisp | Failed | 20 min 47 s | $0.4784 | 43 | 14 min 15 s | 42 | 4 | 0 | 16 min 56 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 21:11 UTC |
| photonic-waveguide-routing | wisp | Passed | 11 min 25 s | $0.2207 | 15 | 9 min 11 s | 14 | 0 | 1 | 10 min 12 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 20:46 UTC |
| react-lead-form | wisp | Failed | 8 min 17 s | $0.2587 | 16 | 7 min 00 s | 15 | 5 | 0 | 7 min 17 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 22:07 UTC |
| session-window-debug | wisp | Passed | 4 min 36 s | $0.1449 | 13 | 3 min 46 s | 11 | 0 | 1 | 3 min 48 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 22:22 UTC |
| vf2-speedup-networkx | wisp | Failed | 18 min 51 s | $0.4426 | 45 | 12 min 47 s | 42 | 10 | 0 | 17 min 42 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 21:48 UTC |
| wal-recovery-ordering | wisp | Passed | 6 min 20 s | $0.0826 | 6 | 2 min 05 s | 5 | 0 | 1 | 2 min 07 s | gpt-6.1-sol | gpt-wisp-5818f6d/r5 · round 5 | 2026-11-19 22:15 UTC |