Fast decision models bring the evaluation fuzziness of LLMs - but with a considerable speed improvement. Setting aside their limitations and their intended use case I wanted to see how they would fit fuzzy sorting with merge sort with its nlogn complexity.
I took 31 apoptosis-related resources from datasets.bio and ranked their metadata by usefulness for identifying therapeutic targets in pancreatic cancer. Each model sorted the same list five times through OpenRouter.
The criterion favours direct functional evidence in human pancreatic cancer, followed by molecular or clinical associations, then transferable apoptosis mechanisms. The models see descriptions and usage information about the resources; they do not analyse the underlying experimental data.
Here are the top ten, ordered by mean rank across the five runs:
| Rank | Jev 1.13 | Kev 4B | D1 |
|---|---|---|---|
| 1 | COLT-Cancer | COLT-Cancer | COLT-Cancer |
| 2 | CancerSCEM | ApInAPDB | CancerSCEM |
| 3 | lnc2cancer | iUUCD | UniHI |
| 4 | Reactome | LncTarD | ApInAPDB |
| 5 | UniHI | CancerSCEM | HomoMINT |
| 6 | ApInAPDB | ADPriboDB | DiSignAtlas |
| 7 | ADPriboDB | lnc2cancer | Reactome |
| 8 | iUUCD | Reactome | iUUCD |
| 9 | Death Domain database | UniHI | LncTarD |
| 10 | LncTarD | LncSEA | lnc2cancer |
All three put COLT-Cancer first, but then diverge. Kev puts ApInAPDB and iUUCD near the top; Jev brings lnc2cancer and Reactome forward; D1 places UniHI and HomoMINT higher. HomoMINT is problematic - it's been dead for a while.
Cost: 3 models x 5 runs, 1,762 calls, $0.099395.
How repeatable was it? Kev returned the same complete ranking in all five runs. Jev and D1 varied: agreement on repeated A/B comparisons was 94.28% and 94.69%, respectively.
Merge sort needed roughly 115–119 comparisons for each 31-item list, rather than comparing all 465 possible pairs.
The completed sorts took about 40 seconds for Jev, 77 seconds for Kev and 53 seconds for D1 on average. This includes API overhead - local inference can get you there 1.5 - 2 orders of magnitude faster.
A model can also prefer A to B, B to C and C to A. Merge sort will still return a list, but that list need not agree with every possible comparison.