fkrebs/docs
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: remove claude high-effort variants, set temperature=0.7 on Claude models (58 total)

Browse Source
This commit is contained in:
Florian Krebs
2026-05-23 00:41:26 +02:00
parent 9ab9033ab0
commit 396aa0e48e
1 changed files with 8 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
services/llm-benchmark.md
+7 -11
View File
@@ -1,6 +1,6 @@
# LLM Model Benchmark & Service Catalogue
_Last updated: 2026-05-23 | Models: 60 | Source: LiteLLM /model/info + live benchmarks_
_Last updated: 2026-05-23 | Models: 58 | Source: LiteLLM /model/info + live benchmarks_
## Overview
@@ -12,7 +12,7 @@ Speed tiers are based on live measurements and published inference benchmarks.
| Provider | Models | Notes |
|----------|--------|-------|
| Claude (Anthropic via openai-compat) | 8 | claude-max subscription; aliases included |
| Claude (Anthropic via openai-compat) | 6 | claude-max subscription; temperature=0.7; aliases included |
| Mistral API | 10 | voxtral voice family + codestral + OCR |
| Gemini API | 12 | Flash/Pro/embedding families |
| SAIA (self-hosted GPU cluster) | 22 | OpenAI-compatible; local GPU inference |
@@ -41,11 +41,9 @@ Speed tiers are based on live measurements and published inference benchmarks.
| Model ID | Provider | Backend | Context | Vision | Tools | Cost In $/1M | Cost Out $/1M | Live Latency | Speed Tier | Notes |
|----------|----------|---------|---------|--------|-------|-------------|--------------|-------------|------------|-------|
| claude-sonnet-4-6 | Anthropic | openai-compat | 200K | ✓ | ✓ | $3.00 | $15.00 | 2 083 ms | 🔵 | Flagship balance model |
| claude-opus-4-7 | Anthropic | openai-compat | 200K | ✓ | ✓ | $15.00 | $75.00 | 3 218 ms | 🔵 | Highest capability Claude |
| claude-haiku-4-5 | Anthropic | openai-compat | 200K | ✓ | ✓ | $0.80 | $4.00 | 1 453 ms | 🔵 | Fast + cheap Claude tier |
| claude-sonnet-4-6-high | Anthropic | openai-compat | 200K | ✓ | ✓ | $3.00 | $15.00 | 1 746 ms | 🔵 | Extended thinking variant |
| claude-opus-4-7-high | Anthropic | openai-compat | 200K | ✓ | ✓ | $15.00 | $75.00 | 2 285 ms | 🔵 | Extended thinking variant |
| claude-sonnet-4-6 | Anthropic | openai-compat | 200K | ✓ | ✓ | $3.00 | $15.00 | 2 083 ms | 🔵 | Flagship; temp=0.7 |
| claude-opus-4-7 | Anthropic | openai-compat | 200K | ✓ | ✓ | $15.00 | $75.00 | 3 218 ms | 🔵 | Highest capability; temp=0.7 |
| claude-haiku-4-5 | Anthropic | openai-compat | 200K | ✓ | ✓ | $0.80 | $4.00 | 1 453 ms | 🔵 | Fast + cheap; temp=0.7 |
| voxtral-small-latest | Mistral | Mistral API | 256K | ✓ | ✓ | — | — | 160 ms | 🚀 | Voice+text multimodal |
| mistral-small-latest | Mistral | Mistral API | 131K | ✓ | ✓ | $0.06 | $0.18 | 222 ms | 🚀 | Cheapest Mistral chat |
| voxtral-mini-latest | Mistral | Mistral API | 100K | ✓ | ✓ | — | — | ERROR | ❌ | Proxy config error (saia-image-proxy unreachable) |
@@ -280,13 +278,13 @@ _Complex problem solving, research, multi-step tasks_
| Role | Model | Reasoning |
|------|-------|-----------|
| **Primary** | `claude-opus-4-7-high` | Extended thinking mode; highest reasoning capability available. |
| **Primary** | `claude-opus-4-7` | Highest Claude capability; temp=0.7. |
| **Cheaper** | `gemini-2.5-pro` | Strong reasoning at 888 ms, $1.25/$10.00; good for research tasks. |
| **Local reasoning** | `deepseek-r1-distill-llama-70b` | R1 chain-of-thought via SAIA at 325 ms; outputs `<think>` tokens. |
| **Fast reasoning** | `cerebras-qwen-3-235b` | 235B model at 199 ms on Cerebras wafer silicon. |
```
model: claude-opus-4-7-high # deep analysis
model: claude-opus-4-7 # deep analysis
model: deepseek-r1-distill-llama-70b # local reasoning
```
@@ -356,8 +354,6 @@ All measurements from `2026-05-23`. Single `max_tokens=5` completion, prompt: _"
| claude-sonnet-4-6 | 200 | 2 083 | 3 | 4 |
| claude-opus-4-7 | 200 | 3 218 | 6 | 6 |
| claude-haiku-4-5 | 200 | 1 453 | 10 | 41 |
| claude-sonnet-4-6-high | 200 | 1 746 | 3 | 4 |
| claude-opus-4-7-high | 200 | 2 285 | 6 | 6 |
| voxtral-small-latest | 200 | 160 | 8 | 2 |
| mistral-small-latest | 200 | 222 | 20 | 2 |
| voxtral-mini-latest | 500 | 48 | — | — |