From 396aa0e48ee39991a232dad269903b9d11b6530a Mon Sep 17 00:00:00 2001 From: "fkrebs (via Claude)" Date: Sat, 23 May 2026 00:41:26 +0200 Subject: [PATCH] docs: remove claude high-effort variants, set temperature=0.7 on Claude models (58 total) --- services/llm-benchmark.md | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/services/llm-benchmark.md b/services/llm-benchmark.md index 338a471..098e7bd 100644 --- a/services/llm-benchmark.md +++ b/services/llm-benchmark.md @@ -1,6 +1,6 @@ # LLM Model Benchmark & Service Catalogue -_Last updated: 2026-05-23 | Models: 60 | Source: LiteLLM /model/info + live benchmarks_ +_Last updated: 2026-05-23 | Models: 58 | Source: LiteLLM /model/info + live benchmarks_ ## Overview @@ -12,7 +12,7 @@ Speed tiers are based on live measurements and published inference benchmarks. | Provider | Models | Notes | |----------|--------|-------| -| Claude (Anthropic via openai-compat) | 8 | claude-max subscription; aliases included | +| Claude (Anthropic via openai-compat) | 6 | claude-max subscription; temperature=0.7; aliases included | | Mistral API | 10 | voxtral voice family + codestral + OCR | | Gemini API | 12 | Flash/Pro/embedding families | | SAIA (self-hosted GPU cluster) | 22 | OpenAI-compatible; local GPU inference | @@ -41,11 +41,9 @@ Speed tiers are based on live measurements and published inference benchmarks. | Model ID | Provider | Backend | Context | Vision | Tools | Cost In $/1M | Cost Out $/1M | Live Latency | Speed Tier | Notes | |----------|----------|---------|---------|--------|-------|-------------|--------------|-------------|------------|-------| -| claude-sonnet-4-6 | Anthropic | openai-compat | 200K | ✓ | ✓ | $3.00 | $15.00 | 2 083 ms | 🔵 | Flagship balance model | -| claude-opus-4-7 | Anthropic | openai-compat | 200K | ✓ | ✓ | $15.00 | $75.00 | 3 218 ms | 🔵 | Highest capability Claude | -| claude-haiku-4-5 | Anthropic | openai-compat | 200K | ✓ | ✓ | $0.80 | $4.00 | 1 453 ms | 🔵 | Fast + cheap Claude tier | -| claude-sonnet-4-6-high | Anthropic | openai-compat | 200K | ✓ | ✓ | $3.00 | $15.00 | 1 746 ms | 🔵 | Extended thinking variant | -| claude-opus-4-7-high | Anthropic | openai-compat | 200K | ✓ | ✓ | $15.00 | $75.00 | 2 285 ms | 🔵 | Extended thinking variant | +| claude-sonnet-4-6 | Anthropic | openai-compat | 200K | ✓ | ✓ | $3.00 | $15.00 | 2 083 ms | 🔵 | Flagship; temp=0.7 | +| claude-opus-4-7 | Anthropic | openai-compat | 200K | ✓ | ✓ | $15.00 | $75.00 | 3 218 ms | 🔵 | Highest capability; temp=0.7 | +| claude-haiku-4-5 | Anthropic | openai-compat | 200K | ✓ | ✓ | $0.80 | $4.00 | 1 453 ms | 🔵 | Fast + cheap; temp=0.7 | | voxtral-small-latest | Mistral | Mistral API | 256K | ✓ | ✓ | — | — | 160 ms | 🚀 | Voice+text multimodal | | mistral-small-latest | Mistral | Mistral API | 131K | ✓ | ✓ | $0.06 | $0.18 | 222 ms | 🚀 | Cheapest Mistral chat | | voxtral-mini-latest | Mistral | Mistral API | 100K | ✓ | ✓ | — | — | ERROR | ❌ | Proxy config error (saia-image-proxy unreachable) | @@ -280,14 +278,14 @@ _Complex problem solving, research, multi-step tasks_ | Role | Model | Reasoning | |------|-------|-----------| -| **Primary** | `claude-opus-4-7-high` | Extended thinking mode; highest reasoning capability available. | +| **Primary** | `claude-opus-4-7` | Highest Claude capability; temp=0.7. | | **Cheaper** | `gemini-2.5-pro` | Strong reasoning at 888 ms, $1.25/$10.00; good for research tasks. | | **Local reasoning** | `deepseek-r1-distill-llama-70b` | R1 chain-of-thought via SAIA at 325 ms; outputs `` tokens. | | **Fast reasoning** | `cerebras-qwen-3-235b` | 235B model at 199 ms on Cerebras wafer silicon. | ``` -model: claude-opus-4-7-high # deep analysis -model: deepseek-r1-distill-llama-70b # local reasoning +model: claude-opus-4-7 # deep analysis +model: deepseek-r1-distill-llama-70b # local reasoning ``` --- @@ -356,8 +354,6 @@ All measurements from `2026-05-23`. Single `max_tokens=5` completion, prompt: _" | claude-sonnet-4-6 | 200 | 2 083 | 3 | 4 | | claude-opus-4-7 | 200 | 3 218 | 6 | 6 | | claude-haiku-4-5 | 200 | 1 453 | 10 | 41 | -| claude-sonnet-4-6-high | 200 | 1 746 | 3 | 4 | -| claude-opus-4-7-high | 200 | 2 285 | 6 | 6 | | voxtral-small-latest | 200 | 160 | 8 | 2 | | mistral-small-latest | 200 | 222 | 20 | 2 | | voxtral-mini-latest | 500 | 48 | — | — |