[hussein1362]

Problem

Local model servers (Ollama, llama.cpp, vLLM) close idle HTTP connections on their own timeout schedule. When two LLM calls happen seconds apart — for example the heartbeat _decide() phase followed immediately by process_direct() — the second call reuses a pooled connection that the server has already closed, causing a transient APIConnectionError on every first attempt.

Observed behavior: 100% first-attempt failure rate on heartbeat execution when using llama.cpp over LAN (192.168.x), with successful retry every time. The retry adds ~1-6 seconds of unnecessary latency to every heartbeat cycle.

08:45:21 | Heartbeat: tasks found, executing...
08:45:27 | LLM transient error (attempt 1/3), retrying in 1s: connection error.
09:18:40 | Heartbeat: tasks found, executing...
09:18:46 | LLM transient error (attempt 1/3), retrying in 1s: connection error.
...  (repeats every heartbeat)

Root Cause

The AsyncOpenAI client uses httpx with default connection pooling (keepalive_expiry=5.0s). After the first LLM call completes (~30s for a local model), the connection returns to the pool. The next call a few seconds later finds the connection in the pool, but the server (llama.cpp with Keep-Alive: timeout=5) may have already closed it. httpx detects the dead connection only after attempting to use it → APIConnectionError.

Fix

Detect local endpoints via two mechanisms:

1. ProviderSpec.is_local — covers Ollama, LM Studio, vLLM, OVMS
2. Private-network URL heuristic — covers the common case where users configure provider: openai with apiBase pointing at a LAN IP (localhost, 127.x, 192.168.x, 10.x, 172.16-31.x, host.docker.internal, [::1])

For detected local endpoints, create the AsyncOpenAI client with a custom httpx.AsyncClient that sets keepalive_expiry=0, forcing a fresh TCP connection per request. This is cheap on LAN (sub-5ms connect time) and eliminates the stale-connection retry entirely.

Cloud providers keep the default 5s keepalive.

Tests

21 new tests covering:

• _is_local_endpoint() detection for all private network patterns
• Keepalive configuration verification for local, LAN, and cloud endpoints
• Edge cases: IPv6 loopback, Docker internal, 172.16-31 range boundaries

All 295 provider tests pass (274 existing + 21 new).

[Re-bin]

This is the right fix for the local/LAN provider path.

The problem is real: local model servers can close idle HTTP connections before the OpenAI/httpx client notices, so reusing pooled keepalive connections creates predictable first-attempt APIConnectionErrors. Disabling keepalive only for local endpoints is a narrow fix and keeps cloud providers on the normal pooled behavior.

I pushed one small follow-up to tighten the endpoint detection. The original heuristic used substring matching, which could classify public hostnames like notlocalhost.example or api10.example.com as local. The updated version parses the URL hostname first, then uses ipaddress for loopback/private IPs, plus explicit localhost / host.docker.internal handling.

Validated locally:

• python -m pytest tests/providers/test_local_endpoint_detection.py -q
• python -m pytest tests/providers -q
• python -m ruff check nanobot/providers/openai_compat_provider.py tests/providers/test_local_endpoint_detection.py
• git diff --check origin/main...HEAD

From my side, the behavior is now well-scoped and ready to merge once CI is green.