[Re-bin]

Summary

Add end-to-end streaming output infrastructure across the entire stack: LLM providers → agent loop → channels / CLI. Content is delivered token-by-token instead of waiting for the full response, significantly improving perceived latency.

Telegram is the first channel to ship with real streaming support (default on). Other built-in channels (Discord, Slack, Feishu, DingTalk, etc.) remain unchanged and can be adapted incrementally — the infrastructure is ready, each just needs a send_delta() override.

What Changed

1. Provider Layer — chat_stream / chat_stream_with_retry

All four providers now implement native token-by-token streaming:

Provider	Streaming Implementation
LiteLLMProvider	acompletion(stream=True) + async chunk iteration
CustomProvider	httpx.AsyncClient().stream("POST", ...) + SSE line parsing
AzureOpenAIProvider	AsyncAzureOpenAI.chat.completions.create(stream=True)
OpenAICodexProvider	AsyncOpenAI.chat.completions.create(stream=True)

Base class (LLMProvider) provides two methods:

• chat_stream() — abstract with default fallback: calls chat() and delivers the full content as a single delta. Providers that don't override it still work.
• chat_stream_with_retry() — retry wrapper with exponential backoff, mirrors the existing chat_with_retry() behavior.

Files: nanobot/providers/base.py, litellm_provider.py, custom_provider.py, azure_openai_provider.py, openai_codex_provider.py

2. Agent Loop — Stateful Think-Tag Filter + Delta Dispatch

_run_agent_loop() now accepts optional on_stream / on_stream_end callbacks. When present, it calls chat_stream_with_retry instead of chat_with_retry.

Key design: think-tag filtering happens once at the source. A stateful wrapper in the loop accumulates raw LLM output, applies strip_think(), and only emits the clean incremental text. No downstream consumer (CLI, Telegram, any future channel plugin) ever sees <think> tags.

strip_think() is defined in nanobot/utils/helpers.py as the single source of truth, replacing three previously duplicated implementations.

In _dispatch(), bus-backed callbacks are built when _wants_stream is set on the inbound message. Outbound messages carry these metadata flags:

Metadata Flag	Meaning
_stream_delta: True	A content chunk (delta contains new text)
_stream_end: True	Streaming round finished
_resuming: True	More rounds coming (tool calls in progress, spinner should restart)
_streamed: True	Final response — content already delivered via deltas, send() is skipped

Files: nanobot/agent/loop.py, nanobot/utils/helpers.py

3. Channel Infrastructure — BaseChannel.send_delta + supports_streaming

BaseChannel gains two new members:

• async send_delta(chat_id, delta, metadata) — no-op by default. Override in subclass to handle streaming chunks.
• supports_streaming (property) — returns True only when both conditions are met:
  1. Channel config has "streaming": true
  2. Subclass actually overrides send_delta()

_handle_message() automatically sets _wants_stream in message metadata when supports_streaming is true. If either condition fails, the agent loop falls back to the standard non-streaming chat_with_retry → send() path. Existing channels are completely unaffected.

ChannelManager._dispatch_outbound() routes messages based on metadata:

• _stream_delta / _stream_end → channel.send_delta()
• _streamed → skip (already delivered)
• everything else → channel.send() (unchanged)

Files: nanobot/channels/base.py, nanobot/channels/manager.py, nanobot/config/schema.py

4. Telegram — Real Streaming via Progressive Message Editing

Replaces the previous fake streaming (send_message_draft simulation) with real progressive output:

1. First delta → send_message (creates the message, captures message_id)
2. Subsequent deltas → edit_message_text (throttled to every 0.6s to respect Telegram rate limits)
3. Stream end → final edit_message_text with full HTML formatting via _markdown_to_telegram_html

When streaming: false, Telegram sends a single one-shot message — no fake streaming, no draft simulation. Clean and simple.

TelegramConfig adds streaming: bool = True (default on for new installs).

Removed: the entire _send_with_streaming() method and send_message_draft logic.

File: nanobot/channels/telegram.py

5. CLI — StreamRenderer + ThinkingSpinner

Extracted to a dedicated nanobot/cli/stream.py module to keep commands.py focused on orchestration.

• ThinkingSpinner: shows "nanobot is thinking..." with pause() support for clean progress output
• StreamRenderer: uses Rich Live with auto_refresh=False for manual-refresh control. Renders accumulated Markdown on each delta. Spinner starts on init, stops on first visible content, restarts between tool call rounds.

Both single-message mode (nanobot agent -m "...") and interactive mode wire up StreamRenderer.on_delta / on_end as callbacks. Falls back to one-shot _print_agent_response if streaming didn't occur.

Files: nanobot/cli/stream.py, nanobot/cli/commands.py

6. Documentation — Channel Plugin Streaming Guide

Updated docs/CHANNEL_PLUGIN_GUIDE.md with a new Streaming Support section covering:

• How supports_streaming works (dual condition: config + implementation)
• send_delta() method signature and metadata flags reference
• Complete example: webhook channel with streaming
• Config example for enabling streaming per channel
• Updated BaseChannel API table with send_delta and supports_streaming

For channel plugin authors who want to add streaming: see docs/CHANNEL_PLUGIN_GUIDE.md — Streaming Support section.

Channels Streaming Status

Channel	Streaming	Notes
Telegram	✅ Real streaming	Progressive edit_message_text, default on
CLI	✅ Rich Live	Markdown rendering with spinner
Discord	⬚ Not yet	Has edit_message API, good candidate
Slack	⬚ Not yet	Has chat.update API, good candidate
Feishu	⬚ Not yet	Has message update API
DingTalk	⬚ Not yet	Has message update API
Others	⬚ Not yet	Infrastructure ready, just needs send_delta() override

Each channel can be enabled independently. The infrastructure is fully in place — adding streaming to a new channel requires:

1. Override send_delta() in the channel class
2. Add streaming: bool to the channel's config model
3. That's it. The loop, bus, and manager handle the rest.

Test Plan

• 559 tests pass, 0 lint errors
• Manual: nanobot agent CLI streaming renders markdown correctly
• Manual: Telegram streaming progressively updates messages, no <think> tags visible
• Manual: Telegram with streaming: false sends clean one-shot messages