Repository navigation
reduce system prompt overhead - #654
Merged
Lazarus-931 merged 4 commits intoOct 7, 2026
Merged
Conversation
Lazarus-931
marked this pull request as ready for review
October 3, 2026 00:15
Collaborator
|
@Lazarus-931 Can you file an issue for the DeepSeek model failure? Otherwise this looks fine to me. |
lucasnewman
approved these changes
Oct 7, 2026
Contributor
Author
I'll follow up with that! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
this shortens the built-in system prompt so nativ sends fewer prompt tokens during prefill while keeping the same profile, style, tool, browser, terminal, and safety instructions.
i tested the original and shorter prompts on five model sizes: qwen3-4b, llama-3.1-8b, glm-4.7-flash, qwen3.5-122b-a10b, and deepseek-v4-flash optiq. the shorter prompt matched the original semantic score on every model in this test suite, used around 46% fewer prompt tokens, and reduced median uncached prefill by 42.5% in this benchmark.
i also compared the original pr 635 chat_work prompt with the rewritten routing-focused version using exact tool-name, action, argument, generated-content, and plain-text assertions. the original and rewritten scores were:
deepseek loaded and passed the plain-text case, while its six tool cases hit the same engine keyerror in both arms.
across that matrix, the rewritten chat_work prompt scored 47/84 versus 38/84 for the original, used 13.7% fewer prompt tokens on average, and reduced estimated prefill by 15.1%. it matched or improved the original score on every model in the matrix.
llama 8b scored 26/27 with both prompts because it labeled one meta-question incorrectly, but its response still followed the requested terse, no-emoji, unformatted style. these numbers are from these model benchmarks and are not meant as a universal speed claim.
the nativ app builds and 78 focused NativSettingsTests and ChatWorkTests pass.