Skip to content

fix(build): compile the ggml objects with -DNDEBUG - #1050

Merged
aittalam merged 1 commit into
mainfrom
add-dndebug
Sep 11, 2026
Merged

aittalam merged 1 commit into
mainfrom
add-dndebug

Conversation

@aittalam

Copy link
Copy Markdown
Member

Description

llama.cpp/BUILD.mk used the -DNDEBUG flag with llama and common targets, but not with the ggml
objects, so plain assert() stayed live in the CPU hot path (upstream's Release build, instead, compiles them out).

-$(LLAMA_OBJS) $(COMMON_OBJS): private CCFLAGS += -DNDEBUG
+$(LLAMA_CPP_OBJS): private CCFLAGS += -DNDEBUG

LLAMA_CPP_OBJS is the existing umbrella variable (GGML + LLAMA + COMMON + GGUF).

What was running: 25 #ifndef NDEBUG blocks in ops.cpp that re-read each FFN
activation output row checking isnan/isinf (swiglu, silu, gelu, geglu…), plus
per-row asserts in the quant dot-products.

Measured (token generation, interleaved A/B, medians):

path gain why
CPU x86 (9B / 27B) +1.25% / +1.19% ggml built here — was missing the flag
CPU arm64 (9B) +1.00% same
CUDA +0.34% kernels already had it; only shared host code gained
Metal 0.00% metal.c rebuilds ggml at runtime, already with -DNDEBUG

Binary shrinks 77–94 KB. make check passes.

Trade-off: two GGML_LOG_DEBUG diagnostics go quiet (ggml-backend.cpp:496
"slow copy", :899 "no backend supports op"), both useful when debugging offload.
Same as upstream Release.

Not covered: whisper.cpp and transcribe.cpp build their own ggml copies and
still lack the flag — separate PR. stable-diffusion.cpp reuses $(GGML_OBJS) and
benefits for free.

PR Type

  • 💅 Refactor

Relevant issues

#980 - see this in particular, TY @dexhunter !

Checklist

  • I understand the code I am submitting.
  • I have run this code locally and verified the change.
  • New and existing tests pass locally, or I have explained why tests were not run.
  • I have read and followed the contribution guidelines.
  • AI Usage:
    • No AI was used.
    • AI was used in an assistive capacity (to run tests for the 1-line manual change)
    • This PR includes substantial AI-generated content.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@aittalam aittalam self-assigned this Sep 11, 2026
@aittalam
aittalam merged commit 6719ace into main Sep 11, 2026
2 checks passed
@aittalam
aittalam deleted the add-dndebug branch September 11, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant