dist : releases and versioning of ggml-org projects

[ggerganov]

Overview

Below is a summary of the release process of the ggml-org projects. The document should be useful for downstream project to decide how to build and distribute them.

Projects

ggml-org/ggml

• Core library used by the rest of the projects

• Release builds use semantic versioning: vX.Y.Z

  • Git tags: vX.Y.Z

• Release list: v0

  Downstreams

  • Homebrew: ggml package, formula
  • Debian:
    • ggml package, packaging repository
    • ggml-cuda package (separate build because of non-free CUDA dependencies)

ggml-org/llama.cpp

• Includes an internal copy of ggml that may deviate from the official released version

• The internal copy is used for nightly development builds from the master branch

• The nightly builds use commit number [NUM] for version:

  • Git tags: b[NUM]
  • Build with CMake option -DLLAMA_BUILD_IS_DEV=ON

• Official release builds use semantic versioning: vX.Y.Z

  • Git tags: vX.Y.Z
  • Build with CMake option -DLLAMA_BUILD_IS_DEV=OFF
  • The release tags are created on the condition that the internal ggml copy is identical to the released ggml copy. This guarantees that a downstream building from the release tags and linking to a system-wide ggml will be compatible.

• Release list: v0

  Downstreams

  • ggml-org: llama.app webpage, packaging repository
    • Does not link to a system-level ggml (i.e. packs the internal ggml copy)
    • Publishes the nightly builds instead of the official releases (TODO: switch to official releases?)
  • Homebrew: llama.cpp package, formula
    • Depends on the ggml package and links to the installed system-wide ggml
  • Debian: llama.cpp package, packaging repository
    • Depends on the ggml and ggml-cuda packages and links to the installed system-wide ggml
  • Winget: llama.cpp workflow, manifests
    • Does not link to a system-level ggml (i.e. packs the internal ggml copy)
    • Publishes the nightly builds instead of the official releases (TODO: switch to official releases?)

ggml-org/whisper.cpp

• Includes an internal copy of ggml that may deviate from the official released version

• There are no nightly builds - only official releases

• Official release builds use semantic versioning:

  • Git tags: vX.Y.Z
  • Build with CMake option -DWHISPER_BUILD_IS_DEV=OFF

• The release tags are created on the condition that the internal ggml copy is identical to the released ggml copy. This guarantees that a downstream building from the release tags and linking to a system-wide ggml will be compatible.

• Release list: v1

  Downstreams

  • Homebrew: whisper-cpp package, formula
    • TODO: rename to whisper.cpp?
  • Debian: whisper.cpp package, packaging repository

ggml-org/llama.cpp/gguf-py

• Release builds use semantic versioning: vX.Y.Z

  • Git tags: gguf-vX.Y.Z

  Downstreams

  • pip: gguf package, workflow

ggml-org/llama.vim

• Uses semantic versioning: vX.Y.Z

---

TODOs

• Update homebrew llama.cpp formula to build from release tag
• Add auto-generated summaries for the releases (new models, API changes, etc.)

[ggerganov]

FYI @ggml-org/ci @ckastner @p-linnane

[CISC]

May be worth mentioning gguf-py here as well?

[ckastner]

Also llama.vim?

[ggerganov]

Added, though I'm not sure how to spec it at the moment.

[ckastner]

This is very useful, thanks!

Super-happy to see semantic versioning for llama.cpp coming. That will also enable maintaining a SOVER. And that will allow us in Debian to ship it as a library in public search paths, because we can now guarantee backwards-compatibility (breaking can be communicated and maintained via SOVER bumps).

For llama.cpp and whisper.cpp release builds, it would be great if they could somehow specify the minimum ggml version they need, something like find_package(ggml 0.18 REQUIRED), so that packaging tools can make use of this.

Debian links:

• ggml package, packaging repository
• ggml-cuda package (separate build because of non-free CUDA dependencies)
• llama.cpp package, packaging repository
• whisper.cpp package, packaging repository
• llama.vim package, packaging repository

We also have HIP and CUDA builds using AMD's/NVIDIA's repositories, for bleeding edge hardware and toolkit support. I'd add that in an increment, though.

Below just FYI:

In Debian, we currently tend to update llama.cpp in tandem with new ggml release, since that was the point where both were expected to be ~"in sync":

• If ggml had a minor x.y bump, something key was added, and we increase llama.cpp's dependency on ggml version accordingly
• If ggml had a patch x.y.z bump, we assume a bugfix, and increase llama.cpp's dependency on ggml version only if a build shows it's needed
• We always verify this in practice, with various rebuilds and test runs

With this new release strategy, things are much more deterministic and plannable.

[ggerganov]

For llama.cpp and whisper.cpp release builds, it would be great if they could somehow specify the minimum ggml version they need, something like find_package(ggml 0.18 REQUIRED), so that packaging tools can make use of this.

Right now this can be deduced from the included internal copy of ggml. For example, for llama.cpp: https://github.com/ggml-org/llama.cpp/blob/master/ggml/CMakeLists.txt#L1-L10

But maybe this is too inconvenient for the package managers? We'll look into the find_package suggestion.

[ckastner]

find_package is just a bit more obvious as the build should already fail at the configuration stage, with a useful error message. And this works even outside of managed packages.

[ngxson]

I think the semantic version for llama.cpp makes perfect sense. Two things I want to bring to the discussion:

First: it will probably be better to have a list of requirements to bump major/minor/patch, to avoid forever sticking with something like 0.1.1234

I'm suggesting a initial version of the list for further discussions:

• Patch number bump when:
  • small bug fixes
  • vendor updates
• Minor bump when:
  • new model support
  • perf improvements
• Major bump when:
  • major changes in pubic API (i.e. new array of API is added, deprecated old one)
  • features added to server (which also changes the public REST API)

Second: we can setup a specific pipeline that tests the release against common models before releasing it

• To decide: what is the list of models? Maybe last N well known / most downloaded on HF, plus a fixed list of hand-picked models
• We may not have enough hw capacity to test on all backend, probably starts with CUDA & Metal for now?

[ngxson]

Interesting. At first glance, the 1.0.0 and 2.0.0 just look like additions to the REST API, that doesn't really breaking anything though, right? The 4.0.0 bump was because the /metrics endpoint got new fields, not sure about that.

Right, in general we rarely have breaking changes for server API, only addition (with backward compat), so it's true that the past 100 commit doesn't bump major number at all.

However, that would still mean at least ~50 minor bumps for a time span of 5 days. And if (for example) we choose to release once per week, that would mean (almost) always one minor bump per week, almost no patch bump (unless in the rare case of force triggering the release workflow).

So assuming 1 release per week, the releases will look almost like this:

0.1.0 --> first week
0.2.0 --> second week
0.3.0 --> third week
0.3.1 --> "force" release because 0.3.0 is broken
... (rarely a patch bump)
0.123.0 --> 123-th week after semver was adopted

That got me thinking about home assistant's calver scheme:

2026.1.0
2026.2.0
2026.2.1  --> "force" release for bug fixes
...
2026.8.0  --> latest

[ngxson]

If we choose to release every N commits instead of every week, the homebrew releases is a nice place to measure the speed: https://github.com/Homebrew/homebrew-core/pulls?q=sort:updated-desc+is:pr+llama.cpp+in:title

Things that I observed about this approach:

• We are merging about 10-20 commits per day, so if N less than 20 then it will be more frequent than nightly (which we don't want)
• Minor version bump almost always happen for a span of 10-20 commits, we rarely have 10-20 patches (bug fixes) in a row

[ckastner]

And if (for example) we choose to release once per week, that would mean (almost) always one minor bump per week, almost no patch bump (unless in the rare case of force triggering the release workflow).

One minor bump per week wouldn't really shock me that much, TBH. I wouldn't do more frequent than that, because at some point one just lands back at the nightlies. Perhaps "weekly" is even too strict, something like ~weekly, but shorter if some release needs it, or longer if stuff isn't ready yet.

The calver scheme you linked also looks nice. One can't really tell breaking changes from it, but for the library there is the SOVER to signal that independently, and for the tools -- well, maybe they don't need that signal?

[ngxson]

Agree, I think weekly release the as good as we can guaranteed for now. So these point are pretty much aligned for now:

• Most releases will be minor bump
• The release frequency is not strictly fixed, we can force-release. For example, some cases that I can think of:
  • If a model support is added, it's likely mean a new release --> likely be a minor bump
  • If the release is broken somehow --> likely be a patch bump
• Occasionally major release on these changes:
  • Breaking changes in ABI. Note that we usually try to have function calls backward-compat, meaning new libllama calls don't mean breaking change; however adding a new enum value will still be a breaking change
  • Do we cover llama-ext.h header? or only the llama.h?
  • For server API: only breaking change to inference endpoints (completions, embeddings) are considered major bump. Breaking changes to other things: router API, tools, CORS proxy are patches
  • Changes to components like llama-cli, llama-ui, llama-quantize are always (but not strictly) minor bump

I'll try to run another simulation tomorrow

[ggerganov]

Do we cover llama-ext.h header? or only the llama.h?

The purpose of the llama-ext.h is to be able to experiment with some API changes without affecting the official API of the library and therefore without affecting the minor and major versions. The llama-ext.h is not public so we can change it without announcing this with a version bumps.

[mbaudier]

Will this versioning policy also cover mtmd?

More generally, is it planned that libmtmd would be maintained as a library, like libllama? (vs. in tools/ as is currently the case)
Or will it be merged into libllama at some point?

[ggerganov]

I wish one day we will merge it in libllama, but it might take a while. For now it seems much simpler to remain separate. Though we probably should treat its API the same way as libllama's API and reflect changes in the semantic version accordingly. @ngxson WDYT?

[ngxson]

Good point. I don't have a good idea for now, but I'm considering about this:

Since our goal is to merge libmtmd and libllama into one package, it might be important to always keep the libmtmd version less than libllama until we merge them together.

One idea I'm thinking about: libmtmd will have versioning with major = 0 for now, and I will only bump the minor+patch. Then when we merge it to libllama, the major version will be increase to match libllama (in other words: if we bump mtmd major number right now, when we merge it, we will end up with mtmd's major > llama's major, which will be a messy situation)

[ggerganov]

I meant that we can have a single semantic version that tracks both libllama and libmtmd. So basically as if the llama.h and mtmd.h were part of the same API. Changes in either API results in version update.

[danbev]

I've opened ggml-org/llama.cpp#26839 and libmtmd in which both use the same semantic version.

[mbaudier]

As a consumer (vs. as a packager) of the libraries only, that is, developing directly on top of libllama and libmtmd, this is indeed one same API. The separation currently makes sense because libmtmd is, as stated in the doc, slightly less stable than libllama, and that makes it easier to track the dependencies (simply check the headers). +1 for using the same semantic version, as it will be easier for everyone to manage.

[ggerganov]

Next week will try to implement some automation with @danbev according to the spec above to start semver tagging of llama.cpp and utilizing find_package for discovering the correct ggml version. Until then we can iterate on the spec and discuss.

[danbev]

I've made a start on find_package which can be found here #1582

[danbev]

@ggerganov Regarding the release branches mentioned above. I was thinking of using the current build flow but with the addition of verifying that the ggml version in llama.cpp matches the released version for llama.cpp release builds, and do this in a github action. Basically not having a separate release branch is what I'm suggesting/asking, but perhaps I'm overlooking some additional benefit of using a separate release branch?

[ggerganov]

Yes, I also realized that we don't need the release branches. I will update the spec accordingly to simplify.

[p-linnane]

For Homebrew we perform regression tests of dependents when updating formulae. If a new version of ggml was released that llama.cpp didn't work with, CI would fail. Likewise, as seen in our latest bump, if a version of llama.cpp was released that didn't work with the current ggml, CI would fail.

[danbev]

@ggerganov Do you think we are in a good enough shape with the llama.cpp changes that I should start looking at whisper.cpp and introduce the same/similar changes. I'm mainly thinking about the ggml check, as it would be possible to keep the current release build that happens upon tag creation/push if we wanted to. But perhaps it would be better to make this consistent with llama.cpp and introduce the same changes?

[mbaudier]

@danbev I had a look at talk-llama's CMakeLists.txt in your branch. From what I understand, a given llama.cpp tag is fetched and built, allowing then to use libllama as a private library. (This is essentially what I do too when developing the JJML Java bindings, with ggml, llama.cpp, and whisper.cpp as git submodules)

What I can already tell you is that, from a Debian packaging perspective, we will want to use the packaged libllama. So, it would be very useful if we could from the start have something like WHISPER_USE_SYSTEM_LLAMA (similar to WHISPER_USE_SYSTEM_GMML, or LLAMA_USE_SYSTEM_GGML in llama.cpp) which would completely bypass the git fetch and the llama.cpp build. If this variable is set, libllama would simply be referenced as a public library in target_link_libraries().

If you could make these changes, I would then test the packaging in a fork of the official Debian packages and we could make sure that we have clean Debian packages from day one.

[danbev]

@mbaudier I've taken a stab at this and added a new commit to ggml-org/whisper.cpp#3996. Could you take a look and give this a try and see if this is what you had in mind?

[mbaudier]

@danbev I built your branch manually on Debian sid/unstable, and it looks good.

sudo apt install libggml-dev libllama-dev libsdl2-dev
cmake -B build/ \
 -DWHISPER_SDL2=ON \
 -DWHISPER_USE_SYSTEM_LLAMA=ON \
 -DCMAKE_PREFIX_PATH=/usr/lib/x86_64-linux-gnu/llama/cmake-private
cmake --build build/ -j $(nproc)

The executable is properly linked to the deployed libllama:

  ldd build/bin/whisper-talk-llama
    ...
    libllama.so.0 => /usr/lib/x86_64-linux-gnu/llama/libllama.so.0 (0x00007f10a4a00000)
    ...
    libggml.so.0 => /usr/lib/x86_64-linux-gnu/libggml.so.0 (0x00007f10a4db3000)
    libggml-base.so.0 => /usr/lib/x86_64-linux-gnu/libggml-base.so.0 (0x00007f10a486c000)
    ...

Once we have llama.cpp packages with the new versioning scheme, CMAKE_PREFIX_PATH won't be needed any more, and libllama will be available directly in /usr/lib/<arch>-linux-gnu (like libggml and libwhisper). The includes and CMake configs will also be in the standard locations expected by default.

As soon as we have finalised these new llama.cpp packages, we will enable building and distributing whisper-talk-llama in the whisper.cpp packages. So, from my perspective, this is good to merge. Thanks a lot!

[danbev]

@mbaudier Awesome! Thanks for testing and verifying this!

[mbaudier]

@danbev whisper.cpp v1.9.3 is now available in Debian sid/unstable and provides whisper-talk-llama, which uses the packaged llama.cpp deployed on the system. This closes the related Debian enhancement request.

[ggerganov]

Semantic version in llama.cpp is pretty much ready: https://github.com/ggml-org/llama.cpp/releases/tag/v0.1.2

Marked as "pre-release" for now, until we do a few quick release rounds in the next few days to make sure everything is good. After that, we'll remove the "pre-release" flag and will aim for a slow-paced, steady release cycle.

[danbev]

We have done the same/similar now for whisper.cpp: https://github.com/ggml-org/whisper.cpp/releases/tag/v1.9.3

[danbev]

And now also for llama.vim: https://github.com/ggml-org/llama.vim/releases

[ckastner]

Perfect, thanks!

I wonder, will llama.vim also need some way to communicate a minimum version of the llama.cpp it needs? Even if it's just a .txt or whatever. Or is that generally not an issue?

[danbev]

Regarding llama.vim, I think we could simply start tagging using semantic versioning and that would allow plugin managers to be able to specify a semver tag instead of today where I think the only choice is to follow the master branch, or a specific commit.

[ggerganov]

Yes, we can do that.

[ckastner]

When cutting releases, could we also have the web assets for that release as vX.Y.Z in the HF bucket?

[ckastner]

However, we are thinking about dropping the HF bucket distribution of the WebUI because we already distribute the UI with the Github releases: ggml-org/llama.cpp#27316. Any concerns?

Oh, I missed that. That is fine, I'll switch to that.

Context: we can't ship the WebUI directly with the Debian package because of a technicality: by Policy, all assets need to be rebuildable from source from within Debian, and some of the npm dependencies aren't packaged yet.

So the workaround our package offers is to simply download the UI at package install/update time. We have a second worarkound in progress (a standalone package in the contrib section).

I think they are already there currently as part of the corresponding nightly build?

I wonder: can the corresponding build number be made available as part of the tarballs? The nightly tarballs happen to have the number in the directory name, but nothing in the 0.1.2 tarball suggests b10485.

Just thinking aloud here. If (big IF) there is an easy way, this could be generally useful, for example by initializing BUILD_NUMBER to something other than 0, in the tarballs.

[ggerganov]

Would it work if we also attach the UI asset from the corresponding nightly build along with the source-code tarball in the release page: https://github.com/ggml-org/llama.cpp/releases/tag/v0.1.2

[ggerganov]

I just manually updated the v0.1.2 pre-release by attaching the UI asset to clarify what I mean.

Alternatively, we can include just a simple text file nightly.tag or nightly-tag.txt and it can contain the corresponding tag - in this case: b10485. From there, one could download the asset.

[ckastner]

Yes, both would work. Thanks!

I think having nightly.tag[.txt] is more generally useful, we'd grab it and feed LLAMA_BUILD_NUMBER for example.

Also shipping the UI assets is convenient, but also duplication. I'm not sure the extra convenience (saving 1-2 curl calls) is worth the effort.

[ggerganov]

Ok, going with the nightly-tag.txt then.

[ggerganov]

I think this doc is now complete. Any feedback for improvements is welcome. I've added a few TODOs to the OP for later.

[AHeinlein]

Is it intentional or by accident that the point release assets contain neither prebuilt binaries nor the UI package? I see that the build process is trying to download the UI files at build time - I don't like that because I usually build in isolated environments. It also means that v0.4.0 built now may differ from 0.4.0 built next week or next month - the build script tries to download a version "b0" first from HF, which doesn't exist, and then the latest version which I guess is subject to change. If possible please provide UI files with the point releases or include them in the source package.

[ggerganov]

Is it intentional or by accident that the point release assets contain neither prebuilt binaries nor the UI package?

Yes, the point releases are just a semantic version tag. Each such release also points to a respective nightly build. For example https://github.com/ggml-org/llama.cpp/releases/tag/v0.4.0 points to nightly build https://github.com/ggml-org/llama.cpp/releases/tag/b10809. This nightly build has all the corresponding assets.

I see that the build process is trying to download the UI files at build time - I don't like that because I usually build in isolated environments.

This can be disabled with LLAMA_BUILD_UI=OFF cmake option.

It also means that v0.4.0 built now may differ from 0.4.0 built next week or next month - the build script tries to download a version "b0" first from HF, which doesn't exist, and then the latest version which I guess is subject to change.

That should not be the case - the bXXXX UI builds always exist for the point releases.

If possible please provide UI files with the point releases or include them in the source package.

They are already provided by following the included nightly tag.

[jameschristopherfrederick18-star]

This helps alot