Getting started | JeffHub docs

Jeff is a small decision model. You describe a situation and list the options in plain words; Jeff returns a probability for each option from one forward pass. An adapter is a small add-on (41.5 MB) that tunes Jeff for one job, such as naming the intent behind a customer message or the type of a contract clause. One server loads the base model once and any number of adapters beside it. Jeff v1.3 is meant to be used with an adapter (why).

Prefer llama.cpp? See Running Jeff with llama.cpp.

1. Install the server

You need uv and a clone of the Jeff repository.

git clone https://github.com/firelex/jeff && cd jeff
uv sync --no-default-groups --extra lora                # CPU
uv sync --no-default-groups --extra lora --extra cuda   # NVIDIA GPU
uv sync --no-default-groups --extra lora --extra mac    # Apple silicon (MLX)

2. Download the base model

The base is jeff-base (v1.3), a fine-tune of Qwen3.5-0.8B. Each adapter has its own repository, jeff-adapter-<name>. Nothing under these names is published yet; the commands work once they are.

uv run --no-default-groups hf download mstrasser/jeff-base --revision v1.3 --local-dir checkpoints/jeff-base

3. Download an adapter

Each adapter lives in its own folder inside one adapters folder. The folder's name is the name you use in requests.

uv run --no-default-groups hf download mstrasser/jeff-adapter-support-intents --revision v1.3 \
  --local-dir adapters/support-intents

4. Start the server

JEFF_CHECKPOINT=checkpoints/jeff-base JEFF_ADAPTERS=adapters/ PORT=8765 \
  uv run --no-default-groups jeff-serve

On a Mac, add JEFF_BACKEND=mlx for much faster answers.

5. Ask a question

Name the adapter as the model. Any other name the server does not know is refused.

curl -s localhost:8765/v1/systemone -H 'content-type: application/json' -d '{
  "model": "support-intents",
  "state": {"service": "Customer support chat of an online shop",
            "message": "I sent the jacket back two weeks ago and still have not seen the money."},
  "questions": {
    "intent": {"type": "choice",
               "instructions": "What does the customer want? Choose the request that best matches what the customer is asking for in their message.",
               "criteria": {"track_refund": "Check the status of a refund they are expecting.",
                            "get_refund": "Get their money back for a purchase.",
                            "track_order": "Find out where their order is or its current status."}}
  }
}'

Every adapter's page has a full example in Python, TypeScript and curl. Next, read how requests are built.