Together AI Fine-Tuning: 12 Steps, 70% Cost Cut [2026]

Together AI pushed a major update to its hosted fine-tuning platform on September 11, 2026, and the release changes how small teams can customize open-weight models without owning a GPU cluster. The update adds support for 17 new open-weight models, a Mixture-of-Experts-specific technique called Expert LoRA, a live training dashboard, automatic early stopping, arbitrary batch sizes, and fine-tuning price cuts of up to 70% on select models. This tutorial walks through the entire workflow: picking a base model, preparing and validating a dataset, configuring Expert LoRA for MoE architectures, launching a job, watching it train in real time, and deploying the result to a dedicated inference endpoint.

By the end, you will have a working fine-tuned model running on a hosted endpoint you can call from Python, along with a checklist for avoiding the mistakes that waste the most GPU-hours. This guide targets developers who already know basic Python and have used an LLM API before, but have never run a fine-tuning job of their own.

·

Google · Preferred Sources

Don't miss new tech stories on Google

Add Tech Insider once in the Google app and our stories appear in your news suggestions.

Add Now

What Changed in Together AI’s September 2026 Fine-Tuning Update

Together AI folded seven separate additions into a single push on September 11, 2026, according to release tracking site BriefFlash. The company’s own changelog confirms that newly supported models include zai-org/GLM-5.3, joining a roster that already covered OpenAI’s gpt-oss family, recent DeepSeek releases, Qwen checkpoints, and Meta’s Llama line. That expansion matters because fine tune llm workflows only stay useful if the underlying base models keep pace with what labs actually ship each month, and September 2026 alone saw at least 18 confirmed model releases across eight labs, a pace we track in our roundup of the best AI models of 2026.

The headline additions in the update are practical rather than flashy. A live experiment tracking dashboard now shows loss curves and evaluation scores while a job runs, instead of forcing developers to poll a REST endpoint or tail a log file. Expert LoRA, a Mixture-of-Experts-specific low-rank adaptation variant, lets you fine-tune only the experts that matter for your task while leaving the rest of a MoE model frozen, which keeps compute and cost down for models like GLM-5.3 and Ling-3.0-flash that use sparse expert routing. Dataset preview and validation tools catch formatting and prompt-label issues in the UI before a job burns GPU time on a broken dataset. Automatic early stopping halts a run when validation metrics plateau, and arbitrary batch sizes remove a constraint that used to force developers into a handful of preset configurations.

Together AI’s own documentation describes the platform’s scope plainly: the company handles the full lifecycle of a fine-tuning job, meaning data upload, training, hosting, and inference on a dedicated endpoint all live inside one product instead of being stitched together from separate tools. That end-to-end scope is the main reason this update is worth a dedicated walkthrough: the pieces you used to have to wire together by hand (a training script, a metrics dashboard, a model registry, an inference server) are now one continuous flow.

Prerequisites and Versions

Before starting, confirm you have the following in place. Version mismatches are the most common source of failed setup in fine-tuning workflows, so check each one before moving to Step 1.

• A Together AI account with billing enabled (a payment method is required to run paid fine-tuning jobs; the free tier only covers inference on shared models)
• Python 3.10 or newer installed locally
• The together Python SDK, version 1.5 or later (pip install --upgrade together)
• A Together AI API key, generated from the account dashboard
• A training dataset in JSONL format, with at least a few hundred examples for a meaningful fine-tune (Together AI’s docs recommend larger datasets for full fine-tunes and allow smaller datasets for LoRA-based approaches)
• curl or an HTTP client for testing the deployed endpoint
• Roughly 60-90 minutes for a small-to-medium LoRA job to complete, though full fine-tunes on larger base models take longer

You do not need your own GPUs. That is the entire point of a managed fine-tuning platform: Together AI provisions the training hardware, and you pay per token processed during training, with additional charges for hosting a dedicated inference endpoint once the fine-tune completes.

Step 1: Create Your Together AI Account and API Key

Sign up at together.ai and verify your email. Once inside the dashboard, navigate to the API Keys section and generate a new key. Store it as an environment variable rather than hardcoding it into any script you plan to commit to version control.

export TOGETHER_API_KEY="your_api_key_here"
pip install --upgrade together
python3 -c "import together; print(together.__version__)"

Add a payment method under Billing before moving further. Fine-tuning jobs are billed by tokens processed during training, and the dashboard will block job submission without a valid payment method attached, even if you plan to stay within a free trial credit balance.

Step 2: Pick a Base Model From the Expanded Roster

The September 11 update added 17 open-weight models to the fine-tunable list. Rather than guessing which models are supported, query the API directly, since the roster changes with every changelog entry.

import together

client = together.Together()
models = client.models.list()

for m in models:
    if getattr(m, "type", None) == "chat" and getattr(m, "finetune", False):
        print(m.id)

For this walkthrough, we will fine-tune a mid-sized open-weight chat model on a customer support dataset. If your target task involves a Mixture-of-Experts model such as zai-org/GLM-5.3, keep that in mind for Step 6, where Expert LoRA changes the configuration slightly. For a dense (non-MoE) model, the steps below apply directly with standard LoRA.

Model Family	Architecture	Best Fit For	Fine-Tune Method
zai-org/GLM-5.3	Mixture-of-Experts	Reasoning, coding, agentic tasks	Expert LoRA
Meta Llama family (open weights)	Dense transformer	General chat, instruction following	Standard LoRA or full fine-tune
DeepSeek open releases	Mixture-of-Experts	Coding, long-context tasks	Expert LoRA
Qwen open checkpoints	Dense transformer	Multilingual support, structured output	Standard LoRA
gpt-oss family	Dense transformer	General-purpose chat fine-tunes	Standard LoRA or full fine-tune

Step 3: Prepare Your Training Dataset in JSONL Format

Together AI’s fine-tuning API expects conversational data formatted as JSONL, with each line representing one training example as a list of role-tagged messages. This mirrors the chat completion format most developers already use when calling LLM APIs, which keeps the learning curve low if you have built anything with a chat-based SDK before.

{"messages": [{"role": "system", "content": "You are a support agent for a SaaS billing platform."}, {"role": "user", "content": "Why was I charged twice this month?"}, {"role": "assistant", "content": "I can see two charges on your account this billing cycle. Let me check if one was a duplicate authorization that will drop off, or if you were moved to a new plan mid-cycle. Can you confirm the last four digits of the card on file?"}]}
{"messages": [{"role": "system", "content": "You are a support agent for a SaaS billing platform."}, {"role": "user", "content": "How do I downgrade my plan?"}, {"role": "assistant", "content": "You can downgrade from Settings > Billing > Change Plan. The new rate applies at the start of your next billing cycle, and you'll keep current-tier features until then."}]}

Save this as train.jsonl. Split off roughly 10-15% of your examples into a separate val.jsonl file for validation, since the platform’s automatic early stopping feature depends on having a validation set to measure against during training.

Step 4: Upload and Validate Your Dataset With the New Preview Tools

The September update added dataset preview and validation directly in the UI, which catches malformed JSON lines, missing role fields, and token-length outliers before you commit GPU spend to a job. Upload through the SDK, then check the file object’s status before proceeding.

train_file = client.files.upload(file="train.jsonl", purpose="fine-tune")
val_file = client.files.upload(file="val.jsonl", purpose="fine-tune")

print("Train file ID:", train_file.id)
print("Validation file ID:", val_file.id)

# Confirm the platform accepted and parsed the file correctly
file_status = client.files.retrieve(train_file.id)
print("Processing status:", file_status.processed)

If you are working in the dashboard instead of the SDK, the same upload triggers an automatic preview pane showing a sample of parsed examples plus a warning banner for any row that fails schema validation. Fix flagged rows before launching the job. A dataset with even a handful of malformed lines can silently truncate training or cause a job to fail partway through, wasting the tokens already processed.

Step 5: Configure LoRA Settings for a Dense Model

For a standard dense model, LoRA (Low-Rank Adaptation) trains a small set of additional weight matrices instead of updating the full model, which cuts both compute cost and the risk of overfitting on a small dataset. The original LoRA method, described in the paper “LoRA: Low-Rank Adaptation of Large Language Models”, remains the foundation for how Together AI and most other fine-tuning platforms implement parameter-efficient training in 2026. If you want a deeper look at the underlying method outside a managed platform, see our guide on fine-tuning LLMs with LoRA.

job = client.fine_tuning.create(
    training_file=train_file.id,
    validation_file=val_file.id,
    model="meta-llama/Llama-3.3-70B-Instruct-Reference",
    lora=True,
    lora_r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    n_epochs=3,
    learning_rate=1e-4,
    batch_size=8,
    early_stopping=True,
    suffix="support-agent-v1"
)

print("Job ID:", job.id)
print("Status:", job.status)

The batch_size=8 value reflects the newly unlocked arbitrary batch size support from the September update. Previously, developers had to pick from a limited set of preset values; now you can tune batch size to match your dataset size and how much training variance you’re willing to tolerate. Smaller batches generally give noisier but sometimes more precise gradient updates on small datasets, while larger batches train faster but need more examples to avoid overfitting to a handful of dominant patterns.

Step 6: Configure Expert LoRA for Mixture-of-Experts Models

If your base model is a Mixture-of-Experts architecture like GLM-5.3, standard LoRA config does not map cleanly onto the model’s structure, because MoE models route each token through a subset of specialized expert sub-networks rather than processing every token through the same dense layers. Expert LoRA solves this by applying low-rank adapters only to the experts that receive meaningful traffic for your task, leaving the router and unused experts frozen.

job = client.fine_tuning.create(
    training_file=train_file.id,
    validation_file=val_file.id,
    model="zai-org/GLM-5.3",
    lora=True,
    lora_type="expert",
    lora_r=8,
    lora_alpha=16,
    n_epochs=3,
    learning_rate=8e-5,
    batch_size=4,
    early_stopping=True,
    suffix="glm-support-agent-v1"
)

print("Job ID:", job.id)

Because Expert LoRA touches fewer parameters than a full-model LoRA pass on the same base model, jobs typically run with a lower lora_r value and a smaller batch size than you would use on a dense model of comparable size. Together AI’s fine-tuning pricing update cut costs by up to 70% on select models specifically to make this kind of MoE-targeted training affordable for smaller teams that could not previously justify the compute cost of adapting a large sparse model. Other open-weight MoE families, such as the models covered in our DeepSeek V4.1 Flash API tutorial, follow a similar Expert LoRA configuration pattern.

Step 7: Monitor Training With the Live Experiment Dashboard

Open the Together AI dashboard and navigate to Fine-Tuning > Jobs, then click into your running job. The new live dashboard streams loss curves and evaluation metrics as training progresses, instead of requiring you to poll the API or wait for a completion webhook. If you prefer to check status programmatically, the SDK exposes the same metrics.

import time

while True:
    status = client.fine_tuning.retrieve(job.id)
    print(f"Status: {status.status} | Step: {status.trained_tokens}")
    if status.status in ("completed", "failed", "cancelled"):
        break
    time.sleep(30)

print("Final status:", status.status)

Watch the validation loss curve specifically. If it flattens or starts climbing while training loss keeps dropping, the model is beginning to overfit to your training examples. With automatic early stopping enabled, the platform halts the job at that point instead of continuing to burn tokens on a model that is memorizing rather than generalizing.

Step 8: Evaluate the Fine-Tuned Model Before Deploying

Once the job completes, run your held-out validation examples through the new model and compare outputs to the base model’s responses on the same prompts. Do not skip this step even if the loss curve looked clean; loss values do not always correlate with the qualitative behavior you actually care about, like tone, factual accuracy on domain-specific questions, or refusal patterns.

test_prompts = [
    "My invoice shows a charge I don't recognize.",
    "Can I get a refund for an annual plan I just started?",
    "How do I add a second payment method?"
]

for prompt in test_prompts:
    response = client.chat.completions.create(
        model=job.output_name,
        messages=[
            {"role": "system", "content": "You are a support agent for a SaaS billing platform."},
            {"role": "user", "content": prompt}
        ]
    )
    print(f"Q: {prompt}\nA: {response.choices[0].message.content}\n")

If responses drift off-tone, repeat evaluation, or hallucinate policy details not present in your training data, go back to Step 3 and expand your dataset rather than immediately increasing epoch count. More epochs on a thin dataset tends to deepen overfitting rather than fix coverage gaps.

Step 9: Deploy to a Dedicated Inference Endpoint

Together AI’s documentation is explicit that the platform covers the full lifecycle from training through hosting and inference on a dedicated endpoint, so deployment does not require exporting weights or standing up your own serving infrastructure. Create the deployment directly from the completed job.

deployment = client.endpoints.create(
    model=job.output_name,
    display_name="support-agent-v1-endpoint",
    hardware="1x_a100_80gb",
    min_replicas=1,
    max_replicas=2,
    auto_scale=True
)

print("Endpoint ID:", deployment.id)
print("Endpoint URL:", deployment.endpoint_url)

Set min_replicas to 0 during development if you want the endpoint to scale to zero when idle, which avoids paying for compute between test runs. For production traffic, keep at least one replica warm to avoid cold-start latency on the first request after an idle period.

Step 10: Call the Deployed Model From Your Application

Once the endpoint reports a ready status, it behaves like any other chat completion endpoint on the platform. Point your application’s API calls at the fine-tuned model’s identifier instead of the base model.

curl https://api.together.xyz/v1/chat/completions \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-username/support-agent-v1",
    "messages": [
      {"role": "system", "content": "You are a support agent for a SaaS billing platform."},
      {"role": "user", "content": "I was charged after cancelling my subscription."}
    ],
    "max_tokens": 300,
    "temperature": 0.3
  }'

A typical response looks like this, with the fine-tuned model applying the tone and structure it learned from your training examples rather than the generic base model behavior:

{
  "id": "chatcmpl-9f2a...",
  "model": "your-username/support-agent-v1",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "I'm sorry about that. A charge after cancellation usually means the cancellation processed after your renewal date had already been billed. Let me pull up your account and confirm when the cancellation was submitted so we can issue a prorated refund if it applies."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {"prompt_tokens": 42, "completion_tokens": 58, "total_tokens": 100}
}

Step 11: Set Up Automatic Early Stopping for Future Jobs

For iterative work where you retrain on updated datasets, automatic early stopping saves the most money on repeat runs, since you no longer need to guess an epoch count that avoids overfitting. Configure a patience window and a minimum improvement threshold so the platform knows how much validation-loss noise to tolerate before halting.

job = client.fine_tuning.create(
    training_file=train_file.id,
    validation_file=val_file.id,
    model="meta-llama/Llama-3.3-70B-Instruct-Reference",
    lora=True,
    lora_r=16,
    n_epochs=10,
    early_stopping=True,
    early_stopping_patience=2,
    early_stopping_threshold=0.001,
    suffix="support-agent-v2"
)

Setting n_epochs=10 alongside early stopping is intentional: you set a generous ceiling and let the platform cut the job off once validation loss stops improving by at least the threshold amount for two consecutive checks, rather than manually estimating the right epoch count up front.

Step 12: Compare Costs Across Fine-Tuning Approaches

Pricing depends on base model size, LoRA versus full fine-tune, and now the newly discounted tier for select models. Together AI publishes current rates on its pricing page, and costs are billed per million tokens processed during training rather than a flat per-job fee.

Approach	Typical Use Case	Relative Compute Cost	Notes
Standard LoRA (dense model)	Chat tone, domain adaptation	Low	Fastest to iterate; good default for most teams
Expert LoRA (MoE model)	Reasoning/coding fine-tunes on GLM-5.3, DeepSeek MoE	Low to moderate	New in Sept 2026 update; cuts cost on sparse models up to 70% on select models
Full fine-tune (dense model)	Deep behavioral change, large datasets	High	Best when LoRA underfits your task
Continued pretraining	New domain vocabulary, large unlabeled corpora	Highest	Rarely needed for typical application fine-tunes

For most application-specific fine-tunes, such as a support agent tone, a coding style, or a structured-output format, standard or Expert LoRA is sufficient and dramatically cheaper than a full fine-tune. Reserve full fine-tuning for cases where LoRA experiments plateau below your target quality bar even after dataset and hyperparameter iteration.

Securing API Keys and Training Data

Fine-tuning jobs often involve uploading proprietary data, whether that is internal support transcripts, product documentation, or customer conversations, so the security of your pipeline matters as much as the training configuration. Never hardcode a Together AI API key directly into a script that gets committed to a shared repository. Use environment variables locally, and a secrets manager (such as a CI/CD platform’s encrypted variables, or a dedicated tool like AWS Secrets Manager or HashiCorp Vault) for any automated pipeline that kicks off fine-tuning jobs on a schedule.

Scope your API keys narrowly where the platform allows it. If your workflow only needs to upload files and create fine-tuning jobs, avoid using a key that also has billing or account management permissions. This limits the blast radius if a key leaks through a misconfigured log, a compromised CI runner, or an accidental commit.

Before uploading a training dataset, scrub it for anything you would not want persisted on a third-party platform: customer PII beyond what the task genuinely requires, internal credentials accidentally pasted into a support transcript, or proprietary pricing data that goes beyond what the model needs to learn your support tone. A quick regex pass for email addresses, phone numbers, and card-like digit sequences catches the most common accidental leaks before they ever leave your machine.

import re
import json

def scrub_line(example):
    text = json.dumps(example)
    # Basic patterns; extend for your specific data shape
    text = re.sub(r'[\w.+-]+@[\w-]+\.[\w.-]+', '[EMAIL]', text)
    text = re.sub(r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b', '[PHONE]', text)
    text = re.sub(r'\b(?:\d[ -]*?){13,16}\b', '[CARD]', text)
    return json.loads(text)

with open("train.jsonl") as f_in, open("train_scrubbed.jsonl", "w") as f_out:
    for line in f_in:
        cleaned = scrub_line(json.loads(line))
        f_out.write(json.dumps(cleaned) + "\n")

Run this scrub step before Step 3’s upload, not after. Once a dataset has been used to train a model, removing sensitive content from the raw file does not remove whatever the model may have memorized from it during training.

Understanding Training Costs and Token Budgets

Fine-tuning costs on Together AI scale with the number of tokens processed during training, which is a function of dataset size, epoch count, and base model size, not a flat per-job fee. Before launching a production job, estimate your token budget so a misconfigured epoch count does not turn into a surprise bill.

A rough back-of-envelope calculation: multiply the total token count of your training file by the number of epochs you plan to run. A dataset with 500 examples averaging 200 tokens each totals 100,000 tokens per epoch; run that for three epochs and you have processed 300,000 tokens, before accounting for validation passes. Compare that estimate against the current per-token training rate on the pricing page before committing to a large n_epochs value, especially on larger base models where the per-token rate is higher.

import json

def estimate_tokens(file_path, avg_chars_per_token=4):
    total_chars = 0
    with open(file_path) as f:
        for line in f:
            example = json.loads(line)
            for msg in example["messages"]:
                total_chars += len(msg["content"])
    return total_chars // avg_chars_per_token

train_tokens = estimate_tokens("train.jsonl")
epochs = 3
print(f"Estimated tokens per epoch: {train_tokens:,}")
print(f"Estimated total training tokens ({epochs} epochs): {train_tokens * epochs:,}")

This estimate is approximate, since actual tokenization varies by model and the four-characters-per-token heuristic is a rough average for English text. Treat it as a sanity check to catch order-of-magnitude mistakes, such as accidentally setting n_epochs=100 instead of 10, rather than as a precise billing forecast. Automatic early stopping, covered in Step 11, is the more reliable guardrail against runaway training costs, since it halts a job based on actual validation performance rather than a fixed epoch count you set in advance. Once your fine-tuned endpoint is live, pair it with our LLM prompt caching guide to cut repeat-inference costs on top of the training savings.

Common Pitfalls When Fine-Tuning on Together AI

These mistakes account for most of the wasted GPU-hours and failed jobs reported by developers new to hosted fine-tuning platforms.

• Skipping the validation split. Without a validation file, automatic early stopping has nothing to measure against, and you lose the clearest signal for when a job should stop.
• Using standard LoRA config on a MoE model. Applying dense-model LoRA settings to zai-org/GLM-5.3 or similar sparse architectures wastes compute on experts your task never activates. Use lora_type="expert" instead.
• Training with too few examples. Fewer than a few dozen examples rarely produces a stable fine-tune; the model either underfits or memorizes the handful of examples verbatim.
• Ignoring dataset preview warnings. The new validation tools flag malformed rows for a reason. A job that starts on a partially broken dataset can complete “successfully” while having trained on far fewer usable examples than you intended.
• Setting learning rate too high on a small dataset. A learning rate tuned for a large corpus will overshoot on a few hundred examples, producing a model that forgets general capabilities faster than it learns your task.
• Forgetting to set a batch size appropriate to dataset size. A batch size larger than roughly 10% of your dataset size gives too few gradient updates per epoch to learn effectively.
• Leaving deployment replicas at a fixed minimum during testing. Testing against an always-on endpoint with min_replicas set above zero accrues idle compute charges between test runs.
• Never comparing against the base model. Skipping Step 8’s side-by-side evaluation means you cannot tell whether the fine-tune actually improved anything versus prompting the base model well.

Advanced Tips for Production Fine-Tuning Workflows

Once the basic workflow above is working, a few advanced practices separate a one-off experiment from a repeatable production pipeline.

Version your training datasets alongside your fine-tuning jobs. Tag each dataset file with a version identifier in its filename or metadata, and log which dataset version produced which model suffix. When a fine-tune underperforms in production, this makes it possible to trace the regression back to a specific dataset change rather than guessing.

Run a small grid of lora_r values (commonly 8, 16, and 32) on the same dataset before committing to a production config. Higher rank values give the adapter more capacity to learn complex patterns but increase both training cost and overfitting risk on small datasets. The live dashboard added in the September update makes this kind of comparison much faster, since you can watch multiple jobs’ loss curves in parallel instead of waiting for each to finish before checking results.

For teams fine-tuning MoE models regularly, keep a shared log of which experts activate for which task types. Because Expert LoRA only adapts experts that see meaningful traffic during training, understanding your task’s expert activation pattern helps you predict training cost and catch cases where a dataset is too narrow to exercise the experts you actually want to improve.

Finally, treat the dedicated endpoint’s max_replicas setting as a cost control, not just a performance one. Auto-scaling protects against traffic spikes, but an unbounded max replica count on a fine-tuned model serving low-value internal traffic can quietly become one of the largest line items on a monthly bill.

Troubleshooting Common Fine-Tuning Errors

The following issues cover the errors developers most frequently hit when running fine-tuning jobs on a hosted platform in 2026.

• Job fails immediately with a file validation error. Re-check your JSONL for trailing commas, missing closing braces, or lines that are not valid standalone JSON objects. Use the dataset preview tool before resubmitting.
• Training loss drops to near zero within the first epoch. This usually signals an extremely small or repetitive dataset causing rapid memorization. Expand the dataset or lower the learning rate.
• Validation loss never improves. Check whether your validation examples come from the same distribution as your training examples. A validation set that tests a different task than the training data will always look like the model is failing to learn.
• Endpoint deployment stays in a pending state. This typically reflects hardware provisioning delays during high-demand periods. Wait a few minutes and check the dashboard status page before assuming a failure.
• Fine-tuned model responses ignore the system prompt. If your training examples used inconsistent or absent system prompts, the model may learn to disregard that role entirely. Standardize the system prompt across all training examples.
• Job runs far longer than expected. Confirm your batch size is not set too low for your dataset size, since very small batches increase the number of gradient steps per epoch and slow overall training time.
• API returns a model-not-found error after training completes. Fine-tuned model identifiers typically follow a username/suffix pattern; confirm you are using job.output_name exactly as returned by the completed job object, not a guessed identifier.
• Expert LoRA job costs more than expected on a MoE model. If your dataset activates a wide spread of experts rather than a concentrated subset, Expert LoRA loses some of its cost advantage over standard fine-tuning. Consider narrowing the task scope or trimming the dataset to reduce expert spread.

Complete Working Project: A Support Agent Fine-Tune End to End

Putting every step together, here is the full script that takes a dataset from local files to a live, callable endpoint. Save this as run_finetune.py and run it after completing Steps 1-4 above.

import os
import time
import together

client = together.Together(api_key=os.environ["TOGETHER_API_KEY"])

# 1. Upload dataset
train_file = client.files.upload(file="train.jsonl", purpose="fine-tune")
val_file = client.files.upload(file="val.jsonl", purpose="fine-tune")

# 2. Launch fine-tuning job with LoRA and early stopping
job = client.fine_tuning.create(
    training_file=train_file.id,
    validation_file=val_file.id,
    model="meta-llama/Llama-3.3-70B-Instruct-Reference",
    lora=True,
    lora_r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    n_epochs=10,
    learning_rate=1e-4,
    batch_size=8,
    early_stopping=True,
    early_stopping_patience=2,
    suffix="support-agent-final"
)
print(f"Job launched: {job.id}")

# 3. Poll until complete
while True:
    status = client.fine_tuning.retrieve(job.id)
    print(f"Status: {status.status}")
    if status.status in ("completed", "failed", "cancelled"):
        break
    time.sleep(30)

if status.status != "completed":
    raise RuntimeError(f"Job ended with status: {status.status}")

# 4. Deploy to a dedicated endpoint
deployment = client.endpoints.create(
    model=status.output_name,
    display_name="support-agent-final-endpoint",
    hardware="1x_a100_80gb",
    min_replicas=1,
    max_replicas=2,
    auto_scale=True
)
print(f"Endpoint live at: {deployment.endpoint_url}")

# 5. Smoke-test the endpoint
response = client.chat.completions.create(
    model=status.output_name,
    messages=[
        {"role": "system", "content": "You are a support agent for a SaaS billing platform."},
        {"role": "user", "content": "Why was I charged twice this month?"}
    ]
)
print(response.choices[0].message.content)

Running this script end to end gives you a deployed, callable, fine-tuned model built entirely on Together AI’s hosted infrastructure, with no local GPU required at any stage.

Choosing Between LoRA, Expert LoRA, and Full Fine-Tuning

The decision between these three approaches comes down to model architecture, dataset size, and budget. Standard LoRA remains the right default for dense models and small-to-medium datasets, since it trains fast, costs less, and rarely overfits catastrophically even with limited data. Expert LoRA is the correct choice specifically for Mixture-of-Experts base models, since applying dense-style LoRA to a sparse architecture either wastes compute on inactive experts or fails to concentrate learning where the model actually routes your task’s tokens.

Full fine-tuning is worth the added cost only when LoRA has genuinely plateaued below your quality bar after dataset and hyperparameter iteration, or when the task requires deep behavioral change that a low-rank adapter cannot capture, such as teaching a model an entirely new output format or reasoning style rather than adjusting tone and domain knowledge. Most application-level customization, including the support agent example built throughout this guide, sits comfortably within LoRA or Expert LoRA’s capabilities. For a broader view of which open-weight base models are worth fine-tuning in the first place, see our roundup of the best open source LLMs.

How the September Update Compares to Together AI’s Previous Fine-Tuning Platform

Together AI’s changelog shows a pattern of incremental model-family additions over the prior year, including the general availability of gpt-oss fine-tuning and coverage expanding to more DeepSeek, Qwen, and Meta releases. The September 11, 2026 update is different in scope: it bundles infrastructure changes (the live dashboard, arbitrary batch sizes, automatic early stopping) alongside the model roster expansion, rather than shipping model support alone. That combination is what made a dedicated fine tune llm workflow meaningfully faster to run end to end, since developers no longer need external tooling to monitor a job or guess at stopping points.

The price cuts reaching up to 70% on select models also shift the calculus for smaller teams. Before this update, adapting a large Mixture-of-Experts model was often cost-prohibitive for anyone outside a well-funded engineering org. Expert LoRA’s narrower compute footprint, combined with the discounted pricing tier, brings MoE fine-tuning within reach of teams that previously stuck to smaller dense models purely on cost grounds.

Iterating on a Fine-Tune: When to Retrain vs. Adjust Prompts

Not every quality issue after deployment calls for a full retraining cycle. Before queuing another fine-tuning job, separate problems that a system prompt adjustment can fix from problems that genuinely require new training data. If the model’s factual knowledge is correct but its tone drifts inconsistently across requests, a tighter system prompt or a few-shot example appended at inference time often closes the gap faster and at zero additional training cost.

Retraining is the right move when the model consistently gets domain facts wrong, follows an output format inconsistently across a wide range of inputs, or fails on an entire category of request that was underrepresented in the original dataset. In that case, the fix is adding targeted examples covering the failure category, not tweaking the existing dataset’s balance. Together AI’s changelog-driven model updates mean it is also worth periodically checking whether a newer base model in your fine-tuned family has shipped, since starting a fresh fine-tune on an improved base model sometimes outperforms endless iteration on an older one.

Keep a lightweight regression suite of 20-30 representative prompts that you rerun against every new fine-tuned version before replacing a production endpoint. This catches cases where a dataset update fixed one failure mode while quietly introducing another, a pattern that is easy to miss if you only spot-check the specific examples you were trying to fix.

Frequently Asked Questions

Do I need my own GPU to fine-tune an LLM on Together AI?
No. Together AI provisions training hardware for you, and you are billed per token processed during training plus hosting costs for the deployed endpoint. This is the core advantage of a managed fine-tuning platform over running your own training infrastructure.

What is Expert LoRA and when should I use it?
Expert LoRA is a Mixture-of-Experts-specific low-rank adaptation technique introduced in Together AI’s September 2026 update. Use it when fine-tuning a sparse MoE model like zai-org/GLM-5.3, since it adapts only the experts your task activates rather than treating the whole model as dense.

How large does my training dataset need to be?
There is no universal minimum, but datasets in the low hundreds of examples typically produce stable LoRA fine-tunes for tone and domain adaptation tasks. Full fine-tuning generally benefits from larger datasets than LoRA approaches.

Can I fine-tune zai-org/GLM-5.3 on Together AI right now?
Yes. Together AI’s changelog confirms GLM-5.3 fine-tuning support as part of the recent platform updates, and it is one of the models best suited to the new Expert LoRA method given its MoE architecture.

What does automatic early stopping actually save me?
It stops a training job once validation metrics stop improving by your configured threshold, which prevents paying for additional epochs that would otherwise push the model toward overfitting without any accuracy benefit.

How much cheaper is fine-tuning after the September 2026 price cuts?
Together AI’s update cut fine-tuning prices by up to 70% on select models, though the exact discount varies by model and should be confirmed against the current pricing page before budgeting a large training run.

Should I use LoRA or a full fine-tune for a customer support agent?
Standard or Expert LoRA is sufficient for most support-agent tone and domain adaptation tasks. Reserve full fine-tuning for cases where LoRA experiments plateau below your target quality after dataset and hyperparameter iteration.

Can I scale my deployed endpoint down to zero when not in use?
Yes. Setting min_replicas to 0 lets the endpoint scale down during idle periods, which is useful during development and testing to avoid paying for compute between test runs. Keep at least one replica warm for production traffic to avoid cold-start latency.

Related Coverage

• LLM Prompt Caching: Cut API Costs 90% in 12 Steps [2026]
• How to Use GPT Image 2.5 API: 13 Steps, 90 Min [2026]
• OpenAI Realtime API: gpt-live-1 Setup in 12 Steps [2026]
• AI Image Editing Workflow Setup: 12 Steps, 2 New Models [2026]
• Qwen Image 3.0 API Setup: 12 Steps, $0.03/Image [2026]