关掉推理反而更贵,省钱的旋钮换了地方

博客分析了在按输出token计费的AI模型(如GPT-6.1 Terra)中,盲目关闭推理反而导致单任务成本上升、错误率升高和输出冗余的问题。通过实测六档推理强度数据,指出'不推理'档位虽标价低但输出token多、硬答率高,实际成本高于low档;强调应依据任务类型(分类/问答/规划)动态配置推理档位,并借助Auto Routing、配置化管理和细粒度账单拆分实现成本优化。

有个做内部工具的朋友上周把 API 调用记录甩给我:为了压成本,他把新模型的 reasoning 档位从 high 一路调到了"不推理",理由是"少想就少花钱"。结果一个月跑下来,单任务均价比调之前还高了一截,更麻烦的是工单里"答非所问"的反馈多了起来。他对着账单反复确认,怀疑是不是计量出了 bug。
不是计量的问题。是省钱的那个旋钮,位置变了。

一个模型,六档价

OpenAI 上周发布的 GPT-6.1 Terra,标价是输入每百万 token 1 美元、输出 5 美元、命中缓存的输入 0.05 美元——比上一代 Terra 直接减半,只有旗舰 Astra 的十分之一,官方称任务完成速度还比 Astra 和 Sol 快三倍以上。看起来是个标准的降价故事。

但更值得看的是另一组数字。独立评测机构 Artificial Analysis 在 10 月 26 日把 Terra 的六个推理档位各跑了一遍,得到的是这样一条曲线:
不推理:智能指数 33.5,单任务 $0.107,输出 4,737 token,硬答率 72%
low:36.2,$0.055,2,681 token,46%
medium:41.3,$0.095,5,835 token,44%
high:43.9,$0.134,9,027 token,43%
xhigh:45.4,$0.194,14,551 token,42%
max:48.1,$0.403,35,096 token,38%
同一份权重、同一份价目表,最贵档和最便宜档之间差了七倍多。max 比 low 多花 7.4 倍的钱,换回 11.9 个指数点;而 max 比 xhigh 只多 2.7 分,钱却要多一倍。这条曲线的最后两级,明显越往上越不划算。

"少想"为什么反而更花钱

最反直觉的是第一行:不推理档 $0.107,比开 low 推理的 $0.055 贵了将近一倍,分数反而低 2.7 分。
原因藏在输出 token 那一列。不推理档每个任务平均写 4,737 个 token,low 档只写 2,681 个。模型不"打草稿",就把想到的第一句直接往外吐,来回绕、反复改口,写得更长,还没写对。硬答率的数据也是同一个方向:遇到不会的问题时,不推理档硬答比例 72%,low 档 46%,max 档降到 38%。
OpenAI 自己的内部测试指向同一个结论——用一组容易诱发错误的提示词测,低推理强度下错误率从上一代的 53.6% 降到 16.7%,拉满推理降到 4.3%。

所以"少想就等于少花钱"这个直觉,在按输出 token 计费的模型上是反的。你省下的思考 token,会以几倍的篇幅回到输出那一栏,还附赠一批需要人工返工的错答案。这和过去几个月大家慢慢摸出来的规律是同一条:账单真正敏感的不是标价,而是模型为一个任务写了多少字。AA 在 10 月 24 日那轮测算里已经显示过,两个标价一模一样的模型,单任务成本能差 2.75 倍,差别全在输出量上。

档位不该只设一个值

既然档位本身就是一张价格表,它就不该是一个全局配置。
这两天有个可以抄的做法:Requesty 上线了 Auto Routing 的公测,允许团队用自然语言写规则,把 Claude Code 的会话按首条消息自动分给不同模型,一次判定、整段会话锁定。他们自己给出的数据是模型成本降 42%、延迟降 30%,对比基准是全程用 Opus 5.5。

这套思路搬到档位上完全成立。我现在的分法大概是这样,供参考:
分类、抽取、格式转换、打标签这类有标准答案的活,走 low 或不推理,答错了也容易在下游校验里拦下来;
常规问答、摘要、中等长度生成走 medium,这是覆盖面最大的一档;
多步规划、要读图的电脑操作、复杂代码改动才上 high 及以上;
卡住了再升级,而不是一上来就把档位拉满。

落地就三件事。第一,先量自己的:拿 50 到 100 条真实任务,每个档位都跑一遍,记下"分数 / 单任务成本 / 输出 token"三个数,画出来的曲线很可能和上面那张不太一样。

第二,把档位和模型名一起挪进配置,别写死在业务代码里——我自己是把多家来源的调用收在 EasyAPI 这样的聚合入口后面,换档位就是改一行配置的事;当然这不是非它不可,自己包一层薄代理也一样。

第三,账单得能按模型和档位拆开看,EasyAPI 这类入口的用处也就在这里,省的是找数据的时间,不然你根本不知道那 42% 是从哪一档省出来的。

写在最后

同一天 Anthropic 交了 S-1。里面有个数字挺有意思:上半年算力与基础设施支出近 90 亿美元,占营收比例从去年同期的 201% 降到了 55%。公司把其中一部分归因于训练和推理算法的持续改进——同样的算力,能干更多活。

供给端确实在变便宜。但这不保证你的账单会跟着降,中间还隔着一层你自己设的档位。
以前的成本功课是"选哪个模型",现在是"同一个模型,这一路请求该开哪一档"。旋钮变小了,也变多了。

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值