The dataset viewer is not available for this split.
Error code: StreamingRowsError
Exception: TypeError
Message: Couldn't cast array of type
struct<target_mass_at_prediction: double, uniform_accuracy_baseline: double, fixed_answer_accuracy_baseline: double, baseline_adjusted_skill: double, baseline_adjusted_score: double>
to
{'binary_brier': Value('float64'), 'accuracy': Value('float64'), 'positive_recall': Value('float64'), 'specificity': Value('float64'), 'precision': Value('float64'), 'balanced_accuracy': Value('float64'), 'f1': Value('float64'), 'true_positive': Value('float64'), 'false_positive': Value('float64'), 'false_negative': Value('float64'), 'true_negative': Value('float64'), 'positive_prevalence': Value('float64'), 'majority_accuracy_baseline': Value('float64'), 'constant_brier_baseline': Value('float64'), 'brier_skill': Value('float64'), 'baseline_adjusted_skill': Value('float64'), 'baseline_adjusted_score': Value('float64')}
Traceback: Traceback (most recent call last):
File "/src/services/worker/src/worker/utils.py", line 147, in get_rows_or_raise
return get_rows(
dataset=dataset,
...<4 lines>...
column_names=column_names,
)
File "/src/libs/libcommon/src/libcommon/utils.py", line 272, in decorator
return func(*args, **kwargs)
File "/src/services/worker/src/worker/utils.py", line 127, in get_rows
rows_plus_one = list(itertools.islice(safe_iter(ds, dataset=dataset), rows_max_number + 1))
File "/src/services/worker/src/worker/utils.py", line 483, in safe_iter
yield from ds.decode(False) if ds.features else ds
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2951, in __iter__
for key, example in ex_iterable:
^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2461, in __iter__
for key, pa_table in self._iter_arrow():
~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2486, in _iter_arrow
for key, pa_table in self.ex_iterable._iter_arrow():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 547, in _iter_arrow
for key, pa_table in iterator:
^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 430, in _iter_arrow
for key, pa_table in self.generate_tables_fn(**gen_kwags):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 343, in _generate_tables
self._cast_table(pa_table, json_field_paths=json_field_paths),
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 132, in _cast_table
pa_table = table_cast(pa_table, self.info.features.arrow_schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2378, in table_cast
return cast_table_to_schema(table, schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2312, in cast_table_to_schema
cast_array_to_feature(
~~~~~~~~~~~~~~~~~~~~~^
table[name] if name in table_column_names else pa.array([None] * len(table), type=schema.field(name).type),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
feature,
^^^^^^^^
)
^
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 1861, in wrapper
return pa.chunked_array([func(chunk, *args, **kwargs) for chunk in array.chunks])
~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2158, in cast_array_to_feature
raise TypeError(f"Couldn't cast array of type\n{_short_str(array.type)}\nto\n{_short_str(feature)}")
TypeError: Couldn't cast array of type
struct<target_mass_at_prediction: double, uniform_accuracy_baseline: double, fixed_answer_accuracy_baseline: double, baseline_adjusted_skill: double, baseline_adjusted_score: double>
to
{'binary_brier': Value('float64'), 'accuracy': Value('float64'), 'positive_recall': Value('float64'), 'specificity': Value('float64'), 'precision': Value('float64'), 'balanced_accuracy': Value('float64'), 'f1': Value('float64'), 'true_positive': Value('float64'), 'false_positive': Value('float64'), 'false_negative': Value('float64'), 'true_negative': Value('float64'), 'positive_prevalence': Value('float64'), 'majority_accuracy_baseline': Value('float64'), 'constant_brier_baseline': Value('float64'), 'brier_skill': Value('float64'), 'baseline_adjusted_skill': Value('float64'), 'baseline_adjusted_score': Value('float64')}Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
S1MB evaluation results
Public English and Japanese model evaluation results for System One Mosaic
Benchmark (S1MB), combining Choice, Noul and Score tasks. Both english-v1
and japanese-v1 measurements belong in this Dataset. The same leaderboard
selects English
or Japanese
through the category query parameter. Evaluate and rank each scope independently.
Japanese evaluation inputs and targets remain private; publishing measurements
does not redistribute the evaluation Dataset.
Repository layout and submissions
Each <organization-or-user>__<model-id>/ folder represents one canonical model
and may contain benchmarks from either language or both. It contains metadata.json plus per-benchmark <benchmark-id>.json.xz files.
Measurements retain original run IDs, exact model and evaluation Dataset revisions,
evaluator provenance and effective settings. Different recorded Dataset revisions
may coexist; validate each result against its own recorded revision. Historical
Dataset repo IDs and SHAs remain unchanged after repository renames.
Follow the submission guide to evaluate, validate, export and open a Dataset PR. Include evaluation commands, coverage, failures, model/source pins and material runtime settings. Explain replacements and configuration changes; do not assemble an undisclosed best-of-many row. Fix settings independently of test labels and scores. Missing and partial measurements remain incomplete and cannot claim complete aggregate scores.
Adapter/code changes belong in a separate source PR. Upload only the intended model metadata and compressed measurements; exclude credentials, raw input text, checkpoints, caches, reports and local lock files. Authored training disclosures are provenance, never inference instructions or scoring adjustments; missing metadata means unknown. S1MB scores do not establish unseen-task generalization or training-data non-overlap.
Leaderboard display updates
The offline publisher prepares English and Japanese scopes from one exact
results commit. After merging a results Dataset PR, run from evaluator/:
uv run python ../viewer/scripts/publish-display.py
# Review the report, then publish the summary:
uv run python ../viewer/scripts/publish-display.py --publish
The reserved root file viewer-summary.json contains prepared display data, not a
measurement submission. Maintainers generate it offline from validated public
results, compare with the published summary, review reductions, and publish with
a parent-SHA guard. Rebuild/redeploy the Space to load it; measurement publication
alone does not update the running leaderboard. The viewer reads bundled JSON
without runtime Hub access. Local viewers also require explicit preparation or
fetching and a restart. See the
deployment runbook.
License and usage terms
This collection has source-dependent rights and usage terms, represented by
license: other; it has no blanket MIT grant. The S1MB project's MIT license
covers source code only. Submitted measurements retain their contributor's
applicable terms, and source-derived material retains the terms and attribution
requirements of each recorded evaluation Dataset revision. Consult the
English source notices
and Japanese source notices
for the applicable recorded release; access to Japanese inputs requires authorization.
Model checkpoint and inference-service terms remain separate. Where permission
is unspecified, obtain it from the relevant rights holder before redistribution;
an unknown license is not a reuse grant.
See the S1MB project for code and documentation.
Japanese results
Publish Japanese measurements evaluated against hotchpotch/S1MB-japanese-v1.
Historical measurements referencing the private hotchpotch/JDMB-v1 Dataset
have been removed from this shared results Dataset; those models require fresh
S1MB evaluations. JDMB-v1 and the original result archives remain private.
The S1MB evaluations for PelaAI/KnowLine-4B-Gen4 and convaiinnovations/laya-multilingual are retained: 54 measurements across two model folders, with 27 benchmarks / 8,067 judgments per model. Recorded Dataset IDs, exact SHAs, model revisions and evaluator provenance remain unchanged. Existing English measurements and canonical model metadata are preserved.
Submit new English and Japanese measurements here. Do not upload private raw inputs, targets, local lock files or reports. Prepare and publish the display summary, then rebuild/redeploy the Space to show updated results.
- Downloads last month
- 2,666