saihai v1 training dataset

A Japanese and English training collection for saihai, an LLM-based Typed Decision Model. Each example pairs a decision input with reference probabilities over its ordered options.

118 subset configurations · 33,532 training cases · 8,729 validation cases. Training contains 17,509 Japanese and 16,023 English cases. Each subset has training data and up to 100 validation cases, sampled as complete groups.

For strengthening Japanese typed decisions, this collection includes training examples from source families evaluated by S1MB Japanese v1, imported from JDMB. Use the train split for training and the validation split for development. See S1MB Japanese source-family training data for the corresponding sources, and S1MB submissions and Known domains for required training disclosures.

Quick start

Choose a subset configuration explicitly. Some configurations provide only a train split.

import json
from datasets import load_dataset

data = load_dataset("hotchpotch/saihai-v1-dataset", "jp-long-context")
row = data["train"][0]
model_input = json.loads(row["input_json"])
reference_probabilities = row["probabilities"]

Each row represents one typed judgment. input_json encodes the state, decision type, instructions, and ordered options. probabilities aligns with that option order. Score options carry explicit numeric values; retain these values and soft distributions rather than replacing them with class indices or hard labels.

record_json contains the full training record, including its target. provenance_json records source identity, revision, original partition, generation model, and license conditions. id, parent_id, and group_id identify records and related examples. source_order preserves ordering. Only the decoded model input belongs in inference prompts; targets and provenance remain outside them.

Configuration names replace / in subset identities with --; for example, load LocalLLaMA/typed-decisions as LocalLLaMA--typed-decisions.

Dataset contents

The collection combines authored tasks with conversions of existing datasets. The names distinguish their origins:

All configurations, split counts, source links, and licenses

Subset	Train	Validation	License	Source
LocalLLaMA/typed-decisions	1,780	100	Apache-2.0	huggingface.co/datasets/LocalLLaMA/typed-decisions latentnode.pages.dev/articles/typed-decisions
PolyAI-LDN/BANKING77	2,824	100	CC-BY-4.0	arxiv.org/abs/2003.04807 github.com/PolyAI-LDN/task-specific-datasets
caiovicentino1/eikos-decisions	4,666	100	CC-BY-4.0; preserve upstream MIT or CC-BY-4.0	github.com/NExTplusplus/TAT-QA github.com/caiovicentino/eikos github.com/openai/grade-school-math huggingface.co/datasets/caiovicentino1/eikos-decisions
nvidia/HelpSteer2	2,730	100	CC-BY-4.0	arxiv.org/abs/2406.08673 huggingface.co/datasets/nvidia/HelpSteer2
bekko-aegis2	41	100	cc-by-4.0	huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0
bekko-aqua_rat	41	100	apache-2.0	huggingface.co/datasets/deepmind/aqua_rat
bekko-arc	40	100	cc-by-sa-4.0	huggingface.co/datasets/allenai/ai2_arc
bekko-arct	40	100	apache-2.0	huggingface.co/datasets/tasksource/arct
bekko-argument_quality	47	100	cc-by-3.0 (official IBM HF dataset metadata; card body conflicts)	huggingface.co/datasets/ibm-research/argument_quality_ranking_30k
bekko-babi_nli	41	100	bsd	huggingface.co/datasets/tasksource/babi_nli
bekko-bitext	37	0	cdla-sharing-1.0	huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset
bekko-blimp	41	0	cc-by-4.0	huggingface.co/datasets/nyu-mll/blimp
bekko-boardgameqa	42	100	cc-by-4.0	huggingface.co/datasets/tasksource/Boardgame-QA
bekko-boolq	41	100	cc-by-sa-3.0	huggingface.co/datasets/google/boolq
bekko-canttalk	40	0	cc-by-4.0	huggingface.co/datasets/nvidia/CantTalkAboutThis-Topic-Control-Dataset
bekko-circa	41	0	cc-by-4.0	huggingface.co/datasets/google-research-datasets/circa
bekko-civil_comments	36	98	cc0-1.0	huggingface.co/datasets/google/civil_comments
bekko-clinc	42	100	cc-by-3.0	huggingface.co/datasets/clinc/clinc_oos
bekko-commonsense_qa	41	100	mit	huggingface.co/datasets/tau/commonsense_qa
bekko-commonsense_qa2	41	100	cc-by-4.0	huggingface.co/datasets/tasksource/commonsense_qa_2.0
bekko-contract_nli	41	85	cc-by-4.0	github.com/stanfordnlp/contract-nli
bekko-copa	36	100	bsd-2-clause (COPA config)	asgordon.github.io/copa.html#license-bsd-2-clause-license huggingface.co/datasets/aps/super_glue
bekko-corr2cause	37	100	mit	huggingface.co/datasets/tasksource/corr2cause
bekko-cosmos_qa	42	100	cc-by-4.0	huggingface.co/datasets/Samsoup/cosmos_qa
bekko-creak	41	100	MIT (repository license; data grant not explicit)	github.com/yasumasaonoe/creak huggingface.co/datasets/amydeng2000/CREAK
bekko-dbpedia	41	0	cc-by-sa-3.0	huggingface.co/datasets/fancyzhx/dbpedia_14
bekko-defeasible_nli	41	100	apache-2.0, cc-by-sa-4.0	huggingface.co/datasets/tasksource/defeasible-nli
bekko-esci	42	0	apache-2.0	huggingface.co/datasets/tasksource/esci
bekko-ethics	41	100	mit	huggingface.co/datasets/hendrycks/ethics
bekko-feasibility_qa	40	0	mit	huggingface.co/datasets/tasksource/feasibilityQA
bekko-few_nerd	41	100	cc-by-sa-4.0	huggingface.co/datasets/DFKI-SLT/few-nerd
bekko-fig_qa	40	100	mit	huggingface.co/datasets/nightingal3/fig-qa
bekko-fol_nli	41	100	apache-2.0	huggingface.co/datasets/tasksource/FOL-nli
bekko-go_emotions	41	84	apache-2.0	huggingface.co/datasets/google-research-datasets/go_emotions
bekko-gretel_pii	41	100	apache-2.0	huggingface.co/datasets/gretelai/gretel-pii-masking-en-v1
bekko-hans	40	100	mit	huggingface.co/datasets/tasksource/hans
bekko-hh_rlhf	40	0	mit	huggingface.co/datasets/Anthropic/hh-rlhf
bekko-hwu64	42	100	cc-by-4.0	github.com/xliuhw/NLU-Evaluation-Data huggingface.co/datasets/FastFit/hwu_64
bekko-lexcomp	42	100	apache-2.0	huggingface.co/datasets/tasksource/lexcomp-nc-relation
bekko-logical_entailment	41	100	apache-2.0	huggingface.co/datasets/tasksource/logical-entailment
bekko-massive	42	100	cc-by-4.0	huggingface.co/datasets/AmazonScience/massive
bekko-measuring_hate	41	0	cc-by-4.0	huggingface.co/datasets/ucberkeley-dlab/measuring-hate-speech
bekko-minds14	39	0	cc-by-4.0	huggingface.co/datasets/PolyAI/minds14
bekko-monotonicity	41	0	apache-2.0	huggingface.co/datasets/tasksource/monotonicity-entailment
bekko-mtop	41	100	cc-by-sa-4.0	huggingface.co/datasets/tasksource/mtop
bekko-natural_logic	41	0	apache-2.0 (HF card declaration)	huggingface.co/datasets/tasksource/naturallogic
bekko-nlsat	41	100	apache-2.0	huggingface.co/datasets/tasksource/nlsat
bekko-nlupp	35	0	cc-by-4.0	github.com/PolyAI-LDN/task-specific-datasets
bekko-oasst1	41	100	apache-2.0	huggingface.co/datasets/OpenAssistant/oasst1
bekko-open_jev__amount-extraction-control-v1	32	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__browser-control-v1	43	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__citation-control-v1	43	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__context-retention-control-v1	42	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__drone-control-v1	43	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__email-selection-control-v1	28	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__entity-alignment-control-v1	43	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__ir-control-v1	40	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__mailroom-control-v1	17	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__painting-geometry-v1	44	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__phone-extraction-control-v1	41	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__reasoning-control-v1	42	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__silent-failure-control-v1	41	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__snake-v1	41	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__sponsor-segment-control-v1	42	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__tic_tac_toe-v1	40	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__tile_platformer-v1	41	0	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__trex_runner-v1	23	0	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__vizdoom-basic-v1	41	100	CC0-1.0 (generated numeric trajectories; no game assets)	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__workflow-controls-v1__agent_trace_observability	41	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__workflow-controls-v1__customer_service	41	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__workflow-controls-v1__invoice_processing	42	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-open_jev__workflow-controls-v1__security_incidents	41	100	CC0-1.0	huggingface.co/datasets/ZefanCai/Open-Jev
bekko-openbookqa	40	100	apache-2.0	huggingface.co/datasets/allenai/openbookqa
bekko-patent_similarity	39	100	cc-by-4.0	huggingface.co/datasets/tasksource/patent-phrase-similarity
bekko-paws	42	100	PAWS dataset: Google custom grant (“may be freely used for any purpose”); PAWS-Wiki source text: CC BY-SA 3.0 / GFDL source terms; PAWS-QQP excluded due to QQP terms	github.com/google-research-datasets/paws huggingface.co/datasets/google-research-datasets/paws
bekko-piqa	39	100	afl-3.0	huggingface.co/datasets/lighteval/piqa yonatanbisk.com/piqa
bekko-plane	37	0	cc-by-2.0	huggingface.co/datasets/tasksource/PLANE-ood
bekko-poem_sentiment	40	100	cc-by-4.0	huggingface.co/datasets/google-research-datasets/poem_sentiment
bekko-qasc	40	100	cc-by-4.0	huggingface.co/datasets/allenai/qasc
bekko-qasper	38	89	cc-by-4.0	huggingface.co/datasets/allenai/qasper
bekko-quartz	41	100	cc-by-4.0	huggingface.co/datasets/allenai/quartz
bekko-ropes	42	100	cc-by-4.0	huggingface.co/datasets/allenai/ropes
bekko-ruletaker	37	100	apache-2.0	huggingface.co/datasets/tasksource/ruletaker
bekko-scitail	42	100	apache-2.0	huggingface.co/datasets/allenai/scitail
bekko-scone	40	0	cc0-1.0	huggingface.co/datasets/tasksource/scone
bekko-sdoh_nli	39	100	cc-by-4.0	huggingface.co/datasets/tasksource/SDOH-NLI
bekko-sgd	41	100	cc-by-sa-4.0	github.com/google-research-datasets/dstc8-schema-guided-dialogue
bekko-sms_spam	40	0	cc-by-4.0	archive.ics.uci.edu/dataset/228/sms+spam+collection huggingface.co/datasets/ucirvine/sms_spam
bekko-snips	42	100	cc0-1.0	github.com/sonos/nlu-benchmark
bekko-snli	35	100	cc-by-sa-4.0	huggingface.co/datasets/stanfordnlp/snli
bekko-social_iqa	40	100	cc-by-4.0	huggingface.co/datasets/baber/social_i_qa
bekko-space_nli	41	0	mit	huggingface.co/datasets/tasksource/SpaceNLI
bekko-spartqa	41	100	mit	huggingface.co/datasets/tasksource/spartqa-mchoice
bekko-squad2	40	100	cc-by-sa-4.0	huggingface.co/datasets/rajpurkar/squad_v2
bekko-stepgame	40	100	mit	huggingface.co/datasets/tasksource/stepgame
bekko-strategyqa	41	0	mit	github.com/eladsegal/strategyqa huggingface.co/datasets/tasksource/strategy-qa
bekko-taskmaster	41	0	cc-by-4.0	github.com/google-research-datasets/Taskmaster
bekko-temporal_nli	39	0	apache-2.0	huggingface.co/datasets/tasksource/temporal-nli
bekko-tracie	41	0	apache-2.0	huggingface.co/datasets/tasksource/tracie
bekko-ud_ewt	43	100	cc-by-sa-4.0	huggingface.co/datasets/universal-dependencies/universal_dependencies universaldependencies.org/treebanks/en_ewt/index.html
bekko-winogrande	41	100	apache-2.0	github.com/allenai/winogrande huggingface.co/datasets/allenai/winogrande
bekko-wiqa	41	100	apache-2.0	huggingface.co/datasets/allenai/wiqa
jp-catalog-retained	6,762	100	CC BY-SA 4.0; underlying source terms where applicable	Yuichi Tateno; generated using LLM-jp 4.1 32B A3B Thinking
jp-catalog-revised	4,860	100	CC BY-SA 4.0; underlying source terms where applicable	Yuichi Tateno; generated using LLM-jp 4.1 32B A3B Thinking
jp-jdmb-commonsensemoralja-noul	400	40	MIT	github.com/Language-Media-Lab/commonsense-moral-ja
jp-jdmb-jamp-choice	358	0	CC-BY-SA-4.0	github.com/tomo-vv/temporalNLI_dataset
jp-jdmb-janli-choice	400	40	CC-BY-SA-4.0	github.com/verypluming/JaNLI
jp-jdmb-jcola-noul	358	0	CC-BY-SA-4.0	github.com/osekilab/JCoLA
jp-jdmb-jcommonsenseqa-choice	400	40	CC-BY-SA-4.0	github.com/yahoojapan/JGLUE
jp-jdmb-jnli-choice	382	40	CC-BY-SA-4.0	github.com/yahoojapan/JGLUE
jp-jdmb-jsick-choice	395	40	CC-BY-SA-4.0	github.com/verypluming/JSICK
jp-jdmb-jsts-score	396	40	CC-BY-SA-4.0	github.com/yahoojapan/JGLUE
jp-jdmb-kuci-choice	400	40	CC-BY-SA-4.0	github.com/ku-nlp/KUCI
jp-jdmb-mintaka-ja-boolean-noul	400	40	CC-BY-4.0	github.com/amazon-science/mintaka
jp-long-context	756	100	CC BY-SA 4.0; underlying source terms where applicable	Yuichi Tateno; generated using LLM-jp 4.1 32B A3B Thinking
jp-seed-guided	342	54	CC BY-SA 4.0; underlying source terms where applicable	Yuichi Tateno; generated using LLM-jp 4.1 32B A3B Thinking
jp-unknown	900	100	CC BY-SA 4.0; underlying source terms where applicable	Yuichi Tateno; generated using LLM-jp 4.1 32B A3B Thinking
programmatic-v0	103	99	CC BY-SA 4.0; underlying source terms where applicable	Yuichi Tateno; programmatic generation

S1MB Japanese source-family training data

JDMB is the former name of the Japanese benchmark, now maintained as S1MB japanese-v1. For convenience and compatibility with existing dataset consumers, this training dataset retains legacy identifiers such as the jdmb_source_family column and jp-jdmb-* subset IDs. These names identify training-source lineage; they do not mean that benchmark evaluation records are included in training.

The dataset includes 3,889 training cases from ten original dataset families represented in S1MB Japanese v1. These cases come from original training partitions, not the S1MB Japanese evaluation subset. This collection is not a source-family holdout; disclose exposure to the corresponding S1MB benchmarks when submitting results. The original jp-jdmb-* configuration names and jdmb_source_family provenance flag are preserved. That flag identifies these dedicated subsets, not every possible source-family relationship in other sources or generated material.

S1MB Japanese source-family training subsets

Training configuration	S1MB benchmark	Train	Original dataset	Recorded license
jp-jdmb-commonsensemoralja-noul	commonsensemoralja-noul	400	Language-Media-Lab/commonsense-moral-ja	MIT
jp-jdmb-jamp-choice	jamp-choice	358	tomo-vv/temporalNLI_dataset	CC-BY-SA-4.0
jp-jdmb-janli-choice	janli-choice	400	verypluming/JaNLI	CC-BY-SA-4.0
jp-jdmb-jcola-noul	jcola-noul	358	osekilab/JCoLA	CC-BY-SA-4.0
jp-jdmb-jcommonsenseqa-choice	jcommonsenseqa-choice	400	yahoojapan/JGLUE	CC-BY-SA-4.0
jp-jdmb-jnli-choice	jnli-choice	382	yahoojapan/JGLUE	CC-BY-SA-4.0
jp-jdmb-jsick-choice	jsick-sts-score	395	verypluming/JSICK	CC-BY-SA-4.0
jp-jdmb-jsts-score	jsts-score	396	yahoojapan/JGLUE	CC-BY-SA-4.0
jp-jdmb-kuci-choice	kuci-choice	400	ku-nlp/KUCI	CC-BY-SA-4.0
jp-jdmb-mintaka-ja-boolean-noul	mintaka-ja-boolean-noul	400	amazon-science/mintaka	CC-BY-4.0

Construction

Authored tasks

The Japanese synthetic subsets were created using llm-jp/llm-jp-4.1-32b-a3b-thinking, with recorded revision cda260706786758045e5e96bf4d738bbc01155b5. Their construction combines predefined task specifications, generated content, and subset-specific reference-label rules.

Where incorporated, Wikipedia-derived material retains CC BY-SA 4.0 / GFDL conditions, and Nemotron Personas Japan retains CC BY 4.0 conditions. The generation model and incorporated sources are recorded separately.

Source-derived tasks

Source-derived records preserve original answer semantics and candidate alignment. Boolean labels become Noul judgments; candidate labels become Choice references; numeric values and supplied probability distributions remain numeric or soft targets. Nominal categories are not assigned arbitrary numeric Score values. Related examples retain group identities for validation sampling.

bekko-* identifies records converted from the Bekko System One v0 training collection. It describes the data source, rather than a generation teacher. The 98 converted Bekko configurations record upstream revision 5f67e4e2fd2542e2fc3f82d86d7c3be705bda7b9 in sources.json. Use those recorded source pins when reproducing this release; an update to the upstream Dataset does not change already converted Saihai records.

The current Bekko card documents removal of two complete training cases matching active S1MB evaluation inputs after whitespace normalization. That targeted check is not comprehensive semantic decontamination, and its later upstream revision must not be described as automatically applied to this Saihai release. Declare effective source-family exposure even when evaluation-overlap filtering is performed.

Validation and evaluation

Validation is selected from source validation pools or development partitions constructed from training records. No validation records are moved into training. Grouping keeps related records together; validation measures development performance within this collection rather than generalization to unseen source families.

Reproducible whole-group selection with a 100-case cap

Group by group_id, falling back to parent_id and then id. Order groups by SHA-256 of the compact UTF-8 JSON array [seed_namespace, original_subset, group_id], using seed namespace saihai-v1-validation-whole-group-v1 and original subset identities recorded in the manifest. Use bounded subset-sum selection to maximize retained cases without exceeding 100. Keep the first solution for each reachable count under that group order, include every case in selected groups, and preserve source order. Groups larger than 100 cannot be selected. Pools within the cap are retained in full; indivisible groups can leave unused capacity.

Use this dataset for training and development of typed decisions. Shared source families, related examples, and generated content limit claims about independent generalization. Source-overlap checks do not establish the absence of paraphrases, translations, or prior model exposure. When reporting results, identify the dataset revision, evaluated configurations, model revision, and inference settings.

S1MB submissions and Known domains

Evaluate a fixed model configuration on english-v1, japanese-v1, or both. Submit validated measurements to the shared hotchpotch/s1mb-result Dataset, following the submission guide. Use separate training or validation data to select settings; do not tune against S1MB evaluation labels or scores.

For every benchmark whose source family was actually used in training, add a training_exposure entry with relationship: "same_source_family" to the model folder's metadata.json. The leaderboard displays it as a Known domain. Record the exact Saihai Dataset revision, configuration, split, pinned evidence and effective filtering. Apply this to the Japanese source-family configurations and to any applicable English source families in the bekko-* configurations. Declare only retained sources when training uses a filtered subset. Training on JSICK NLI and evaluation on JSICK STS still share a source family and require that disclosure; they do not establish use of the same evaluation judgments.

The maintainer reports removing duplicate examples and overlaps with the adopted Japanese evaluation examples during preparation. Deduplication does not remove the Known-domain disclosure requirement or establish absence of semantic overlap. If actual evaluation examples were used, declare evaluation_examples instead. Missing training disclosure means unknown, not an unseen domain. These declarations are provenance; they do not change inference instructions or benchmark scores. See the training disclosure rules.

For example, a model trained on the pinned jp-jdmb-jnli-choice train configuration would include the following entry in its existing metadata. Add equivalent entries for its other effective source families; this fragment is not a complete submission:

{
  "training_exposure": [
    {
      "benchmark_id": "jnli-choice",
      "relationship": "same_source_family",
      "dataset": "hotchpotch/saihai-v1-dataset",
      "revision": "906504ac4b72e133a613011318afd8206fb5e8ec",
      "config": "jp-jdmb-jnli-choice",
      "split": "train",
      "evidence_url": "https://huggingface.co/datasets/hotchpotch/saihai-v1-dataset/blob/906504ac4b72e133a613011318afd8206fb5e8ec/dataset-manifest.json",
      "note": "Training used the JNLI source-family train configuration. This declares source-family exposure, not use of S1MB evaluation examples."
    }
  ]
}

Licensing and attribution

Synthetic material authored by Yuichi Tateno is licensed under CC BY-SA 4.0. This covers the project-authored material in jp-catalog-retained, jp-catalog-revised, jp-long-context, jp-seed-guided, jp-unknown, and programmatic-v0. Attribute Yuichi Tateno and this dataset, indicate modifications, and distribute adapted material under the applicable ShareAlike terms.

Third-party material retains its original licenses and usage conditions. The synthetic-data license does not replace those conditions or apply to unrelated source datasets, generation code, or model weights. Collection-level metadata is therefore license: other. sources.json inventories source revisions, licenses, original partitions, and counts; per-record provenance identifies incorporated sources.

Eikos-derived GSM8K material retains its MIT conditions, and TAT-QA-derived material retains CC BY 4.0. BANKING77 follows the original PolyAI CC BY 4.0 declaration. The full inventory above lists source links and license qualifications, including conflicting declarations. LocalLLaMA's exact generation teacher is undisclosed, and HelpSteer2 lacks complete individual source URLs for underlying ShareGPT records.

For source attribution, consult sources.json and the linked original dataset publications. dataset-manifest.json provides configuration mappings, original subset identities, and split counts.