End-to-End Pipelines for Large-Scale Audio, Text, and Network Analysis David Jurgens and Dallas Card. 12th International Conference on Computational Social Science (IC²S²), Burlington VT, July 2026. Three hours.
If you are attending, start at davidjurgens.github.io/ic2s2-podcast-tutorial.
Podcasts are a fast-growing, under-used source for computational social science: long-form,
conversational public discourse, with recurring speakers who move across shows and a meso-level
network of hosts and guests. This tutorial moves podcast data through a full research pipeline: collect your own,
structure it, analyze at scale, annotate, then adapt the whole thing to your own question. It uses
new crawling and extraction code alongside the released Structured Podcast Research Corpus
(SPoRC) and its sporc tooling. Everything runs locally on a CPU (no GPU, no Colab).
Each module is built around worked research questions rather than a tour of features: whether talk about policing changed around 25 May 2020, who holds the floor in an interview, why the obvious version of the guest network is wrong. Ethics is part of every module, not a closing section.
Collection and analysis meet at a shared record schema: a corpus you build from raw audio uses the same fields as SPoRC, so the analysis in modules 3–6 runs unchanged on your own data.
| Path | What it is |
|---|---|
notebooks/ |
One self-contained notebook per module (+ a multilingual bonus and a capstone template). Start here. The .ipynb files are generated from notebooks/src/*.py. |
podcast_pipeline/ |
Installable Python package: the single-server audio→turns pipeline (whisperX, openSMILE, pyannote), distilled from the production SPoRC system. |
scripts/ |
Reproducible data-prep + launch scripts (build the local corpus, the audio pack, the annotation data; launch Potato). |
potato_interfaces/ |
Ten ready-to-run Potato annotation designs (turn, episode, span, waveform segment, ELAN-style tiers, pairwise) over real episodes. See its README. |
docs/ |
Source of the tutorial website, served by GitHub Pages. Slide decks are in docs/slides/, linked from the agenda. |
requirements.txt |
The tutorial environment. requirements-audio.txt adds the speech stack for processing your own audio. |
INSTALL.md |
Full setup instructions (do this before the session). |
Full instructions are in INSTALL.md; the short version:
# 1. environment
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt # the tutorial
# If you want to process audio data, uncomment and run the following line:
# pip install -r requirements-audio.txt
python -m spacy download en_core_web_sm
# 2. Hugging Face: accept the SPoRC terms
# https://huggingface.co/datasets/blitt/SPoRC
huggingface-cli login
# 3. Start exploring the code
cd notebooks && jupyter lab # open 01_crawl_and_extract.ipynbThat is all of the setup. Modules 1, 2 and 3 each fetch what they need on first run (~527 MB, resumable) and skip it thereafter, so you can start at whichever of them you like. Module 2 also builds its audio clip on demand, though it requires you to install the speech libraries.
Launch Jupyter from notebooks/, not from the repo root: every path inside a
notebook is relative to notebooks/, which is the working directory JupyterLab
gives a notebook by default.
To get the data in advance, we have provided scripts that will pull small slices of each genre that you can use in the subsequent notebooks.
python scripts/fetch_genre.py --list # 12 genre slices, what you have
python scripts/fetch_genre.py --genre news # -> data/genres/news/
# add --acoustics (+137 MB) only for module 5
# and the other data-building scripts (nothing here blocks a notebook)
python scripts/prepare_audio_pack.py # -> data/audio_pack/ (a short real clip)
python scripts/build_potato_data.py # -> potato_interfaces/*/data/
python scripts/setup_mallet.py # topic models in module 4 (needs Java)Each notebook is standalone on the pre-built data and should run fine on a CPU.
Notebooks 2, 7, 8 and the capstone import from podcast_pipeline, so the same code runs in a server pipeline.
Notebooks 3–6 work directly against the corpus.
Notebooks 1–6 are organised around four research questions each.
Some of the notebooks vary in their length of running; here's approximate runtimes from a macbook pro:
| # | Notebook | Focus | Runs in |
|---|---|---|---|
| 1 | 01_crawl_and_extract |
Feed anatomy and discovery. What you can rely on across feeds, what polite crawling costs, whether SPoRC's 2020 window could be rebuilt by crawling today, and whether to believe a corpus's "guest name" | ~1 min |
| 2 | 02_audio_to_transcript |
whisperX + openSMILE + diarization → SPoRC-aligned turns. Does the ASR model change your answer, which recording conditions break ASR, and can you find broken episodes without listening | ~4 min |
| 3 | 03_sporc_load_and_navigate |
sporc loading, as an object model and as frames over the same files; why a contiguous slice, and the three caveats every later module depends on |
~6 s |
| 4 | 04_text_conversation |
Units of observation. Did talk about policing change around 25 May 2020, or did the mix of shows change; do hosts and guests frame it differently; what are these shows about beyond what you thought to ask | ~2 min |
| 5 | 05_audio_acoustic |
Who holds the floor and how speakers hand off; does vocal arousal track contentious content; do genres sound different or just talk about different things | ~15 s |
| 6 | 06_networks |
The guest network: why the obvious version of it is wrong, what circulation looks like after validation, and whether communities follow genre | ~15 s |
| 7 | 07_annotation_potato |
launch annotation interfaces; read labels back | interactive |
| 8 | 08_multilingual_bonus |
multilingual ASR + cross-lingual analysis (take-home) | ~10 s |
| — | capstone_template |
end-to-end template + transcribe-your-own-episode (take-home) | ~30 s |
The
.ipynbfiles are generated from plain-Python sources innotebooks/src/and are committed with their outputs, so you can read a module before running anything. Edit the source, not the notebook. Seenotebooks/README.mdfor prerequisites per notebook.
To get you started creating your own datasets, we've put together a teaching-oriented, single-server distillation of the production system that built SPoRC. The pipeline is focused on one episode at a time to show you how each step works. If you decide to scale up, you'll likely want to add more infrastructure (e.g., work queues, slurm, GPU pools, etc.). Each stage here is CPU-based but you can switch it to GPU or MPS if you have those available.
from podcast_pipeline import process_episode
turns = process_episode("episode.mp3", out_dir="./out") # -> SPoRC-aligned turns DataFrameaudio ─► transcribe ─► prosody ─► merge ─► turns [─► optional cross-episode speaker linking]
16 kHz whisperX openSMILE word-level SPoRC-aligned
mono (word ts) eGeMAPSv2 joins speaker turns
Full details, the SPoRC schema mapping, and CLI usage are in
podcast_pipeline/README.md.
Most annotation tools are not easily setup for podcast data, which is long and multimodal.
If you will need to annotate podcast data as a part of your research, we've inlcuded
ten annotation designs for common tasks using the Potato
annotation tool, which has multiple built-in features for working with podcasts.
Full reference (what each is for, what it stores, how to adapt it) in
potato_interfaces/README.md.
To get started annotating, we put together easy demo scripts to show how the interfaces look and let you try annotating yourself.
python scripts/launch_potato.py --list # names, ports, descriptions
python scripts/launch_potato.py dialogue_sporc # build data if needed, then serveOpen the printed URL and type any username (no password). Ctrl-C stops it.
| Interface | Port | Unit | What the annotator does |
|---|---|---|---|
| text | 8000 | turn | stance, topics, engagement, plus in-vivo codebook coding |
| audio | 8001 | time segment | draws the segments on the waveform, then labels each act |
| video | 8002 | clip | judges whether the picture adds anything the audio does not |
| dialogue_sporc | 8003 | turn in context | per-turn acts, claim/evidence spans, evidence→claim links |
| dialogue_whisper | 8004 | ASR segment | assigns speakers to an undiarized transcript |
| transcript_quality | 8005 | ASR segment | tags ASR errors and types the correction |
| interaction | 8006 | the overlap | classifies barge-ins on a dual-track timeline |
| episode | 8008 | episode | format, topic taxonomy, soft labels, confidence |
| tiers | 8007 | time interval | ELAN-style layers: speaker → act → stance → production |
| compare | 8009/8010 | pair / tuple | pairwise and best-worst over real host questions |
Audio streams from the publisher's CDN. Nothing is copied into the repo, and
scripts/build_potato_data.pyprobes every URL before writing it. A working network connection is therefore required. To work offline, drop the file in<interface>/media/and point the config at/media/<file>.mp3; Potato serves that route itself.Run one interface at a time. Flask session cookies are scoped to the host, not the port, so two Potato servers on
localhostoverwrite each other's sessions.
All of the scripts require you to either access SPoRC through Huggingface or build your own dataset. We've included a few scripts to help you access and locally package the SPoRC data.
| Script | Produces | Notes |
|---|---|---|
scripts/fetch_genre.py |
data/genres/<genre>/ |
Downloads one category-contiguous SPoRC slice and filters the catalogs to match, so counts and searches are true of what is on disk. Twelve genres in data/genres.yaml; 220–310 MB each, 340–450 MB with acoustics. Prints the byte cost before downloading; resumable. Callable from a notebook as nbsetup.fetch("culture"). |
scripts/make_subset.py |
(used by the above) | Vendored from the sporc repo, because the pip package ships the library and not its repo scripts. |
scripts/build_notebooks.py |
notebooks/*.ipynb |
Renders the notebooks from their Python sources in notebooks/src/. Notebook JSON is unreviewable in a diff, so the sources are what you edit. |
scripts/setup_mallet.py |
tools/Mallet-*/ |
Downloads MALLET for module 4's topic model. Optional; falls back to scikit-learn NMF. |
scripts/prepare_audio_pack.py |
data/audio_pack/ |
Downloads a short real clip + a SPoRC-derived RTTM + reference turns, and verifies with a small ASR model that the audio still matches SPoRC's 2020 transcript (hosts re-cut their back catalogues). Also builds link_pack/, windows from three episodes of one show, for the ECAPA cross-episode speaker linking in module 2 §8b (--link-episodes 0 to skip). Podcast feeds go up and down, so re-run if a download fails. |
scripts/build_potato_data.py |
potato_interfaces/*/data/ |
Fills the ten annotation designs with real SPoRC turns, real episode metadata, and the pipeline's own whisperX output. Probes every podcast URL before writing it, and falls back to another episode when one 404s. --only <name> rebuilds one; --list shows them all. |
SPoRC packs podcasts into ~128 MB parts sorted by category, with one row group per podcast. A random 1,000-podcast sample spans 116 of 127 turn parts, so you'd end up needing to download 11.6GB---too much for the time of our tutorial! In contrast, a 1,000 consecutive in-category podcasts span 1-2 slices which is around 0.2-0.3GB. Every slice here is a contiguous in-genre run for that reason. The ~230 MB of metadata catalogs that come down once describe all the 228,099 podcasts in every genre, so metadata-only work (module 6's networks) is cross-genre and free.
- SPoRC corpus,
blitt/SPoRCon Hugging Face (gated). sporcpackage, github.com/davidjurgens/sporc, to load, search and iterate the corpus.- Potato, github.com/davidjurgens/potato, the annotation tool.
- Paper: Litterer, Jurgens & Card, "Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus," ACL 2025. aclanthology.org/2025.acl-long.1222
Python 3.10+ and a Hugging Face account that has accepted the SPoRC terms. GPU optional. The tutorial runs entirely on a laptop CPU, and the notebooks call out where a GPU changes the speed/quality trade-off.
There are two requirement sets, and the session only needs the first:
| Install | Adds | Needed for | |
|---|---|---|---|
| Work with SPoRC | pip install -r requirements.txt |
— | modules 1, 3, 4, 5, 6, 7 |
| Make your own data | pip install -r requirements-audio.txt |
whisperX, pyannote, openSMILE, librosa, ffmpeg |
module 2's live sections, module 8, the capstone |
The data pack ships SPoRC's own diarization as clip.rttm and the genre slices ship precomputed
acoustics, so nothing in the session requires a speech model. The two gated
pyannote models are needed only if you
diarize audio of your own.
If you use this tutorial's code or materials in your research, please cite the tutorial:
@misc{jurgens2026podcasts,
author = {Jurgens, David and Card, Dallas},
title = {Podcasts as Social Data: End-to-End Pipelines for
Large-Scale Audio, Text, and Network Analysis},
year = {2026},
note = {Tutorial at the 12th International Conference on
Computational Social Science (IC\textsuperscript{2}S\textsuperscript{2})},
address = {Burlington, VT},
howpublished = {\url{https://github.com/davidjurgens/ic2s2-podcast-tutorial}}
}If your work uses the SPoRC corpus itself, also cite the paper that introduced it:
@inproceedings{litterer-etal-2025-mapping,
title = "Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus",
author = "Litterer, Benjamin Roger and
Jurgens, David and
Card, Dallas",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.1222/",
doi = "10.18653/v1/2025.acl-long.1222",
pages = "25132--25154"
}The tutorial's own material is CC BY-NC-SA 4.0: use it, teach from it, adapt it, credit the source, share adaptations on the same terms, and do not sell it. SPoRC, podcast audio, and the third-party packages are under their own terms — see LICENSE.
