Skip to content

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Podcasts as Social Data — IC²S² 2026 Tutorial

End-to-End Pipelines for Large-Scale Audio, Text, and Network Analysis David Jurgens and Dallas Card. 12th International Conference on Computational Social Science (IC²S²), Burlington VT, July 2026. Three hours.

If you are attending, start at davidjurgens.github.io/ic2s2-podcast-tutorial.

Podcasts are a fast-growing, under-used source for computational social science: long-form, conversational public discourse, with recurring speakers who move across shows and a meso-level network of hosts and guests. This tutorial moves podcast data through a full research pipeline: collect your own, structure it, analyze at scale, annotate, then adapt the whole thing to your own question. It uses new crawling and extraction code alongside the released Structured Podcast Research Corpus (SPoRC) and its sporc tooling. Everything runs locally on a CPU (no GPU, no Colab).

Each module is built around worked research questions rather than a tour of features: whether talk about policing changed around 25 May 2020, who holds the floor in an interview, why the obvious version of the guest network is wrong. Ethics is part of every module, not a closing section.


The arc

Collect, structure, analyze, apply — the four stages of the tutorial pipeline and the notebook that covers each

Collection and analysis meet at a shared record schema: a corpus you build from raw audio uses the same fields as SPoRC, so the analysis in modules 3–6 runs unchanged on your own data.


Repository layout

Path What it is
notebooks/ One self-contained notebook per module (+ a multilingual bonus and a capstone template). Start here. The .ipynb files are generated from notebooks/src/*.py.
podcast_pipeline/ Installable Python package: the single-server audio→turns pipeline (whisperX, openSMILE, pyannote), distilled from the production SPoRC system.
scripts/ Reproducible data-prep + launch scripts (build the local corpus, the audio pack, the annotation data; launch Potato).
potato_interfaces/ Ten ready-to-run Potato annotation designs (turn, episode, span, waveform segment, ELAN-style tiers, pairwise) over real episodes. See its README.
docs/ Source of the tutorial website, served by GitHub Pages. Slide decks are in docs/slides/, linked from the agenda.
requirements.txt The tutorial environment. requirements-audio.txt adds the speech stack for processing your own audio.
INSTALL.md Full setup instructions (do this before the session).

Quick start

Full instructions are in INSTALL.md; the short version:

# 1. environment
python -m venv .venv
source .venv/bin/activate              # Windows: .venv\Scripts\activate
pip install -r requirements.txt        # the tutorial
# If you want to process audio data, uncomment and run the following line:
# pip install -r requirements-audio.txt
python -m spacy download en_core_web_sm

# 2. Hugging Face: accept the SPoRC terms
#    https://huggingface.co/datasets/blitt/SPoRC
huggingface-cli login

# 3. Start exploring the code
cd notebooks && jupyter lab      # open 01_crawl_and_extract.ipynb

That is all of the setup. Modules 1, 2 and 3 each fetch what they need on first run (~527 MB, resumable) and skip it thereafter, so you can start at whichever of them you like. Module 2 also builds its audio clip on demand, though it requires you to install the speech libraries.

Launch Jupyter from notebooks/, not from the repo root: every path inside a notebook is relative to notebooks/, which is the working directory JupyterLab gives a notebook by default.

To get the data in advance, we have provided scripts that will pull small slices of each genre that you can use in the subsequent notebooks.

python scripts/fetch_genre.py --list           # 12 genre slices, what you have
python scripts/fetch_genre.py --genre news     # -> data/genres/news/
# add --acoustics (+137 MB) only for module 5

# and the other data-building scripts (nothing here blocks a notebook)
python scripts/prepare_audio_pack.py      # -> data/audio_pack/   (a short real clip)
python scripts/build_potato_data.py       # -> potato_interfaces/*/data/
python scripts/setup_mallet.py            # topic models in module 4 (needs Java)

The modules

Each notebook is standalone on the pre-built data and should run fine on a CPU. Notebooks 2, 7, 8 and the capstone import from podcast_pipeline, so the same code runs in a server pipeline. Notebooks 3–6 work directly against the corpus. Notebooks 1–6 are organised around four research questions each. Some of the notebooks vary in their length of running; here's approximate runtimes from a macbook pro:

# Notebook Focus Runs in
1 01_crawl_and_extract Feed anatomy and discovery. What you can rely on across feeds, what polite crawling costs, whether SPoRC's 2020 window could be rebuilt by crawling today, and whether to believe a corpus's "guest name" ~1 min
2 02_audio_to_transcript whisperX + openSMILE + diarization → SPoRC-aligned turns. Does the ASR model change your answer, which recording conditions break ASR, and can you find broken episodes without listening ~4 min
3 03_sporc_load_and_navigate sporc loading, as an object model and as frames over the same files; why a contiguous slice, and the three caveats every later module depends on ~6 s
4 04_text_conversation Units of observation. Did talk about policing change around 25 May 2020, or did the mix of shows change; do hosts and guests frame it differently; what are these shows about beyond what you thought to ask ~2 min
5 05_audio_acoustic Who holds the floor and how speakers hand off; does vocal arousal track contentious content; do genres sound different or just talk about different things ~15 s
6 06_networks The guest network: why the obvious version of it is wrong, what circulation looks like after validation, and whether communities follow genre ~15 s
7 07_annotation_potato launch annotation interfaces; read labels back interactive
8 08_multilingual_bonus multilingual ASR + cross-lingual analysis (take-home) ~10 s
— capstone_template end-to-end template + transcribe-your-own-episode (take-home) ~30 s

The .ipynb files are generated from plain-Python sources in notebooks/src/ and are committed with their outputs, so you can read a module before running anything. Edit the source, not the notebook. See notebooks/README.md for prerequisites per notebook.


The podcast_pipeline package

To get you started creating your own datasets, we've put together a teaching-oriented, single-server distillation of the production system that built SPoRC. The pipeline is focused on one episode at a time to show you how each step works. If you decide to scale up, you'll likely want to add more infrastructure (e.g., work queues, slurm, GPU pools, etc.). Each stage here is CPU-based but you can switch it to GPU or MPS if you have those available.

from podcast_pipeline import process_episode
turns = process_episode("episode.mp3", out_dir="./out")   # -> SPoRC-aligned turns DataFrame
audio ─► transcribe ─► prosody ─► merge ─► turns   [─► optional cross-episode speaker linking]
16 kHz   whisperX      openSMILE   word-level  SPoRC-aligned
mono     (word ts)     eGeMAPSv2   joins       speaker turns

Full details, the SPoRC schema mapping, and CLI usage are in podcast_pipeline/README.md.


The annotation interfaces

Most annotation tools are not easily setup for podcast data, which is long and multimodal. If you will need to annotate podcast data as a part of your research, we've inlcuded ten annotation designs for common tasks using the Potato annotation tool, which has multiple built-in features for working with podcasts. Full reference (what each is for, what it stores, how to adapt it) in potato_interfaces/README.md.

To get started annotating, we put together easy demo scripts to show how the interfaces look and let you try annotating yourself.

python scripts/launch_potato.py --list          # names, ports, descriptions
python scripts/launch_potato.py dialogue_sporc  # build data if needed, then serve

Open the printed URL and type any username (no password). Ctrl-C stops it.

Interface Port Unit What the annotator does
text 8000 turn stance, topics, engagement, plus in-vivo codebook coding
audio 8001 time segment draws the segments on the waveform, then labels each act
video 8002 clip judges whether the picture adds anything the audio does not
dialogue_sporc 8003 turn in context per-turn acts, claim/evidence spans, evidence→claim links
dialogue_whisper 8004 ASR segment assigns speakers to an undiarized transcript
transcript_quality 8005 ASR segment tags ASR errors and types the correction
interaction 8006 the overlap classifies barge-ins on a dual-track timeline
episode 8008 episode format, topic taxonomy, soft labels, confidence
tiers 8007 time interval ELAN-style layers: speaker → act → stance → production
compare 8009/8010 pair / tuple pairwise and best-worst over real host questions

Audio streams from the publisher's CDN. Nothing is copied into the repo, and scripts/build_potato_data.py probes every URL before writing it. A working network connection is therefore required. To work offline, drop the file in <interface>/media/ and point the config at /media/<file>.mp3; Potato serves that route itself.

Run one interface at a time. Flask session cookies are scoped to the host, not the port, so two Potato servers on localhost overwrite each other's sessions.


Data & reproducibility

All of the scripts require you to either access SPoRC through Huggingface or build your own dataset. We've included a few scripts to help you access and locally package the SPoRC data.

Script Produces Notes
scripts/fetch_genre.py data/genres/<genre>/ Downloads one category-contiguous SPoRC slice and filters the catalogs to match, so counts and searches are true of what is on disk. Twelve genres in data/genres.yaml; 220–310 MB each, 340–450 MB with acoustics. Prints the byte cost before downloading; resumable. Callable from a notebook as nbsetup.fetch("culture").
scripts/make_subset.py (used by the above) Vendored from the sporc repo, because the pip package ships the library and not its repo scripts.
scripts/build_notebooks.py notebooks/*.ipynb Renders the notebooks from their Python sources in notebooks/src/. Notebook JSON is unreviewable in a diff, so the sources are what you edit.
scripts/setup_mallet.py tools/Mallet-*/ Downloads MALLET for module 4's topic model. Optional; falls back to scikit-learn NMF.
scripts/prepare_audio_pack.py data/audio_pack/ Downloads a short real clip + a SPoRC-derived RTTM + reference turns, and verifies with a small ASR model that the audio still matches SPoRC's 2020 transcript (hosts re-cut their back catalogues). Also builds link_pack/, windows from three episodes of one show, for the ECAPA cross-episode speaker linking in module 2 §8b (--link-episodes 0 to skip). Podcast feeds go up and down, so re-run if a download fails.
scripts/build_potato_data.py potato_interfaces/*/data/ Fills the ten annotation designs with real SPoRC turns, real episode metadata, and the pipeline's own whisperX output. Probes every podcast URL before writing it, and falls back to another episode when one 404s. --only <name> rebuilds one; --list shows them all.

SPoRC packs podcasts into ~128 MB parts sorted by category, with one row group per podcast. A random 1,000-podcast sample spans 116 of 127 turn parts, so you'd end up needing to download 11.6GB---too much for the time of our tutorial! In contrast, a 1,000 consecutive in-category podcasts span 1-2 slices which is around 0.2-0.3GB. Every slice here is a contiguous in-genre run for that reason. The ~230 MB of metadata catalogs that come down once describe all the 228,099 podcasts in every genre, so metadata-only work (module 6's networks) is cross-genre and free.


Related resources

Requirements at a glance

Python 3.10+ and a Hugging Face account that has accepted the SPoRC terms. GPU optional. The tutorial runs entirely on a laptop CPU, and the notebooks call out where a GPU changes the speed/quality trade-off.

There are two requirement sets, and the session only needs the first:

Install Adds Needed for
Work with SPoRC pip install -r requirements.txt — modules 1, 3, 4, 5, 6, 7
Make your own data pip install -r requirements-audio.txt whisperX, pyannote, openSMILE, librosa, ffmpeg module 2's live sections, module 8, the capstone

The data pack ships SPoRC's own diarization as clip.rttm and the genre slices ship precomputed acoustics, so nothing in the session requires a speech model. The two gated pyannote models are needed only if you diarize audio of your own.

Citing

If you use this tutorial's code or materials in your research, please cite the tutorial:

@misc{jurgens2026podcasts,
  author       = {Jurgens, David and Card, Dallas},
  title        = {Podcasts as Social Data: End-to-End Pipelines for
                  Large-Scale Audio, Text, and Network Analysis},
  year         = {2026},
  note         = {Tutorial at the 12th International Conference on
                  Computational Social Science (IC\textsuperscript{2}S\textsuperscript{2})},
  address      = {Burlington, VT},
  howpublished = {\url{https://github.com/davidjurgens/ic2s2-podcast-tutorial}}
}

If your work uses the SPoRC corpus itself, also cite the paper that introduced it:

@inproceedings{litterer-etal-2025-mapping,
    title = "Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus",
    author = "Litterer, Benjamin Roger  and
      Jurgens, David  and
      Card, Dallas",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.acl-long.1222/",
    doi = "10.18653/v1/2025.acl-long.1222",
    pages = "25132--25154"
}

License

The tutorial's own material is CC BY-NC-SA 4.0: use it, teach from it, adapt it, credit the source, share adaptations on the same terms, and do not sell it. SPoRC, podcast audio, and the third-party packages are under their own terms — see LICENSE.

About

No description, website, or topics provided.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages