DEV Community

Cover image for Breath Actor: I recorded my breathing with a webcam and turned it into character animation
Vishal Singh
Vishal Singh

Posted on

Breath Actor: I recorded my breathing with a webcam and turned it into character animation

Presage SmartSpectra SDK Challenge Submission

This is a submission for the Presage SmartSpectra SDK Challenge

What I Built

Breath Actor is motion capture for breathing.

Animated characters need to breathe, or they look dead. Most of the time an animator makes one breath and repeats it forever. It works, but it looks like a machine, because real breathing is uneven. It slows down, it catches, and it stops when you are listening or scared. To get that, animators watch reference video and move keyframes by hand.

Voice actors perform a character's voice. I wanted to perform a character's breath.

With Breath Actor you sit in front of a laptop camera, pick a direction like "Hiding" or "Relief", and breathe the scene. The app:

  1. Records the rise and fall of your chest with the Presage SmartSpectra SDK. While you record, the character on screen breathes with you live.
  2. Saves a take as a short curve of numbers, with notes worked out from it (how many breaths, where you held).
  3. Lets you edit it: trim, make it stronger, smooth it, change the speed, or turn it into a clean loop.
  4. Puts it on a character. There are three, all built in code: Bramble (a sleeping fox), Understudy (a wooden mannequin) and Wisp (a lantern jellyfish with no lungs). You can also load your own .glb.
  5. Exports a standard animated .glb file, or the bare curve as JSON or CSV.

One detail I care about: one breath drives many body parts, and each part has its own lag. The shoulders follow the chest a little late, the head follows the shoulders, and a tail follows everything. That small delay is what makes one curve look like a whole body.

There is also an A/B button that swaps your take for an even, hand-keyed style loop, so you can see the difference yourself.

Recording breath as a performance is not my idea. Tsoli, Mahmood and Black did it in 2014 with a spirometer in Breathing Life into Shape. What is new here is that it needs only a webcam.

Demo

The video shows a real live take. You can see my camera view in the corner, the live breathing signal at the bottom, and the fox moving with it. About one minute of SDK warm-up is cut out. At one point the signal drops and the take restarts; I left that in on purpose, because it shows the quality check working.

The narration is an AI voice. The breathing is mine.

Code

https://github.com/vishalsingh06102000/breath-actor

Setup is npm install and npm start. You need a free SmartSpectra API key to record, but the app opens with a sample take, so you can try editing, characters and export without a camera or a key. There is no login.

npm install
npm start
npm test     # 23 tests
Enter fullscreen mode Exit fullscreen mode

How It Responds to Human Signals

Breath Actor uses the SmartSpectra Node.js SDK inside an Electron app. The SDK runs in the main process, so the API key never reaches the page.

SDK output What I do with it
Chest breathing trace This is the performance. Every keyframe in the export comes from it.
stable flag on each sample Quality check. A sample that is not stable ends the take.
Validation status (face, light, movement, chest in frame) Live coaching text in the booth, and the second half of the quality check.
Breathing rate Saved with the take, as a cross-check for my own breath counting.
Video output The small mirror in the booth. Never saved.
this.sdk = new S.SmartSpectraSDK({
    apiKey,
    requestedMetrics: [S.MetricType.CHEST_BREATHING, S.MetricType.BREATHING_RATE],
    enableTelemetry: false,
});

// every chest-breathing sample goes through the quality gate
for (const sample of metrics.breathing?.upperTrace || []) {
    const ok = this.collector.push(sample, this.validation, now);
    ...
}
Enter fullscreen mode Exit fullscreen mode

In real time: while the camera is on, each accepted sample moves the character on stage, so the performer can see their own breath on the fox, the mannequin or the jellyfish. The live trace is drawn bright when the SDK calls it stable and grey when it does not.

The rule I built around: a take is the longest unbroken stretch that the SDK itself marked as stable. If the signal drops, the take starts again. The app never joins two pieces together and never fills a gap. I would rather show "no take" than fake one.

Things I learned about the signal while building:

  • Breathing data arrives about one second after the movement. That would be bad for a game or an instrument. Here it does not matter, because you record first and animate later. I picked this idea partly because it fits what the signal is good at.
  • Warm-up took about 70 seconds on my MacBook Air before the SDK marked the signal stable. The app waits, counts the seconds, and tells you when it is rolling.
  • You must sit still and act with breath only. Moving or talking breaks the signal.
  • I only use the chest trace. The abdomen trace was not reliable enough for me.

After the take, the curve goes through an edit step (trim, smooth, loop) and a rig step (amount and lag per body part), and is baked to keyframes at 30 fps. Every .glb is checked with the Khronos glTF Validator and loaded back and compared before it is saved. In the demo, the jellyfish export has 49 animated channels from one breath.

Prize Categories

I am entering both.

Most Creative or Delightful Use. Most breathing apps tell you how to breathe. This one does the opposite: it asks you to act, and it takes your breath as a performance. "Breath acting" is the idea I had not seen before, and making a jellyfish with no lungs breathe with my own held breath is the part that still makes me smile.

Best Thoughtful Real-World Use. The need is real and small: animators and indie game developers spend time hand-keying breathing idles, and the result is usually a loop. Breath Actor gives them a recorded curve in a standard file format, with a loop tool, per-part lag, and CSV/JSON for any other tool. I tried to make it usable for someone who has never seen the SDK: plain-language coaching while recording, a sample take to try first, and clear messages when a take fails and why. Consent and data handling are below.

Consent and Data Handling

Consent

  • The camera cannot start until you read and accept a consent screen. It says what is recorded, what is stored, what is sent, and that this is not a medical tool.
  • You can withdraw consent in Settings. After that the camera cannot start until you accept again.
  • The camera runs only while a take is rolling. Cut, Discard, leaving the booth or closing the window stops it. A red "Rolling" light shows on screen while it is on.
  • The app is for recording yourself, or someone who is with you and has agreed. It has no background or hidden recording mode.

Storage

  • Breath Actor never saves or uploads pictures, video or sound. Frames go to the SmartSpectra SDK on your computer. The booth mirror is a small preview held in memory, and you can blur it.
  • A take is one small JSON file: a 0 to 1 curve at 30 fps, its name, your edits. It is stored in the app's data folder on your computer.
  • Exports (.glb, JSON, CSV, a clip of the 3D stage) are saved only where you choose. They contain no camera data.
  • If you paste your API key into the app, it stays in the main process and is encrypted with the operating system keychain.

Sharing

  • Breath Actor shares nothing. It has no account, no analytics and no server, and its window is blocked from making network requests.

Retention and deletion

  • Takes stay on your computer until you delete them. You can delete one take from its card, or all takes in Settings.
  • Nothing is kept anywhere else by Breath Actor.

SDK telemetry

  • The SmartSpectra SDK contacts Presage to verify the API key, and downloads its measurement model on first use. Key verification is tied to your Presage account, so it is not anonymous.
  • Breath Actor starts the SDK with enableTelemetry: false. Presage's Telemetry and Privacy page says that with telemetry disabled the SDK "starts no telemetry session and transmits nothing", and that telemetry never includes video frames or measured values.

AI services

  • The app uses no AI service. Nothing you record is sent to any AI model.
  • For transparency: I built the app with help from an AI coding assistant, and the narration in the demo video is an AI voice.

Not medical

  • The curve is a relative shape for animation. It is not a measurement of health. The app makes no diagnosis or treatment claims and is not for monitoring anyone.

Sample take

  • The repository includes one sample take. It is my own 14 second breathing curve, so people can try the app without a camera.

Top comments (0)