This is a submission for the Presage SmartSpectra SDK Challenge
What I Built
Human Health Insight is a web app for a quick wellness check using only a webcam. You sit in front of the camera, and in about a minute it shows your heart rate and breathing rate, live, using the Presage SmartSpectra SDK. When both readings have settled, it stops on its own and gives you simple, friendly health tips written by an LLM. Examples are "rest and measure again", "drink some water" or "consider a routine check-up with a doctor".
What it lets people do:
- Check their vitals without a wearable. No smartwatch or pulse oximeter, just the laptop camera.
- Get guided while measuring. The app passes on SmartSpectra's live feedback, such as "Move a little closer", "Too dark, add more light" or "Hold still".
- Understand their numbers. Readings are put into plain-language bands (typical, low or high), and the app suggests a sensible next step.
- Stay private and secure. Users sign in through Keycloak. Video is processed live in memory and never stored, and the AI only ever receives the two numbers, never who you are.
Human Health Insight gives wellness information, not a medical diagnosis. SmartSpectra's own documentation says its metrics are for general wellness and are not cleared for diagnosis or treatment. The app says this on every results screen.
The user flow
- Log in on a custom Keycloak login page (health illustration on the left, form on the right).
- From the dashboard, open Measure vitals and click Start measurement. The camera turns on with face-scan guides and a LIVE badge.
- Watch heart rate and breathing update in real time, each marked Stabilizing… and then ✓ Stable reading.
- The measurement stops automatically once both readings are stable.
- Click Get health tips to see a 🟢 / 🟡 / 🔴 notice and personalised wellness tips.
Tech Stack
| Layer | Technology |
|---|---|
| Frontend | React 19, TypeScript, Vite |
| Authentication |
react-oidc-context, OIDC + PKCE |
| Backend | Node.js, Express 5, TypeScript |
| Real-time Communication | WebSocket (ws) |
| JWT Verification | jose |
| Vitals / Health Signals |
@smartspectra/node-sdk (Presage SmartSpectra) |
| Identity Provider | Keycloak 26 |
| Database | PostgreSQL |
| Keycloak UI | Keycloakify (React) |
| AI / Health Tips | Ollama (gpt-oss:20b-cloud or other local models) |
| Fallback | Rule-based health recommendation engine |
| Deployment | Docker |
Demo
Code
abhisek247767
/
Human-Health-Insight
A camera-based human health insights platform that captures and visualizes real-time physiological signals, including pulse, breathing rate, HRV, and facial expressions, through an interactive dashboard.
How It Responds to Human Signals
Signals used
| Presage signal | Where it comes from | How the app uses it |
|---|---|---|
| Pulse rate (heart rate, bpm) | metrics.cardio.pulseRate |
Live reading card with a beating-heart icon; input to the health tips |
| Breathing rate (breaths/min) | metrics.breathing.rate |
Live reading card with a breathing icon; input to the health tips |
stable flag + confidence |
Each MeasurementWithConfidence sample |
Shows Stabilizing… or ✓ Stable reading; auto-stops the session; only stable values are used for tips |
| Validation status |
validationStatus event (ValidationCode) |
Real-time coaching: "No face found", "Too dark", "Too much movement", "Move closer"… |
How the experience responds in real time
-
While measuring: every SmartSpectra
metricsevent goes straight to the browser over a WebSocket, so the heart-rate and breathing numbers update live. The heart icon beats and the breath icon moves while readings come in. -
Coaching: each
validationStatuscode becomes a plain instruction. If the user leans away, turns off a light or starts moving, the message changes right away and tells them how to fix it. -
Knowing when to stop: the backend watches the SDK's per-sample
stableflag. When heart rate and breathing have both stayed stable for 3 seconds, the session ends by itself and the camera turns off. There's no fixed timer, so a calm, well-lit user finishes faster. A 120-second safety limit covers the case where readings never settle. - After measuring: only stable readings are sent for tips. An unsettled value (for example a breathing reading still "Stabilizing…") is left out and the user is told why, which avoids false alarms.
-
Health tips:
- Fixed rules, not the AI, decide the band and urgency. Heart rate: under 40 or over 120 is 🔴, 40–49 or 101–120 is 🟡, 50–100 is 🟢. Breathing: under 8 or over 30 is 🔴, 8–11 or 25–30 is 🟡, 12–24 is 🟢.
- The Ollama model then writes the friendly summary and 3–5 tips within those limits. It's told never to diagnose or suggest medication.
- If the LLM is unavailable, built-in tips are shown instead, so the feature always works.
Prize Categories
Most Creative or Delightful Use
The project turns a simple camera session into an interactive human-sensing experience. Instead of showing raw physiological signals as isolated numbers, it combines real-time human signals with an AI-powered explanation layer, making the results easier and more engaging to understand.
Best Thoughtful Real-World Use
The project is designed as a preliminary human health insights experience for people who want to understand their current physiological signals in a simple and accessible way. Users authenticate securely, provide explicit camera consent, and receive an easy-to-understand explanation of the signals captured during the session. The application is designed for informational and wellness purposes only and does not provide medical diagnosis, treatment, or medical decision-making.
Consent and Data Handling
Camera consent
- The camera never starts on its own. The Start measurement button stays disabled until the user ticks "I agree to use my camera. Video is processed live and not stored."
- It's always visible when the camera is on. The page shows the live preview, face-scan guides and a red LIVE · 12s badge, and the computer's camera light is on.
- The user stays in control. Stop ends the session at any time. The camera is also released automatically when both readings are stable, when the 120-second safety limit is reached, or when the user leaves the page.
- Only signed-in users can measure. The measurement connection requires a valid Keycloak login and accepts one session at a time. There's no background, scheduled or remote monitoring.
- In
browsermode (for remote deployments), the browser's own camera permission prompt is shown as well.
What data is processed, and where
| Data | Where it goes | Stored? |
|---|---|---|
| Camera frames | Processed in memory by the SmartSpectra SDK in the app's backend. In camera mode a small preview is sent only to the user's own browser | ❌ Never saved to disk, never logged |
| Heart rate and breathing readings | Shown in the user's browser. Sent to the backend only when the user clicks Get health tips | ❌ Not saved in any database. Gone when the page is refreshed or closed |
| Readings sent for tips | Just the two numbers and their bands (e.g. heartRate: 68, level: "normal"). No name, email or user ID
|
❌ Not stored by the app |
| Account (username, email, password hash) | Keycloak, backed by the app's own Postgres database | ✅ Until the account is deleted |
| Server logs | Only frame counts (e.g. "29.8 fps, 149/149 accepted") and error codes | No frames, readings or identities |
Sharing with third parties
-
SmartSpectra (Presage): the SDK runs locally in the backend. SDK telemetry is turned off (
enableTelemetry: false). The SDK still needs an API key that is checked with Presage's servers, per Presage's documentation. The app itself never uploads camera frames anywhere. -
Optional AI service (Ollama): health tips are optional and only generated when the user clicks Get health tips. With the default cloud model (
gpt-oss:20b-cloud), only the anonymous readings and bands are sent to Ollama's cloud; Ollama states it doesn't use prompts to train models. SettingOLLAMA_MODELto a local model (e.g.qwen2.5:3b) keeps everything on the machine. If no AI is available, built-in rule-based tips are used and nothing is sent.
Retention and deletion
- Video and readings: nothing is kept, so there's nothing to delete. Each measurement exists only for the length of the session.
- Account: an administrator can delete a user in the Keycloak admin console (Users → user → Delete), which removes their login data from the database.
-
Developer console logs: readings are printed to the browser console only in development builds (
npm run dev). Production builds don't log them.
Security
- Login: Keycloak with OIDC Authorization Code + PKCE. Tokens are kept in browser memory, not in localStorage. Every API call and the measurement WebSocket check the JWT's signature, issuer, audience and expiry.
- WebSocket: only accepts connections from the app's own origin, requires a valid token as the first message, and has a 1 MB message limit and an authentication timeout.
-
Secrets: the SmartSpectra and Ollama keys live only in the backend's
.env, never in frontend code.
Wellness only, not medical
- The app does not diagnose, treat or make medical decisions. Every results screen says the readings are wellness information only and that camera-based readings can be inaccurate.
- The 🟢 / 🟡 / 🔴 bands come from fixed general resting ranges, not from the AI. The most they suggest is resting and measuring again, a routine check-up with a doctor, or seeking care if the user feels unwell.
- The AI is told never to diagnose, name a condition or suggest medication.
Top comments (0)