Dungeons Daily 🐉

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

DungeonsDaily: Turning Workouts Into Voice-Driven Fantasy Adventures With Gemma

What if your next workout felt less like completing a set of exercises and more like surviving a dungeon, exploring a mysterious forest, or answering a Dungeon Master's call?

That idea led me to build DungeonsDaily, a voice-driven fantasy fitness RPG that combines AI-generated storytelling, real-world exercise, browser-native voice interaction, and deterministic game mechanics.

The goal is simple: make technology a reason to get moving rather than another reason to stay glued to a screen.

Instead of constantly watching a video or counting repetitions manually, players can interact with a Dungeon Master through their voice, follow exercise challenges, and experience a story that evolves throughout their workout.

At the heart of the project is Gemma, an open-weight AI model used as the Dungeon Master, while a separate game engine remains responsible for the actual rules and progression.

What I Built

DungeonsDaily is a voice-first fitness RPG designed around short, workout-sized adventures.

When a player starts an adventure, they choose a duration of 10, 20, or 30 minutes. The application then creates a session in which the Dungeon Master narrates the experience, responds to the player's input, and guides them through challenges.

The experience brings together several components:

• Voice-driven interaction: Players can speak to the Dungeon Master instead of typing every response.
• Dynamic fantasy storytelling: Gemma generates Dungeon Master narration through a structured AI provider layer.
• Exercise challenges: The game engine tracks the current challenge and validates exercise completion.
• Persistent sessions: MongoDB stores users, sessions, and recent conversation turns.
• Immersive audio: Environmental ambience and sound effects make the adventure feel more atmospheric.
• Progression and health: Game rules manage health, XP, progression, scene transitions, and session completion.
• Authentication: Clerk protects session and user routes.
• Resilient gameplay: A deterministic Fake DM keeps the core gameplay loop available even when an AI API key is not configured.

Why a fitness RPG?

Traditional fitness applications often focus on repetitions, timers, statistics, and progress dashboards. Those features are useful, but they don't necessarily make the workout itself more engaging.

Gamification adds a different layer. A workout can become part of a larger experience, where the player is responding to a challenge rather than merely following a list of exercises.

DungeonsDaily explores this idea through interactive storytelling. The Dungeon Master provides the narrative motivation, while the player still has to perform the physical activity.

The application is intended for people who enjoy fantasy RPGs, interactive stories, or gamified fitness experiences and want a more engaging way to structure short workouts.

The central principle behind the project is:

AI shouldn't just capture your attention. It should help you put the screen down and do something in the real world.

Demo

Live application: https://dungeonsdaily-frontend.onrender.com
Youtube Video Link::https://youtu.be/ISPRL0y4h58
The application is deployed on Render. Authentication and other environment-dependent services may require configuration before every part of the experience is available.

Source code: DungeonsDaily on GitHub

The project README documents the architecture, local setup, API endpoints, and the browser-native voice implementation.

Code

The application uses a modular client-server architecture. The frontend handles the adventure interface and voice interaction, while the backend manages sessions, game rules, AI integration, validation, and persistence.

System architecture

flowchart TD
    Player["Player"]

    subgraph Client["React + Vite Frontend"]
        UI["Adventure UI"]
        STT["Speech Recognition"]
        TTS["Speech Synthesis"]
        Audio["Ambient Audio + SFX"]
    end

    subgraph Backend["Express + TypeScript API"]
        Auth["Clerk Authentication"]
        Routes["Session API Routes"]
        Validation["Zod Validation"]
        Engine["Authoritative Game Engine"]
        DM["Dungeon Master Service"]
    end

    subgraph AI["AI Layer"]
        Gemma["Gemma via Vercel AI SDK"]
        FakeDM["Deterministic Fake DM"]
    end

    Storage["MongoDB Atlas / In-Memory Store"]
    Monitoring["Optional Sentry Monitoring"]

    Player --> UI
    UI <--> STT
    UI --> TTS
    UI --> Audio
    STT --> Routes
    UI <--> Auth
    Auth --> Routes
    Routes --> Validation
    Validation --> Engine
    Engine --> DM
    DM --> Gemma
    DM -. Fallback .-> FakeDM
    Engine <--> Storage
    Backend -. Errors and Tracing .-> Monitoring
    DM --> UI

The most important architectural decision

One of the key decisions in DungeonsDaily is separating narrative intelligence from game authority.

An AI model is good at generating descriptions, creating atmosphere, and responding naturally to a player's words. However, a language model should not be the final authority on whether an exercise was completed, how much XP was awarded, or whether a session has ended.

For that reason, DungeonsDaily uses a deterministic game engine as the authoritative rules layer.

The model contributes narrative output. The engine validates game transitions before they are persisted.

For example, if the player is facing a particular exercise, the backend checks that the submitted exercise ID matches the current challenge before accepting the completion request. Progression and session status remain controlled by the application rather than by the model's claims.

This separation makes the game more predictable and gives us a clearer boundary between AI-generated content and application state.

Example: the session API

The backend exposes a versioned REST API for session management.

Request validation is handled with Zod, and errors follow a consistent structure:

{
  "error": {
    "code": "SESSION_NOT_FOUND",
    "message": "Session not found"
  }
}

The message shown here illustrates the documented error format; exact error messages depend on the condition being handled.

How I Built It

DungeonsDaily uses TypeScript across a Node.js backend and a React frontend. The repository is organized with pnpm workspaces and Turborepo, allowing the frontend and backend to be developed and built together.

Here is the core technology stack:

1. Using Gemma as the Dungeon Master

The AI component is one of the defining parts of DungeonsDaily.

Rather than using an AI model only to answer questions, I use Gemma as a narrative layer inside an interactive application.

The backend integrates Gemma through the Vercel AI SDK and Google's provider package, @ai-sdk/google. The model is configured through the GEMMA_MODEL environment variable, and the Google API key stays on the server.

This separation means the frontend never needs direct access to the AI provider credentials.

The Dungeon Master service receives the player's response and relevant session context, then generates the next narrative turn. That turn is returned to the frontend, where it can be displayed and spoken aloud.

The README documents generateText and Output.object as part of the model integration.

2. Keeping the game playable without an AI key

An interesting engineering challenge is that a hosted AI service introduces an external dependency.

API credentials might be missing during local development, a provider request might fail, or a developer might simply want to work on the game loop without making AI calls.

To handle this, DungeonsDaily includes a deterministic Fake DM.

When the hosted model is unavailable or not configured, the fallback keeps the core loop usable. This allows development and testing to continue without making the entire game dependent on a live AI request.

The fallback is not intended to replace the richer narrative capabilities of Gemma. Instead, it provides a predictable baseline for the rest of the application.

3. Building voice interaction without a paid speech service

I wanted players to interact with the Dungeon Master through their voice without making server-side text-to-speech a requirement.

DungeonsDaily therefore uses browser-native speech APIs.

Speech-to-text (STT) uses the browser's SpeechRecognition or webkitSpeechRecognition implementation. Final transcripts are submitted to the same session-response endpoint used by typed input.

Text-to-speech (TTS) uses window.speechSynthesis and SpeechSynthesisUtterance to read the Dungeon Master's narration aloud. Voice selection and speech rate, pitch, and volume can be configured.

This keeps the architecture relatively simple:

1. The player speaks into the microphone.
2. Browser speech recognition produces a transcript.
3. The transcript is sent to the session API.
4. The backend processes the response and generates the next turn.
5. The frontend displays and speaks the narration.

Typed input remains available if speech recognition is unsupported or microphone permission is denied.

An important limitation is that browser speech recognition support and behavior vary by browser. Using a browser-native API does not guarantee that all speech processing happens locally or offline.

The project disables server-side TTS. It does not send or store audio recordings on the backend as part of this voice flow.

4. Adding ambience without making audio a dependency

Fantasy storytelling becomes more immersive when the environment has a sound of its own.

DungeonsDaily uses the Web Audio API to manage environmental ambience and sound effects in the browser. Ambient audio can be ducked while the Dungeon Master is speaking, reducing competition between narration and background sound.

Audio is deliberately treated as an enhancement rather than a critical dependency. If audio fails or the player mutes it, the gameplay loop should remain usable.

This is important for a workout application: sound should improve the experience without preventing someone from completing the adventure.

5. Persisting sessions

DungeonsDaily supports MongoDB Atlas through the official MongoDB Node.js driver.

The backend stores users, sessions, and recent turns, making session history available beyond a single browser interaction. For local development, an in-memory store provides an alternative when a MongoDB connection string is not configured.

This makes it easier to work on the core mechanics before setting up every external service.

The two storage modes have different purposes: in-memory storage is useful for development, while MongoDB provides persistent storage across application restarts.

6. Authentication and operational visibility

Clerk protects session and user routes, while the health endpoint remains publicly accessible for deployment checks.

Zod validates incoming data, centralized error handling provides consistent API responses, and structured request logging helps with debugging. Optional Sentry integration adds error reporting and AI tracing when SENTRY_DSN is configured.

These components may not be the most visible part of the application, but they matter when moving from a local prototype to a deployed service.

7. Running the project locally

To get started, configure the environment variables in the repository root. The main optional integrations documented in the README include MongoDB Atlas and Google's API key for Gemma.

Install dependencies and start the development environment:

pnpm install
pnpm dev

The README documents these local endpoints:

• Frontend: http://localhost:5173
• API: http://localhost:4000
• Health check: http://localhost:4000/api/v1/health

Without the hosted AI key, the deterministic Fake DM keeps the core gameplay loop available. Add the Google API key to enable Gemma narration.

Why Does Open Innovation Matter?

For DungeonsDaily, open innovation matters because it lets the project explore a different relationship between AI and software.

I did not want the application to be just another chat interface wrapped around a proprietary model. I wanted AI to be one component in a larger system, with clearly defined responsibilities and rules.

Gemma provides the narrative intelligence, while ordinary application code controls exercise validation, progression, persistence, and session lifecycle.

That division is useful regardless of which compatible model is used in the future. The narrative provider can evolve without giving up the deterministic game engine.

Using an open-weight model also makes the model choice more transparent and gives developers more flexibility to experiment with different deployment and inference approaches, subject to the chosen model's license and available infrastructure.

There is an important distinction here: DungeonsDaily currently uses Gemma through a hosted Google API integration. The architecture does not imply that the current deployment runs Gemma locally, and the project does not claim to have trained or fine-tuned the model.

Open-weight models can make local inference, customization, and alternative deployment strategies possible, but those options require additional engineering and compute resources.

The project also demonstrates that an AI-powered experience does not need to depend on AI for every operation. A deterministic fallback keeps the gameplay loop available when the model provider is not configured, and browser-native voice APIs reduce the need for a separate paid speech service.

This is the kind of flexibility I value in open innovation: the freedom to combine components, experiment with alternatives, and retain control over the system's most important rules.

The bigger idea: AI that encourages real-world action

The most interesting part of DungeonsDaily is not simply that an AI can narrate a fantasy adventure.

It is that the AI is being used to support an activity that takes place away from the screen.

A player interacts with the application to start an adventure and receive guidance, but the intended outcome is physical movement. The technology becomes the interface to the experience rather than the experience itself.

That is what makes the project a good fit for the Touch Grass theme.

What's Next?

DungeonsDaily is a starting point for exploring voice-first, AI-driven fitness experiences.

Potential next steps include improving the variety of generated encounters, refining the transition between narration and exercise challenges, testing speech recognition across browsers, and making the deployed authentication and session flow more robust.

I also want to keep the central design principle intact as the project evolves: AI should make the experience more engaging, but it should not take control of the rules that make the game reliable.

If you're interested in AI applications, browser-native voice interfaces, deterministic game engines, or open-weight models, I'd love to hear what you think.

The dungeon is waiting. Get off the screen and start moving.

Prize Categories

Best use of Render,
Best use of Sentry agent tracking,
Best use of MongoDb Atlas,
Best use of Gemma