This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

The next chapter is outside

Most digital stories ask us to sit still.

We read, tap “next,” watch another animation, and stay exactly where we are. When I saw the Touch Grass prompt, I wanted to reverse that relationship. What if a story could begin on a screen, send you outdoors, and refuse to continue until you came back with something you had discovered?

That idea became Outside the Lines: a Python and Streamlit story experience in which the player chooses a comic adventure, reads one short chapter, receives a real-world mission, and photographs something they find outside. An open-weight vision model interprets the discovery, and the object becomes part of the next chapter.

The screen is not the destination. It is the doorway.

What I Built

Outside the Lines is an interactive outdoor storybook for children, families, and anyone who needs a playful excuse to step away from a screen.

The player chooses one of three stories:

• Barnaby’s Great Bush Disaster — an overconfident orange cat attempts an heroic squirrel chase and discovers gravity.
• Leo and the Lunch-Lover — a peaceful mountain picnic becomes a very fast argument with a hungry goat.
• Maya’s Flying Masterpiece — a gust of wind turns an artist’s canvas into a kite and the hillside into a rescue mission.

Each story has three chapters and three outdoor missions. A mission might ask the player to find a leaf that looks brave, a natural shape that points toward an escape route, or a color that belongs in Maya’s recovered painting.

The interaction loop is intentionally simple:

Choose a story
      ↓
Watch and listen to a short chapter
      ↓
Receive a safe outdoor mission
      ↓
Step away from the screen
      ↓
Photograph a natural object
      ↓
OpenCLIP suggests what it might be
      ↓
The player confirms or changes the answer
      ↓
That discovery appears in the next chapter

The most important product decision was that the AI does not replace the player’s imagination. It gives a suggestion, but the player can edit it. If the model thinks a seed pod is a stone, the child does not have to accept the machine’s answer. The discovery still belongs to the person who found it.

The missions also avoid collecting, damaging, or feeding wildlife. The player can photograph an object where it is and leave it outside.

Demo

Try the Streamlit app here:

👉 Launch Outside the Lines

The first use of the vision or narration features can take longer because the open model weights need to be downloaded and cached. The story itself remains playable while those optional features load.

Deployment note: the Streamlit Community Cloud app must be set to public before submitting, otherwise visitors will see a sign-in page.

Code

The complete Python project is open source:

cd outside-the-lines
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/streamlit run app.py

Repository: github.com/nikisha77/outside-the-lines

The project is deliberately small enough to understand:

outside-the-lines/
├── app.py              # Streamlit UI, navigation, and session state
├── story_data.py       # Three stories and nine outdoor missions
├── ai_services.py      # Animation, vision, and narration
├── requirements.txt    # Reproducible Python dependencies
└── assets/             # Comic artwork and story panels

How I Built It

1. I started with the story loop, not the model

The first version was a static web story. It looked polished at a glance, but it had the wrong kind of motion: leaves floated, objects pulsed, and the page moved without the characters actually doing anything.

That exposed an important difference. An animated interface is not the same thing as an animated scene.

I rewrote the experience around character actions: Barnaby crouches, launches, and crashes; Leo and Billy race around the boulder; Maya chases a canvas down the hill. The movement needed to support the comedy and make each chapter feel like a moment in a comic, not a collection of decorative effects.

I also expanded the stories. Instead of a single shallow scene, each theme now has anticipation, action, impact, and an ending affected by what the player finds outside.

2. I moved the complete application to Python and Streamlit

I wanted one readable stack for the interface, story state, model inference, audio, and animation. Streamlit made that possible without maintaining a separate JavaScript application.

app.py controls the screens and stores the current story, chapter, discoveries, and audio state in Streamlit session state. story_data.py keeps the narrative separate from the UI, so a story can be edited, translated, or replaced without changing the application logic.

Expensive results are cached:

@st.cache_data(show_spinner=False)
def scene_bytes(panel: int, chapter: int) -> bytes:
    return animated_scene(panel, chapter)

This matters because model-backed interactions feel much better when rerunning the Streamlit script does not regenerate the same media every time.

3. OpenCLIP turns a trail photo into a story ingredient

For image understanding, I used OpenCLIP ViT-B/32 with LAION pretrained weights.

A mission supplies a small set of relevant labels such as “a leaf,” “a twig,” “a stone,” “tree bark,” or “a seed pod.” OpenCLIP embeds the uploaded image and each candidate label into the same feature space. The app then chooses the label with the highest similarity.

The core is compact:

model, _, preprocess = open_clip.create_model_and_transforms(
    "ViT-B-32",
    pretrained="laion2b_s34b_b79k",
)
tokenizer = open_clip.get_tokenizer("ViT-B-32")

image_features = model.encode_image(image)
text_features = model.encode_text(text)

image_features /= image_features.norm(dim=-1, keepdim=True)
text_features /= text_features.norm(dim=-1, keepdim=True)

scores = (100 * image_features @ text_features.T).softmax(dim=-1)
best_label = candidates[int(scores.argmax())]

I chose constrained candidate labels on purpose. This is not a general “tell me everything in this image” chatbot. It is a small, understandable classification step tied to the current mission. That makes it faster, easier to explain, and easier for the player to correct.

If the model cannot load, the app falls back to a neutral “outdoor discovery” instead of breaking the story. AI should enhance the adventure, not hold it hostage.

In the hosted version, the uploaded photo is processed by the Streamlit server. It is not sent to a separate proprietary vision API, and the app does not request location data. For the strongest privacy boundary, the same repository can be run locally so inference stays under the player’s control.

4. Kokoro gives the stories a warmer voice

The first narration attempt sounded like a utility reading a notification. That did not fit a mischievous cat, a terrified hiker, or a painter sprinting downhill.

I replaced it with Kokoro-82M, an open-weight text-to-speech model. Narration is produced as a WAV file and cached using a hash of the voice and story text:

pipeline = KPipeline(lang_code="a")
chunks = [
    audio
    for _, _, audio in pipeline(
        text,
        voice="af_heart",
        speed=0.95,
    )
]

audio = np.concatenate(chunks)
sf.write(cached_path, audio, 24000, format="WAV")

The slightly slower pace makes the voice feel more like story narration than a screen reader. Because the voice layer is independent, another Kokoro voice—or another open TTS model—can be substituted without rewriting the story system.

5. Pillow creates motion without a video-generation bill

The moving scenes are generated in Python with Pillow and NumPy. Each chapter uses a different camera rhythm:

• anticipation uses a gentle push-in,
• chase scenes pan and add motion arcs,
• impacts use a quick zoom, shake, and comic burst.

Twenty frames are rendered into a looping animated WebP. That is not full character animation, and I do not pretend it is. It is a lightweight comic-motion system that runs predictably, costs nothing per scene, and works with the supplied hand-drawn panels.

The architecture also leaves a clean seam for future work: generated or hand-animated frames can replace the source panel without changing the Streamlit story flow.

6. Deployment taught me to respect the Python version

The app worked locally, but the first Streamlit deployment failed while building blis, a dependency pulled in by spaCy. The cloud environment had selected Python 3.14, while the dependency chain expected a version with compatible wheels.

The fix was not an elaborate code change. I redeployed with Python 3.11 and kept the dependencies explicit in requirements.txt.

That debugging moment was a good reminder: an open-source project is only useful when another machine can actually install it. The model choice gets attention, but reproducible environments are part of the product.

Why Does Open Innovation Matter?

Open models are not decoration in this project. They are the parts that make the interaction possible.

The project does not pay for every discovery.

There is no per-photo vision request and no per-sentence narration bill. That matters for a playful app that may be used repeatedly by a family, classroom, or community group.

The models can be inspected and replaced.

OpenCLIP and Kokoro sit behind small Python functions. A developer can swap the vision backbone, change the label set, choose a different voice, or run quantized models on smaller hardware. The application is not designed around one company’s response format.

The local path is real.

The public demo runs on Streamlit Community Cloud for convenience, but the same code can run on a laptop without sending photos to a third-party AI API. Once weights are downloaded and cached, the model layer can be controlled by the person or organization running the app.

Failure does not end the story.

Open model weights can be large, networks can disappear, and low-memory machines exist. The narrative and missions continue even if vision or narration is unavailable. The human activity remains the primary feature.

The player stays in charge of meaning.

A model prediction is shown as an editable suggestion. This is especially important for a children’s experience: the software can notice patterns, but it should not teach the player that an AI label is unquestionable truth.

A closed multimodal API could probably produce a more fluent description with less setup. But it would also introduce a remote dependency, unpredictable usage cost, less control over model behavior, and a harder privacy conversation around children’s outdoor photographs. For this project, the flexibility of open components is a better fit than maximum model size.

What I Learned

The hardest part was not connecting a model. It was deciding where AI belonged.

My first instinct was to add more generation: more animation, more description, more automatic storytelling. The better version uses AI in two narrow places—recognizing a discovery and narrating a chapter—while the player supplies the movement, attention, and imagination.

I also learned that “get people outside” cannot be solved with outdoor-colored pixels. The product needs a reason to close the laptop or put down the phone. In Outside the Lines, progress pauses until the player looks at the real world.

There are still limitations. OpenCLIP chooses from a constrained list and can be wrong. Kokoro’s first load can be slow. The animated WebP scenes are expressive camera motion, not fully articulated character animation. Streamlit Community Cloud also means the hosted version is not an offline trail app.

Those are useful boundaries, not hidden defects. The next steps are to add a downloadable offline mode, more accessible mission alternatives, multilingual stories, and community-authored story packs.

The part I care about most

I do not want Outside the Lines to become another app that rewards a child for staring at it longer.

The ideal session is a few minutes of story, a much longer stretch of noticing leaves, stones, shadows, wind, bark, and paths, and then a quick return to see how that discovery changed the adventure.

If the player remembers the squirrel, the goat, or the flying canvas, that is lovely.

If they remember the strange leaf they found outside, that is the real ending.

Prize Categories

I am entering the overall Hacktoberfest Open-Source AI Challenge: Week 1 — Touch Grass category. The current build does not claim a partner-technology prize category.

Thanks for reading—and please close this tab for a minute when you are done. The next chapter is outside.