DEV Community

Nandini Patel
Nandini Patel

Posted on

WorldCraft — “Capture the world. Create your own.”

#devchallenge #hf26challenge

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

WORLDCRAFT — “Capture the world. Create your own.”

What I Built

Most modern creative apps encourage people to sit in front of a keyboard typing text prompts into black-box generators. WORLDCRAFT flips that paradigm on its head to celebrate the spirit of "Touch Grass": it turns the physical outdoors into your creative palette.

WORLDCRAFT is a creative web application with the tagline:

“Capture the world. Create your own.”

Instead of generating synthetic images with AI prompts, users are motivated to step outside, walk through parks, explore backyards, and interact with the real world. Users photograph real-world natural objects—such as wild daisies, oak bark, mossy rocks, garden ferns, and birds—and import them into an interactive retro 8-bit canvas to arrange, scale, rotate, layer, and compose their own personalized, living worlds.

Who is it for?

  • Outdoor explorers & hikers who want a fun, interactive digital diary of the flora and fauna they discover.
  • Children & students learning biology, ecology, and environmental science through hands-on discovery.
  • Creative digital crafters who love retro arcade aesthetics and collage-style world-building.

Demo

Code

Explore the complete source code on GitHub:
👉 (https://github.com/Nobody-377/worldcraft)

How I Built It

WORLDCRAFT was designed around a clean, modern web stack paired with client-side open-source machine learning:

1. Open-Source AI Vision (Client-Side Local Inference)

  • Model: Quantized MobileNet-v1 (Xenova/mobilenet_v1_1.0_224, Apache-2.0 License).
  • Inference Runtime: Powered by @xenova/transformers (Hugging Face Transformers.js), running ONNX weights directly in the client's browser through WebAssembly.
  • Asynchronous & Non-Blocking: Classification runs in the background. As soon as a photo is snapped or uploaded, it appears immediately on the screen while the model calculates genuine softmax probability scores (e.g., AI: Daisy (89%)), seamlessly updating the label without freezing the UI.
  • Offline Fallback: If internet access is unavailable on remote hiking trails, the app gracefully falls back to smart filename normalization and provides an inline renaming tool.

2. Interactive 2D Canvas Engine (react-konva)

  • Hardware-accelerated HTML5 Canvas with SSR hydration protection (ssr: false).
  • Full multi-touch and mouse transformations: drag, aspect-ratio locked resizing handles (keepRatio={true}), and 360° rotation anchors.
  • Layer-depth ordering controls (Bring to Front, Bring Forward, Send Backward, Send to Back).
  • Single-click 2x high-resolution PNG world export excluding transform handles.

3. Local-First Persistence & Safe Memory Management

  • IndexedDB via idb-keyval: Stores full-resolution raw photographic binary blobs locally on the device, bypassing the restrictive 5MB localStorage limit.
  • Reference-Counted Object URLs: Prevents broken images when multiple canvas elements share the same inventory photo, safely releasing memory upon unmount.
  • Debounced Auto-Save: Background timer (1,000ms debounce) preserves layouts, coordinates, and unplaced inventory photos across browser reloads.

4. Retro Hacktoberfest 2026 Visual Design

  • Styled with the official challenge aesthetic: Deep Teal (#304C48), Forest Teal (#3D5E59), Retro Cream (#E8E8DA), Arcade Yellow (#F5BD2D), Pixel Coral (#E98083), and Pixel Blue (#8BB4DE).
  • Tactile 8-bit offset hard shadows (box-shadow: 3px 3px 0px #171D1B) with active button depression.
  • Mobile-first responsiveness tested from 320px small screens to desktop with 44px minimum touch targets and zero horizontal overflow.

Why Does Open Innovation Matter?

Closed, proprietary vision APIs (such as cloud-hosted vision services) come with substantial drawbacks:

  1. Surveillance & Privacy Concerns: Closed cloud APIs require uploading personal photographs of a user’s private spaces, backyards, or neighborhoods to corporate servers.
  2. Paywalls & API Key Gatekeeping: Closed models require billing accounts, rate limits, and per-query charges.
  3. The "Outdoors" Disconnect: Most crucially for a "Touch Grass" challenge, closed APIs fail the moment you lose cellular connection on a hiking trail or in a national park.

Open innovation—exemplified by open-weight models like MobileNet and open-source runtimes like Transformers.js—democratizes artificial intelligence. It enables privacy-preserving, zero-cost, client-side inference that works offline anywhere on Earth. Open models make it possible for anyone with a browser to explore nature, identify biodiversity, and build art without handing their data to closed walled gardens.

My Agent Session

This project was built following a rigorous, controlled human-in-the-loop pair-programming methodology using Google DeepMind's Antigravity AI Agent:

  • Phase 0: Architecture planning and tech stack selection.
  • Phase 1: Scaffolding Next.js App Router, Tailwind CSS, and core layout shell.
  • Phase 2: IndexedDB storage pipeline and base Konva stage rendering.
  • Phase 3: Transformer manipulation, aspect-ratio locks, and layer ordering.
  • Phase 4: Debounced layout auto-save, reload rehydration, and 2x PNG export.
  • Phase 5: In-app camera integration and Transformers.js open-source AI vision.
  • Phase 6: Retro Hacktoberfest 2026 theme overhaul and mobile-first 320px–desktop responsive verification.

Prize Categories

  • Touch Grass Challenge (Week 1 Winner)
  • Most Creative Open-Source AI Project
  • Best Use of Local / Client-Side Inference

Top comments (0)