Participatory design sessions traditionally rely on sticky notes, whiteboards, and quick sketches so stakeholders externalize ideas together. Participants with motor disabilities, tremor, or visual impairments often cannot sketch fast enough to keep pace, shrinking their influence on outcomes. Speak2Scene, published in SoftwareX in 2026 (doi:10.1016/j.softx.2026.102628), is an open-source ReactJS application that lets people build storyboards by voice: spoken scene descriptions flow through GPT-4o for structuring and gpt-image-1 for panel imagery, supporting tell-make-enact cycles in speculative co-design. Seven participants explored future service scenarios in a facilitated study, enacting storyboard sequences to test narrative coherence. Teams evaluating AI image generator tools for workshops or popular AI tools for facilitation should understand how voice storyboarding changes power dynamics before replacing analog methods entirely.
Tell-Make-Enact Participatory Design
Tell-make-enact extends classic participatory design by having participants narrate scenarios (tell), generate visual storyboard panels (make), and physically or verbally role-play sequences (enact) to surface tacit assumptions. Hand sketching bottlenecks the make phase when some attendees cannot draw or type quickly. Speak2Scene lowers the make barrier: speech captures intent, large language models segment narration into scenes with characters and settings, and image models render panels for collective critique. Enactment then reveals whether generated visuals match lived experience or stereotype, prompting iterative retelling.
Facilitators remain responsible for inclusion norms: dominant speakers must not monopolize voice capture, and multilingual groups need transcription accuracy checks. The SoftwareX paper documents facilitator scripts that alternate turns and confirm scene summaries before image generation, preventing misrepresentation from rushed prompts.
GPT-4o and gpt-image-1 Pipeline
Speak2Scene pipes transcribed speech through GPT-4o to extract scene titles, character lists, setting details, and emotional tone, then calls gpt-image-1 with structured prompts sized for storyboard panels. Separating language structuring from image synthesis reduces garbled prompts and lets facilitators edit JSON-like scene objects before spending image credits. Participants can regenerate individual panels after changing a spoken detail without rebuilding the entire board. Open-source ReactJS components expose hooks for swapping model endpoints, important for teams with data residency constraints.
| Stage | Model or tool | Participant control |
|---|---|---|
| Tell | Speech transcription | Review transcript before scene extraction |
| Structure | GPT-4o scene parsing | Edit characters, settings, and beats |
| Make | gpt-image-1 panel render | Regenerate single panels, adjust prompts |
| Enact | Facilitated role-play | Reject scenes that misrepresent lived experience |
Open-Source ReactJS Architecture
The Speak2Scene repository modularizes voice capture, scene graph state, and panel gallery views so institutions can self-host workshops without proprietary whiteboarding subscriptions. React state stores ordered scenes with version history, supporting undo when image generation misinterprets adjectives. Facilitators project the gallery on shared displays while participants contribute from personal devices with headsets in noisy rooms. API keys for OpenAI services remain organizer-controlled; participants need not create accounts, lowering GDPR friction for community sessions.
Developers extending speak2scene voice storyboarding workflows can plug local open-weight image models for air-gapped civic labs. The paper emphasizes adapter interfaces rather than hard-coded vendor SDKs. Accessibility reviewers should verify keyboard paths for non-speaking participants who still need to edit scene text manually.
Seven-Participant Speculative Co-Design Study
Seven participants joined speculative co-design workshops envisioning future mental health support services, producing multi-panel storyboards entirely through voice-driven generation and enacting key scenes aloud. Sessions compared Speak2Scene to paper storyboards on inclusivity metrics: speaking time equity, panel revision counts, and self-reported agency. Voice storyboarding increased successful contributions from participants who previously declined to sketch. Enactment phases caught unrealistic service tropes, such as omniscient AI therapists, leading to revised tell-make cycles within the same hour-long slot.
Image model bias appeared when gpt-image-1 defaulted to youthful able-bodied characters; facilitators prompted explicit demographic attributes in GPT-4o scene objects to correct representation. Participants noted that generated panels stimulated discussion even when stylistically imperfect, similar to rough sticky-note sketches. Perfectionism slowed analog groups more than AI groups because regeneration was one command away.
When to Use Voice Storyboarding
Voice storyboarding suits early-phase ideation with mixed-ability groups, remote workshops, and speculative futures where visual polish matters less than narrative diversity. It fits poorly when legal discovery requires immutable handwritten artifacts or when communities distrust cloud AI processing of sensitive stories. Facilitators should obtain informed consent describing transcription and image generation destinations. Hybrid formats can reserve analog sketching for those who prefer drawing while others use Speak2Scene in parallel, merging boards during synthesis.
UX researchers documenting service blueprints can export scene JSON and panel images into reports, maintaining traceability from spoken quotes to visual outputs. Compared to static AI image generator demos in isolation, Speak2Scene embeds generation inside facilitated critique, reducing the risk of treating single images as finalized design decisions.
Facilitation Patterns for Mixed-Ability Rooms
Successful Speak2Scene workshops begin with consent scripts explaining voice recording, cloud model processing, and participant rights to delete scenes. Facilitators establish turn order, offer text-entry alternatives, and project live transcripts so deaf participants and non-native speakers follow narration. Noise-canceling headsets reduce transcription errors in community centers. After each tell segment, the group confirms GPT-4o scene summaries on screen before gpt-image-1 renders panels, preventing misattributed dialogue from becoming visual canon.
Power dynamics do not disappear because technology is novel. Dominant stakeholders may steer scene content unless facilitators actively invite marginalized voices and pause generation for reflection. The seven-participant SoftwareX study used speculative futures precisely to avoid locking designs to current vendor roadmaps, giving participants freedom to imagine services without immediate budget constraints. Enactment phases surface ethical discomfort: when generated therapists appear too human-like, groups discuss deception and consent boundaries before continuing.
Remote and Hybrid Co-Design
Open-source ReactJS deployment enables hybrid workshops where remote participants contribute voice scenes while in-room attendees enact panels projected on shared displays. WebRTC voice channels feed the same scene graph, versioning conflicts when two participants edit simultaneously. Facilitators merge branches during synthesis breaks. Time zone spread requires asynchronous tell phases: participants record overnight, structuring happens in morning review, and enactment convenes in shared evening slots. speak2scene voice storyboarding adapts to civic engagement when town halls cannot gather everyone physically.
Data residency concerns push some municipalities toward local transcription models and open-weight image backends plugged into Speak2Scene adapters. The SoftwareX paper documents API-swappable architecture for this reason. Security reviews should cover whether scene narratives contain sensitive health or immigration stories before cloud upload. On-premise deployments trade convenience for privacy, a worthwhile exchange for trauma-informed service design with vulnerable populations.
Measuring Deliberative Quality, Not Only Speed
Metrics beyond panel count include revision depth, enactment disagreement episodes, and post-workshop agency scores captured in validated surveys. Faster storyboarding means little if outcomes ignore participants who could not attend follow-up sessions. Speak2Scene teams should archive scene JSON, transcripts, and facilitator notes together for longitudinal accountability. UX researchers comparing Miro sticky sessions report higher speaking equity when voice capture removed sketch anxiety, but facilitation skill remained the strongest predictor of inclusive outcomes.
Integration with downstream service blueprints requires export formats product teams actually ingest: Figma frames, PDF storyboards, or agile user story tickets. Speak2Scene panel images alone do not replace written acceptance criteria; GPT-4o can generate companion user stories from enacted dialogue if facilitators prompt structured extraction during debrief. Linking generative visuals to testable requirements prevents participatory theater where workshops generate images that never influence roadmaps.
Frequently Asked Questions
What is Speak2Scene?
Speak2Scene is an open-source ReactJS tool for voice-based AI storyboarding in participatory design, using GPT-4o and gpt-image-1. It was published in SoftwareX 2026 (doi:10.1016/j.softx.2026.102628).
What is tell-make-enact?
Participants tell scenarios aloud, make visual storyboard panels with AI, then enact scenes to test realism and ethical implications before iterating.
Do participants need OpenAI accounts?
No. Organizers configure API access on the hosted React app. Participants contribute voice or text edits through the workshop interface.
Can non-speaking users participate?
Manual scene editing and keyboard navigation should be provided alongside voice capture. Facilitators must not design voice-only pathways.
How many people were studied?
Seven participants joined speculative co-design sessions documented in the SoftwareX paper, with metrics on inclusion and narrative revision.
Is Speak2Scene open source?
Yes. The paper describes a ReactJS implementation intended for extension and self-hosting. Check the SoftwareX supplementary materials for repository links.
Civic technology labs running climate adaptation workshops can adapt Speak2Scene to narrate neighborhood flood futures, enacting response scenes with residents who cannot sketch inundation maps quickly. Documenting which panel prompts came from which speaker preserves accountability in downstream planning reports.
Model cost management matters for multi-hour sessions: GPT-4o structuring is inexpensive relative to image regeneration; facilitators should batch approve scenes before rendering full boards. speak2scene voice storyboarding budgets improve when participants rehearse tell segments offline, reducing throwaway generations.
Comparative studies against Miro or FigJam should measure deliberative quality, not only speed. Speak2Scene participants in the SoftwareX evaluation reported higher agency when enactment followed generation, suggesting facilitation training is as important as the popular AI tools stack chosen for the make phase.
Institutional memory improves when organizations store scene graphs alongside policy outcomes years later. Planners revisiting mental health service redesigns can compare 2026 storyboards with implemented programs, auditing whether enacted concerns about data privacy materialized in production systems. Voice storyboarding archives become qualitative evidence complementing quantitative service metrics.
Accessibility consultants booking co-design workshops should pilot Speak2Scene in low-stakes internal sessions before facilitating with external stakeholders, learning transcription failure modes and image bias patterns without reputational risk. Facilitator playbooks in the SoftwareX repository document timing budgets: roughly ten minutes per tell-make cycle for a three-panel board, plus fifteen minutes for enactment and critique.
Cross-cultural co-design demands explicit prompts for setting, dress, and architecture so gpt-image-1 does not default to Western suburban aesthetics when participants describe services in global south contexts. GPT-4o scene objects should capture locale metadata facilitators review aloud before rendering, aligning visuals with community self-image rather than model priors.
University design programs teaching participatory methods should assign Speak2Scene readings alongside classic Sanders and Stappers texts, connecting tell-make-enact theory to contemporary GenAI tooling students will encounter in civic and health service internships after graduation.
NGO field teams conducting participatory rural development workshops can load Speak2Scene on offline-capable laptops when connectivity is intermittent, queuing tell segments locally and syncing scene structures when satellite links return, preserving inclusion for participants who cannot sketch infrastructure proposals on paper maps.
Ethics review boards evaluating GenAI in human subjects research should ask whether storyboard images could identify participants or stigmatize communities, requiring deletion workflows in the open-source React app when consent withdrawals occur after workshops conclude.
Municipal innovation labs comparing Speak2Scene to traditional charrette formats should document participant demographics before and after adoption, verifying that voice storyboarding increases representation from disability communities rather than only accelerating output for stakeholders who already dominated whiteboard sessions. Without that demographic audit, voice tools risk automating exclusion at higher frame rates than analog workshops ever achieved.
