Mingyong Cheng
The Silhouette Seeker
The machine's view of the gesture: tracked hand landmarks rendered as constellations of light.
The Silhouette Seeker (2026) is an AI-powered embodied storytelling experience where players shape a living narrative through hand gestures and shadows. Instead of conventional interfaces, participants communicate with the system using silhouettes formed by their own hands — symbolic acts that invite interpretation rather than issue fixed commands.
The project takes inspiration from a simple perceptual phenomenon: a hand shadow is only a rough outline, yet people immediately recognize animals, characters, or spirits within it. This gap between shape and meaning becomes the foundation of a new storytelling language, where human imagination and machine interpretation collaborate to create story.
Each turn opens with three paths: two named silhouettes and one open to interpretation.
Players enter the world as a Silhouette Seeker, someone who once understood the language of shadows but now remembers it only partially. After an undefined disturbance fragmented the world, landscapes, communities, and ecological systems remain present but broken, as if parts of their stories have been erased. Through gesture and experimentation, the seeker attempts to restore relationships between places, creatures, and forces that no longer communicate clearly.
The Custom Shadow Oracle — hold any hand form steady, and the system reads it as a creature.
Built as a web-based system, the piece combines gesture recognition, generative narrative models, and real-time media synthesis. Hand shadows are detected through camera-based tracking; a language model generates narrative, companion dialogue, and encounters; and image generation produces stylized black-and-white line illustrations that function as visual keyframes — creating a picture-book-like experience where text, image, and gesture operate together. Because interpretation is probabilistic rather than deterministic, the same gesture may lead to different outcomes in different situations, and each playthrough becomes a unique unfolding of intention, interpretation, and consequence.
System architecture: gesture recognition, shadow oracle interpretation, generative narrative, and image synthesis, with fallbacks at every layer.
A companion character accompanies the player throughout the journey — not controlling the narrative, but acting as witness, interpreter, and recorder. When a run ends, it may be transformed into a fog point on an evolving map, marking a place where a player attempted to intervene in a dilemma and where the world responded with a particular configuration of consequences. These nodes accumulate over time, forming a collective history of interpretations and traces left by different participants.
Every ending becomes a fog point: a place where one player's shadow changed the shape of the world.
Mingyong Cheng (she or they) is an interdisciplinary artist, researcher, and creative technologist from Beijing, now based in California, working at the intersection of generative artificial intelligence, computational media, and environmental research. She holds dual BFAs from the Communication University of China (CUC), an MFA in Experimental and Documentary Arts from Duke University, and is completing a PhD in Art Practice and Art History at the University of California San Diego (UC San Diego), with a concentration in interdisciplinary environmental studies.
Her background spans visual arts, filmmaking, and computational media. She creates immersive installations, interactive environments, and real time audiovisual systems that integrate generative AI, sensing technologies, and spatial media. She approaches AI as a creative collaborator that shapes perception, narrative, and experience, and works closely with curators, choreographers, composers, and researchers to design and deploy complex public facing media systems. She has led commissioned projects for the San Diego Museum of Art (SDMA), the Museum of Photographic Arts at SDMA (MOPA), and Jacob's Pillow Doris Duke Theater (Jacob's Pillow Dance Festival), overseeing full system design, real time generative workflows, and onsite installation.
Grounded in a framework she called Speculative Ecology, her research examines how generative AI engages ecological imagination, cultural memory, and embodied experience. Her work has been presented internationally at ACM SIGGRAPH, SIGGRAPH Asia, Creativity and Cognition (ACM C&C), NeurIPS, ISEA, and the IEEE CVPR AI Art Gallery. Her honors include the Gold Muse Design Award in Artificial Intelligence, the Speculative Futures Digital Arts Student Competition award, and the IEEE TCPAMI Art Award presented by the CVPR AI Art Gallery.