Immersive WEb Case Studies

A 3d Immersive WEb Experience to enhance storytelling and participation in frog's product design and experience design case studies

Project Role: Software Developer / Creative Technologist

Technologies: React, React Three Fiber, Three.js, MediaPipe, Tensorflow, AWS

Articles/Demo: This will be featured on frog’s website soon.

As part of an internal initiative to showcase our work in immersive experiences and extended reality, I worked with several developers and designers to convert frog case studies into 3D web experiences and explore natural user interactions with applied machine learning in MediaPipe and Tensorflow via a React web tech stack. This allows for more effective storytelling and a wider audience outreach. Initially, we focused on converting product and service design for Dispatch Goods to a 3D web experience, and then built out a web version of the SFMOMA x frog interactive Magritte gallery.

Specifically, I explored MediaPipe and Tensorflow for pose (hand and body) and face detection as possibilities for user interaction and control in spatial environments. I also conducted preliminary research, iterated, and developed code for many low-to-high fidelity prototypes of the immersive web experience. We collectively presented our work to colleagues, and a demo of the case study experiences will soon be available on frog’s website.