Client: JOVRNALISM
Tools and framework: Apple ARKit, Unity, C#, Depth-kit, Volumetric capture, 360 Video, UI/UX
Role: AR UX designer, Director (Volumetric capture & 360 video), Engineer (AR)
Team size: 15
--------
Exploring the potential of the immersive medium, the class of jovrnalism tell the story of Leimert Park - a historic and influential black neighbourhood of Los Angeles.
The class during the first visit to KAOS network listened to various stories of neighbourhoods ranging from jazz musician to revolutionaries of watt riots. Johnnie Raines who is an active stakeholder in the community shared his story around his house.
The class during the first visit to KAOS network listened to various stories of neighbourhoods ranging from jazz musician to revolutionaries of watt riots. Johnnie Raines who is an active stakeholder in the community shared his story around his house.
![](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/d06fe64b-b57b-4e7d-a89f-98e5f64e686c_rw_3840.jpeg?h=a308f39eae70159710e456c467329820)
JOVRNALISM class at KAOS network interacting with Leimert Park community
The storyline was designed collaboratively with engineers, storytellers, and journalists. The whole project developed as a part of JOVRNALISM class - a hackathon-style course where students from different backgrounds collaborate to decide how to apply emerging technology to a story/theme, often in partnership with a media company.
![Prof. Robert Hernandez supervising videogrammetry process](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/1e226aed-819a-46a8-9c2d-39576472c331_rw_1920.png?h=a20c5ab6b3efaf6c97cb352e4b361268)
Prof. Robert Hernandez supervising videogrammetry process
![Johnnie being captured volumetrically using Depth-kit](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/599a29d0-0edf-4de9-8a63-47a9cba6bff4_rw_1920.png?h=11178da435a2a10e0583c3fff78810c1)
Johnnie being captured volumetrically using Depth-kit
![Formulating the basics of the frame in the spatial domain](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/14dd820c-d132-4387-ac4c-46ed40d278a6_rw_1920.jpg?h=4821eec6d07f93823f3161002bb4dcb0)
Formulating the basics of the frame in the spatial domain
![Raja interviewing Johnnie](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/8e9b958b-8385-42c4-853e-054e090a7f83_rw_1200.png?h=94d31e87bd079a4fd8b6f5bd13c8cdb2)
Raja interviewing Johnnie
![Ideas exploring interactions to start the experience](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/14567b0f-e5c9-48e0-a220-6ee73300f7da_rw_1920.jpg?h=2adcc1af1bcfdfd0dcbb803c6e7da299)
Ideas exploring interactions to start the experience
![AR app - portal design using Unity game engine](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/02e26253-3ab7-4e3a-9b24-51752edba003_rw_3840.jpeg?h=f0d4babcfaf3723c40bcd73c99a8a9ca)
AR app - portal design using Unity game engine
![Idea sketches](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/686ac731-9e40-4f6e-8dad-2a8360624c45_rw_1920.jpg?h=fc97ccc836f3a6b5025bbb8a5657bda2)
Idea sketches
![Loading page to initiate AR](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/a639b8b8-00de-41fe-8574-7bbdfe6ce642_rw_1920.png?h=b9bfaf07b86964e11bb77fa051e80cfa)
Loading page to initiate AR
![The interaction required to begin the experience](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/0d1d8f7b-1560-4102-8637-d077fb44230d_rw_1920.png?h=7883b6367f741132eb14cd78beff8289)
The interaction required to begin the experience
![Start of experience. Closed-door](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/984569ce-13d8-48a2-aa98-0065b310859c_rw_1920.png?h=e2a02ad719b6c270b9939d4177400833)
Start of experience. Closed-door
![The door opens up inviting the user to enter the portal](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/4a0c2136-b287-43d0-b4d6-fd09d5fe5935_rw_1920.png?h=9aa468d480a944ea31f7a090f5c7f4e0)
The door opens up inviting the user to enter the portal
![Guidelines leading to the exit portal](https://cdn.myportfolio.com/48942a696a8df1c9c0c280079bc6645c/c9956c6d-5d01-43d7-871e-90279dd8e667_rw_1920.png?h=2c485b29a8e4f9f278929e3d7bd9808a)
Guidelines leading to the exit portal
Using cutting-edge and innovative techniques, the class created an augmented reality experience of the story. For this, I led the creative engineering design aspect of the project which included responsibilities like videogrammetry for volumetric human capture, AR application development and UX design for AR. The development process used Microsoft's Kinect sensor connected with Depth-kit software for videogrammetry and unity game engine along with apple ARKit for AR experience development.
Initial test of videogrammetry in AR
Initial design of object placement and door interaction in AR