My goal is to map out how users interact with the virtual Cafe menu and how the user interface looks.

In this simple scenario, I combine several universal gestures and user interaction paradigms for smart glasses + smart wirst band.

Select a module and drag it to a different position.

Select a button.

Pull the screen closer to make it look larger.

Push the screen backward to make it look smaller.

Slide the screen or a section in the screen horizontally.

Slide the screen or a section in the screen vertically.

This demo was built in Figma with DraftXR plugin.

I created two versions of mock up in figma, dark mode and light mode.

This was my test in Spline(3D software), trying to demo the 3D version of the design.

The tech stack I applied is Unity + Oculus Interaction SDK hand tracking. I rebuilt the 3D version UI in Unity Engine, and using OVR hand tracking to realize a short demo. This video was recorded in Oculus Quest.

Moving from 2D to 3D prototypes pushes me to design 3D interactions in depth. This 10 seconds demo costs me more then 10 hours to figure out several technical obstacles and countless testing and experimenting, such as

• Build AR experience in VR environment or with a background of my bedroom?
• What tech stack to use? XR Interaction toolkit or MRTK or ARkit? Should I apply hand tracking?
• How to transfer the design in Figma into Unity? Recreate in ShapeXR or build UI Canvas in Unity or apply MRTK Figma Bridge plugin?
• How to define the right scale, position and angle of the 3D menu in virtual environment?
• How to make text and icons clearer and readable without losing the original design?
• How to realize the hand interaction gesture mock up in Unity? Learn programming?:):(
• ......
• There is NEVER an end to keep modifying and refining the design.

I also did a version with XR Interaction toolkit, but later on I find OVR hand tracking performing more intuitively for this demo.

Import Oculus XR Plugin and other packages into the project.

Set and adjust the right position and angle in the scene between Camera rig and 3D menu.

Build up a basic interaction flow using Unity's Event and OVR' sample components.

Animate buttons and 3D models in Unity.

Import external 3D assets and modify materials. 3D modeling by Megan Alcock and Okotaru.

Style Guide

Persona & Problem Statement