A versatile way to communicate emotions, emojis, and compressed ideas in virtual reality
How might we establish a more abstract, expressive, and personalized form of communication in an extended reality environment?
In my undergraduate design course that examines the future of interaction design in mixed reality, our group was tasked with exploring how everyday interactions with mobile devices can be transformed into magical/delightful extended reality (XR) experiences.

Inspired by Gen-Z culture and emojis, we designed a gestural menu in VR that allows people to create, store, and access fun visual expressions and objects, and use them to communicate with others in virtual realty.
My Role
My role mainly involved using Unity to prototype and design VR interactions using VR toolkits, scripts, and particle animations. I also contributed heavily to storytelling and video prototyping.
Ariel Chiang, Daniel Kim, Alvin Jeong, Jasper Xie, Andrew Tang
Unity 3D, Oculus
Through our initial exploration of communication in XR environments, we chose to investigate how typing in VR could benefit from different physical gestures. After trying VR typing ourselves, we learned that aiming and clicking on individual letters felt tedious and slow. When we began brainstorming keyboard and hardware redesigns to solve this problem, our mentors advised us to refocus our efforts on spatial interactions in XR instead.
We brainstormed different types of text and digital communication, categorized them, and created different types of scenarios for them. We landed on three general categories that piqued our interest:
  • Short messages, such as quick replies and emojis
  • Single phrases, such as search, passwords, and dates
  • Long-form content, such as essays, code, and blogs
Justifying Emojis in VR
Brainstorming reasons why someone would use emojis in XR over smartphones revealed two interesting insights:
  • Since people can inhabit the same space in XR, it allows them to see each other react to emojis in real time
  • Emojis could have much greater visual impact in XR, given that they could be presented in 3D space
We designed an emoji communication tool where people can feel the magic of sending and receiving exciting emoji effects in XR and communicated our vision of a use case through a storyboard with the following story elements:
  • Communicating with each other just for the sake of fun
  • Seeing the real-time reaction of the other person in a shared space
  • Making XR emoji communication more exciting than texting
Users of our tool need to bring up emojis in XR just as fast as on a keyboard, and through sketching and brainstorming sessions we based our emoji system off of video game emote systems. We were specifically inspired by online multiplayer video games because they usually have quick ways to communicate so that players can express ideas and play at the same time.
Click to enlarge
Specifically, we referred to League of Legends’ (LoL) emote wheel where players could create a wheel of five emotes that could be taken into any LoL match.  Borrowing this concept, we imagined users in XR could load their own expressions into a plate that could be taken into any online XR space or game.
Click to enlarge
Preset vs. Custom Emojis
We designed the system to have preset emojis to pick from, but we thought the ability to import custom emojis and expressions could allow for more personalized expression.
Concept Breakdown
We came up with three constraints for both preset and imported objects to provide a consistent experience between the two. At this point, we decided to put more emphasis on designing the emojis' visual effects as they are sent to another person.
VR Prototyping
We compiled a list of interactions that we liked and I prototyped them in Unity for feasibility. If they worked, we would show our class and professor to gauge excitement and then fully prototype the most exciting interactions in Unity.
Final Design
We made a video prototype, which I co-directed and filmed in a commercial-style, that focused on four cases that encompassed our entire design. The other members of my team created and supplied 3D models of emojis in Blender and handed them off to me to build in Unity.

The four use cases we decided on were:
  • Using a custom object (troll face)
  • Pranking a friend (poop emoji)
  • Expressing frustration (crying emoji)
  • Celebration (confetti emoji)
We chose these four because they represented many of the purposes in which someone would use XR emojis. Whether its importing your own objects for self-expression, messing with friends, or throwing a virtual party, we believed these four use cases covered these interactions.
The Result
The final design was more magical than we had expected. Other members of our class found it extremely intuitive to grab and throw emojis. Our professor was excited at the potential to throw emojis back and forth with their friends and stated how this felt like the next step for visual communication in XR.
Use Case: Using a custom object (troll face)
Effect: Bounce
Use Case: Pranking a friend (poop emoji)
Effect: Attach
Use Case: Expressing frustration (sad emoji)
Effect: Explode
Use Case: Celebration (confetti emoji)
Effect: Confetti Effect
Next Steps
Future Directions
In our early exploration phase, our professor had suggested object alternatives to emojis, as he neither related to emojis nor thought of them as dynamic. Given these concerns, I would like to explore other abstract visual elements as forms of communications in the future.

If we had more time, we would have tried to develop more effects and gestures for our expressions to create more fun ways to activate emoji effects. We currently only utilize the throwing motion to trigger our expressions, but we had floated ideas that involved more dynamic movements.
Through this project, I learned a lot about VR prototyping and design and hope to pivot into designing more in the VR space. Learning how to utilize virtual space was a huge challenge but an important revelation about how VR designers see the world. Our professor said a quote that still sticks with me: "Designers should be designing for the future, so why are we still designing for phones?"