In my undergraduate design course that focused on the interactions between people, artifacts, and environments, our group was tasked to postulate a ‘What If…?’ question that explores the use of mobile device cameras and to design a future experience for it.

Inspired by the COVID-19 pandemic, lidar sensors, and physical therapy, we designed a remote physical therapy (PT) platform that would allow patients to do physical therapy confidently at home.

I was involved in UX research, storyboarding, and video prototyping, but spent the majority of time designing the user interface.

Ariel Chiang, Vanessa Lin, Chandler Simon, Ryan Stephen, Andrew Tang

3 Weeks

Figma, After Effects, Premiere

In order to understand PT during the COVID-19 pandemic, we interviewed a physical therapist to get a sense of their current workflow. We learned that:

After talking with the physical therapist, we found that proper guidance on exercises was important for patients. We thought of designing an experience that utilized smartphone lidar sensors to help track and accurately measure a patient’s exercises.

Our group articulated three core needs for a patient and their doctor:

We aimed to design an experience that used existing technologies in patients’ homes. Through some explorations, we compiled three technologies that could work together as a system:

With the system, a couple new questions arose that we decided to answer by conducting a small participatory workshop with our physical therapist and a participant.

Yes! In fact, it was better than expected. This solved our first core need of remote video capture.

We worked with our therapist to design a tailored 3D mesh during an exercise that shows the correct exercise position once a patient reaches an exercise’s tension point.

Both the therapist and patient preferred voice controls because the phone is too far away for touch while doing an exercise and gesture control may be too complex for the mobile camera.

We decided to communicate our design through a video demo and developed a storyboard to show how a physical therapist gives a patient exercises, how a patient receives the exercises, and how a patient sees and completes the exercises.

I prototyped two versions of this interface. I designed the first version without much direction and I ended up overdesigning features and interactions.

designed the second version with the intent on emphasizing search and personalization. Given the short project timeframe, I made the assumption that the doctor needed to find the right exercises for the patient, adjust its parameters, and add to the schedule.

I designed the second version with the intent on emphasizing search and personalization. Given the short project timeframe, I made the assumption that the doctor needed to find the right exercises for the patient, adjust its parameters, and add to the schedule.

The TV would display a quick demo of the exercise before prompting the patient to say ‘start’. Once the exercise has started, a display of sets and repetitions is shown and patients are prompted to do the exercise.

The 3D mesh was designed to provide visual clarity for the exercise. If a patient was doing the exercise wrong, the 3D mesh would appear to guide the patient in the right direction.

Our design presentation and final video was shown to our professor and our physical therapist. Our professor praised our design as having the potential to be a real product, and encouraged us to pursue it further. Our physical therapist also expressed interest in using the platform with their patients, and encouraged us to consider implementation.

Our design presentation and final video was shown to our professor and our physical therapist. Our professor praised our design as having potential to exist today, and encouraged us to consider turning it into a real idea. Our physical therapist was jealous that they couldn’t be using the platform now with their patients, and also encouraged us to consider implementation.

This project taught me the importance of decision making behind UI elements. Due to the short timeframe for the project I did not have time for multiple rounds of design iteration. If I were to redo the UI design, I would want to conduct additional usability tests to better quantify the usefulness of the experience. Our color choice was also not the most accessible as we randomly decided on a color to move forward with. Regardless, I’m very proud of what we were able to accomplish in such a short amount of time, but there was definitely a lot of room for more UI testing.