©2023
————————— VIEW ANOTHER PROJECT? —————————
©2023
— REFLECTIONS —
— REFLECTIONS —
What I learned…
What I learned…
🚧 Push back on your assumptions.
While we initially thought that abstraction might be in direct opposition to being able to transfer the concepts to real life, we found that users had a stronger conceptual understanding of how the pipette worked, and thus were able to quickly learn how to use the real pipette.
📚 Get creative recruiting for user tests.
One struggle our team had was with getting enough users who had learning disabilities for our prototype tests — disclosing an LD can be a sensitive subject for some and it needs to be handled carefully and with anonymity.
We were able to solve this problem by creating a functional needs survey, which asked potential participants behavioral questions related to LDs but left identification of LDs optional.
Combined with giving participants more flexibility with their compensation (e.g. being able to choose their own gift card), this helped us recruit many more participants for our hi-fi prototypes that fit our target demographic.
🚧 Push back on your assumptions.
While we initially thought that abstraction might be in direct opposition to being able to transfer the concepts to real life, we found that users had a stronger conceptual understanding of how the pipette worked, and thus were able to quickly learn how to use the real pipette.
📚 Get creative recruiting for user tests.
One struggle our team had was with getting enough users who had learning disabilities for our prototype tests — disclosing an LD can be a sensitive subject for some and it needs to be handled carefully and with anonymity.
We were able to solve this problem by creating a functional needs survey, which asked potential participants behavioral questions related to LDs but left identification of LDs optional.
Combined with giving participants more flexibility with their compensation (e.g. being able to choose their own gift card), this helped us recruit many more participants for our hi-fi prototypes that fit our target demographic.
This project is intended to be passed on for further development. Our main suggestions for the roadmap were:
Develop the feedback system.
Add personalization options (speech speed, ability to turn off audio, etc.) to increase accessibility.
Extend the experience to the full experiment.
With the core interaction of micropipetting down, MotionLab should ideally be able to be expanded to incorporate PBO’s other microbiology experiments.
This project is intended to be passed on for further development. Our main suggestions for the roadmap were:
Develop the feedback system.
Add personalization options (speech speed, ability to turn off audio, etc.) to increase accessibility.
Extend the experience to the full experiment.
With the core interaction of micropipetting down, MotionLab should ideally be able to be expanded to incorporate PBO’s other microbiology experiments.
Feedback & Personalization
Feedback & Personalization
— FUTURE WORK —
— FUTURE WORK —
Jump to Solution→
Jump to Solution→
— FINAL TESTING —
— FINAL TESTING —
Successful Transfer
Successful Transfer
We tested our hi-fi prototype with 5 college students who scored above average on our functional needs survey, which asked students if they identified with common needs present in students with LDs.
We focused on assessing concepts students typically had trouble with when micropipetting including proper hold, selecting volume, and being able to draw and dispense liquid. Most users were able to successfully transfer conceptual learning from the digital experiment to real life.
We tested our hi-fi prototype with 5 college students who scored above average on our functional needs survey, which asked students if they identified with common needs present in students with LDs.
We focused on assessing concepts students typically had trouble with when micropipetting including proper hold, selecting volume, and being able to draw and dispense liquid. Most users were able to successfully transfer conceptual learning from the digital experiment to real life.
A participant walking through the experiment with the Leap Motion.
A participant walking through MotionLab.
Our client, Collin, using MotionLab.
We were able to do a test run with our clients before making our presentation, to positive feedback.
We were able to do a test run with our clients before making our presentation, to positive feedback.
This is exactly what we’re looking for.
This is exactly what we’re looking for.
— FINAL DESIGN —
— FINAL DESIGN —
Experiencing the Lab in Mixed Reality
Experiencing the Lab
in Mixed Reality
Focused Instructions
Focused Instructions
Instructions are broken down into sub-steps that allow students to process one thing at a time, with cues next to relevant objects. This increases student confidence that they will be guided through each step and allow them to move through the experiment at their own pace.
Instructions are broken down into sub-steps that allow students to process one thing at a time, with cues next to relevant objects. This increases student confidence that they will be guided through each step and allow them to move through the experiment at their own pace.
Tidy Virtual Environment
Tidy Virtual Environment
The virtual lab environment eliminates the need for students to run around the classroom to get different equipment, thus reducing distraction and attention switching.
The virtual lab environment eliminates the need for students to run around the classroom to get different equipment, thus reducing distraction and attention switching.
Video demo of MotionLab.
Video demo of MotionLab.
Multisensory User Interface
Multisensory User Interface
Instructions and feedback are conveyed through both text and audio, which has been found to increase comprehension and focus for students with ADHD.
Instructions and feedback are conveyed through both text and audio, which has been found to increase comprehension and focus for students with ADHD.
— MID-FI PROTOTYPE —
— MID-FI PROTOTYPE —
Abstracting Gestures
Abstracting Gestures
Thus, the guiding questions for mid-fi were to find out how to abstract gesture and effectively simulate haptic feedback. We prototyped 3 core interactions in Unity for user tests:
Thus, the guiding questions for mid-fi were to find out how to abstract gesture and effectively simulate haptic feedback. We prototyped 3 core interactions in Unity for user tests:
For this iteration, our main design principles were to:
✋ Separate function of left and right hands.
For clearer interaction, tasks involving refined control like pushing the plunger and dialing volume were given to the non-dominant hand, whereas navigation, like moving and picking up the pipette, were given to the dominant hand.
✅Break down instructions to task labels.
Since the instructions we had before were overwhelming, we changed them to short task labels attached to the active tool.
For this iteration, our main design principles were:
✋ Separate function of left and right hands.
For clearer interaction, tasks involving refined control like pushing the plunger and dialing volume were given to the non-dominant hand, whereas navigation, like moving and picking up the pipette, were given to the dominant hand.
✅Break down instructions to task labels.
Since the instructions we had before were overwhelming, we changed them to short task labels attached to the active tool.
Grabbing the Pipette
Grabbing the Pipette
Users can grab, rotate, and drop the micropipette.
Users can grab, rotate, and drop the micropipette.
Pushing the Plunger
Pushing the Plunger
Users can press down with their left hand to “push” the plunger down. We simulated resistance by slowing down the plunger bar when the user got closer to the second stop.
Users can press down with their left hand to “push” the plunger down. We simulated resistance by slowing down the plunger bar when the user got closer to the second stop.
Reset Button
Reset Button
Users can flip their left hand to open the menu, where they can reset the experiment.
Users can flip their left hand to open the menu, where they can reset the experiment.
— LO-FI PROTOTYPE —
— LO-FI PROTOTYPE —
Simplifying the Experiment
Simplifying the Experiment
Similar to how students felt doing the lab in-person, a challenge we faced was keeping track of which tools to focus on in each step and how they would respond to the user. Because giving immediate feedback to errors was a primary goal, we wanted to avoid prototyping a "happy path," instead allowing users to make mistakes and correct them.
We started off with a text-only "zero-fi prototype" that smiplified the experiment to primary steps, identifying what the user sees and does in each step, and potential mistakes, which was then prototyped as a 2D experience in Figma.
Similar to how students felt doing the lab in-person, a challenge we faced was keeping track of which tools to focus on in each step and how they would respond to the user. Because giving immediate feedback to errors was a primary goal, we wanted to avoid prototyping a "happy path," instead allowing users to make mistakes and correct them.
We started off with a text-only "zero-fi prototype" that smiplified the experiment to primary steps, identifying what the user sees and does in each step, and potential mistakes, which was then prototyped as a 2D experience in Figma.
We tested our participants Wizard of Oz style, then evaluated how well they could use micropipette in real life. Our main insights from the lo-fi tests were that:
Despite cutting down significantly, users still zoned out while reading the instructions.
A 1:1 replication of gestural interaction with the pipette was not necessarily the best way to communicate how to use the tool, especially when accounting for the technical limitations of the LMC.
We tested our participants Wizard of Oz style, then evaluated how well they could use micropipette in real life. Our main insights from the lo-fi tests were that:
Despite cutting down significantly, users still zoned out while reading the instructions.
A 1:1 replication of gestural interaction with the pipette was not necessarily the best way to communicate how to use the tool, especially when accounting for the technical limitations of the LMC.
Users followed experiment instructions while a team member changed the screen behind the scenes.
Users followed experiment instructions while a team member changed the screen behind the scenes.
A walkthrough of the micropipette experiment.
A walkthrough of the micropipette experiment.
ROLE
Research & Design
Project Management
FOR
Pitt Bio Outreach
Learning Disabilities Assoc.
FOR
Pitt Bio Outreach
LDA of PA
TEAM
Audrey Renouf, Adrian Ma, Eric Gan, Jiaqi Wang
TIMELINE
4 months
Spring 2023
MotionLab 🧪 is a XR environment for interactive biology experiments that students can control with hand gestures.
MotionLab 🧪 is a XR environment for interactive biology experiments that students can control with hand gestures.
ROLE
Research & Design
Project Management
FOR
Pitt Bio Outreach
LDA of PA
TEAM
Audrey Renouf, Adrian Ma, Eric Gan, Jiaqi Wang
TIMELINE
4 months
Feb 2023 - May 2023
— DEFINING THE PROBLEM —
— DEFINING THE PROBLEM —
Selecting the Right Technology
Selecting the Right Technology
Our clients initially proposed a VR-based platform, but were unfamiliar with the cost and challenges of using VR in the classroom. However, they found value in VR's hand tracking feature, especially for its potential in assisting procedural learning of lab tools.
Our clients initially proposed a VR-based platform, but were unfamiliar with the cost and challenges of using VR in the classroom. However, they found value in VR's hand tracking feature, especially for its potential in assisting procedural learning of lab tools.
Thus, we explored more budget-friendly methods that offered motion control, such as NFC-tagged objects, interactive wall projections, and Leap Motion Controllers (LMCs). After weighing the pros and cons with our clients, we chose to proceed with the Leap Motion, a compact sensor that connects to a computer and accurately captures hand movements. The Leap Motion offers an experience that is…
Thus, we explored more budget-friendly methods that offered motion control, such as NFC-tagged objects, interactive wall projections, and Leap Motion Controllers (LMCs). After weighing the pros and cons with our clients, we chose to proceed with the Leap Motion, a compact sensor that connects to a computer and accurately captures hand movements. The Leap Motion offers an experience that is…
Technology ideations, illustrated by Jiaqi Wang.
The LMC allows open space for both peer-to-peer and teacher interaction.
The LMC allows open space for both peer-to-peer and teacher interaction.
Shareable
Shareable
The LMC is only around 3 inches in width, packaged in a small box for easy transport.
The LMC is only around 3 inches in width, packaged in a small box for easy transport.
Portable
Portable
Compared to a VR headset, which can be ~$300+, the LMC is around $100.
Compared to a VR headset, which can be ~$300+, the LMC is around $100.
Cost-effective
Cost-effective
— THE CHALLENGE —
— THE CHALLENGE —
How can we support high school students' experiential learning?
How can we support high school students' experiential learning?
Our clients came to us with a vision of creating a virtual reality (VR) prototype based on their existing experiments, which deliver instructions through annotated videos and were designed for students with LDs in mind.
Our clients came to us with a vision of creating a virtual reality (VR) prototype based on their existing experiments, which deliver instructions through annotated videos and were designed for students with LDs in mind.
However, they had a limited amount of kits and were seeking a platform that could expand the availability of these experiments through virtual methods. In addition, they wanted to use the affordances of technology to improve the effectiveness of instruction. We had a 4-month timeline to deliver a MVP that could be pitched to future investors.
However, they had a limited amount of kits and were seeking a platform that could expand the availability of these experiments through virtual methods. In addition, they wanted to use the affordances of technology to improve the effectiveness of instruction. We had a 4-month timeline to deliver a MVP that could be pitched to future investors.
Screenshot from PBO's annotated micropipette experiment video
Screenshot from PBO's annotated micropipette experiment video
Hands-on, lab-based science experiments are critical to high-school STEM education but can be frustrating for some students, especially those with learning disabilities (LDs). Lab work requires focus and precision, and a single step skipped or tube mislabeled can end up ruining the entire process. With limited class time to try again, these common mistakes can cause discouragement from the learning process.
Hands-on, lab-based science experiments are critical to high-school STEM education but can be frustrating for some students, especially those with learning disabilities (LDs). Lab work requires focus and precision, and a single step skipped or tube mislabeled can end up ruining the entire process. With limited class time to try again, these common mistakes can cause discouragement from the learning process.
This was my senior capstone for Human-Computer Interaction. In partnership with Pitt Bio Outreach (PBO) and the Learning Disabilities Association of Pennsylvania (LDA of PA), my team developed an interactive virtual simulation of a lab experiment that aims to improve the learning experience for students with LDs. Our prototype focuses on teaching micropipetting, an essential skill for techniques like DNA extraction and pipetting cells.
This was my senior capstone for Human-Computer Interaction. In partnership with Pitt Bio Outreach (PBO) and the Learning Disabilities Association of Pennsylvania (LDA of PA), my team developed an interactive virtual simulation of a lab experiment that aims to improve the learning experience for students with LDs. Our prototype focuses on teaching micropipetting, an essential skill for techniques like DNA extraction and pipetting cells.
📄
📄
Frequently switching focus between instructions and lab equipment led to errors.
Frequently switching focus between instructions and lab equipment led to errors.
Most errors stemmed from forgetting steps or incorrect use of equipment.
Most errors stemmed from forgetting steps or incorrect use of equipment.
🧪
🧪
👩🏫
👩🏫
Immediate feedback is crucial, but teachers & students can't catch every mistake.
Immediate feedback is crucial, but teachers & students can't catch every mistake.
— UNDERSTANDING OUR USERS —
— UNDERSTANDING OUR USERS —
Experiencing the Lab in Reality
Experiencing the Lab in Reality
Given that none of us had touched high school biology in years, we set out to explore how students learn in the lab. To achieve this, we conducted a contextual inquiry with 4 undergraduate students at the University of Pittsburgh, led through the micropipetting experiment by two professors.
To make things smoother during our inquiry, we tried our hand at the experiment in advance. Through our observation and follow-up questions, we came to the following insights:
Given that none of us had touched high school biology in years, we set out to explore how students learn in the lab. To achieve this, we conducted a contextual inquiry with 4 undergraduate students at the University of Pittsburgh, led through the micropipetting experiment by two professors.
To make things smoother during our inquiry, we tried our hand at the experiment in advance. Through our observation and follow-up questions, we came to the following insights:
Learning how to use a micropipette, observing, and interviewing students.
Learning how to use a micropipette and interviewing students.
While students felt that the videos were an improvement over traditional lab reports, novices still struggled with information overload during learning. Our team saw opportunities to enhance learning by incorporating visual cues, interactive instructions, and guidance on how to fix mistakes.
While students felt that the videos were an improvement over traditional lab reports, novices still struggled with information overload during learning. Our team saw opportunities to enhance learning by incorporating visual cues, interactive instructions, and guidance on how to fix mistakes.
