Quinnipiac Virtual Reality: Healthcare
In the fall of 2016, I received an email from my Computer Science advisor, Jonathan Blake, which informed me of a group that was being assembled to work on a grant project. This group needed someone of my skill set and was hoping that I would be willing to join them. Soon after I was put in contact with professor Corey Kiassat, one of the professors on this project, who was able to give me more information about what we would be doing. The overall idea was to create a virtual reality training simulator for Health Science students in order to help them learn how to transport patients correctly. After discussing this idea further with Corey, I decided to join the group.
|
Our meetings and work began in the spring of 2017 with a group of 5 students and 3 professors. Three of the students, Michael Smizaski, Andrew Wolak, and Gregory Guevara, were majoring in a health related field and were in charge of doing research on our project and collecting the necessary data for me to calculate what the proper angles would be for a person to perform a correct transfer of a patient. While they were collecting this information, both Nicolas Northcutt, an engineering major with little background in programming, and myself began the actual implementation and building of the training simulator using both the Oculus and a Kinect 2 in a game engine called Unity. Every Wednesday, the students and professors would all meet to discuss what we have done in the previous week as well as the next steps each of us should take. Unfortunately, soon after starting this project, Nicolas and I realized how little material there was for us to reference. In fact, after doing a lot of research, we had learned that a simulation like this has never been attempted and that we were, for the most part, on our own. Due to this, we experienced several hiccups along the way which ultimately slowed our project and push the completion time back to the end of the 2017 fall semester. Some of the problems experienced were:
- Getting the Oculus to work correctly in Unity.
- Understanding how the Kinect tracks human movements
- Correctly mapping the joints the Kinnect picked up to the 3D character model being used.
- Fearing motion sickness, the group wanted the user to be able to see their hands while picking up the patient.
- Getting the Oculus to work correctly in Unity.
- Understanding how the Kinect tracks human movements
- Correctly mapping the joints the Kinnect picked up to the 3D character model being used.
- Fearing motion sickness, the group wanted the user to be able to see their hands while picking up the patient.
Unfortunately, in the fall of 2017, due to the busy schedules of each group member, we could not find a time to meet. This caused the project to go untouched for several months. Seeing its potential and not wanting the work we have already done to go to waste, I decided to continue working on this project alone in the spring of 2018. However, to do so, I needed to improve upon my coding abilities in virtual reality. In particular, I needed to know how exactly the Oculus and the Kinect 2 worked as well as how to access the data that they gather from the user
Figure 2: User's Heads Up Display (HUD)
|
After spending my Winter break learning about the Oculus and the Kinect, I began to work on this project again in the Spring of 2018. Throughout the 12 weeks I had, I was able to both improve upon our the prior work as well as build off of the original design of the project by adding in features that the original group never believed possible; such as the ability to record a user's movements and play it back to them after the simulation was completed or creating a colorized skeleton of the user informing them which body parts were in place and which needed to be adjusted (figure 2).
|
With all the challenges I had to overcome, long nights I spent tweaking the simulation, and difficult times I had to push through, this project made me a much better programmer and, in my personal opinion, was well worth it. Virtual Reality is a field which is still fairly new and hardly explored. Due to this, I truly believe that the amount of training and educational impacts it has can be endless. A simulation as detailed and informative as mine could easily revolutionize the ways in which we learn simple everyday tasks and ultimately prevent many unfortunate and unnecessary injuries.
For more information about this project, please feel free to email me or read this article.
For more information about this project, please feel free to email me or read this article.