Problem Statement:

An immersive experience in Virtual Reality requires a method of input that is intuitive and natural. There are two components to this requirement- use of natural gestures and movements to represent the virtual act and some kind of haptic cue as feedback to the user.

This project will be further extended to include applications in telerobotics and motivate alternative methods for robot control and programming.

My solution:

I am going to attempt to make 2 types of devices:

1) Wand Type Controller

2) Exoskeleton Glove

The user will typically hold the wand in the left hand and the glove in the right hand. (Right handed user)

I believe this kind of hybrid scheme will offer features that are not possible with only wands, joysticks or gloves. Refer to DesignDoc1 for more information and justification on how my project addresses the problem statement.

How will I do it?

I intend to primarily use IMU's for tracking. I do understand that errors in IMU's can accumulate very quickly when it comes to position tracking. However I suspect that we can still get data that will allow us to guess user intent for the interaction, this data may be orientation or acceleration data which you can use in a clever way to define an "action or move" in VR.

See Potential Problems for more discussion. I posted this text here because I don't want you to read IMU and then abandon reading the rest of this project which might still be useful...

I will use a DC motor on the wand and probably a motor-spring combination for the glove impedance control. The advantage of using IMU is that it is computationally less expensive than computer vision methods. This makes it more suitable for mobile VR applications. See project log 3 for initial thoughts on glove actuator design.

Distinguishing features:

1) Force Haptic Feedback on glove and simulation of squishy or elastic virtual objects and interactions

2) Use of Hybrid Control Scheme for VR input ( all others use only glove or only controller)

3) Suitability of mobile VR as well as PC/Console based VR.

4) Cheap, Open Source and Easy DIY

I think this project will be useful to:

This project will hopefully be useful to novices in this field. Another reason for keeping it simple and low-cost is that I want to involve more people with a diverse background. Hopefully this will provide some relevant material for Artists,Product Designers, Game Creators and different types of engineers.

So if you have a cool idea for VR but the current set of available controllers is not ideal and you need another set without having to design every component yourself OR if you want to spend time on your own idea of the ideal combination for VR input but need some starting input then this project may be useful to you.

I can also think of some Non VR/AR applications in field of assistive technology, new training methodologies etc. For example: with the glove you can use it to teach someone to play the flute. You can take a song and preprogram a key sequence for the fingers. Thus the learner will get cues while playing the song. This is useful because being able to play a song somewhat well in the early stages of learning will be very encouraging and will keep the person motivated in the early stage when they are most likely to give-up.

Possible Problems:

I have not made a similar project before but IMU drift may be a problem.

Possible Solutions and Future Projects:

I believe an external on-the-fly re-calibration system using a leap motion device or the light-house type system could be used to periodically obtain a more accurate position of the system to correct for the drift. I do not yet know how exactly to do that but I think this could be possible by getting the leap motion to recognize a feature on the device as a tool and then using that reading for the accurate position thereby resetting the error periodically.

Valve which is behind the light-house system intended to make this technology available to hardware manufacturers. From what I understand the reason it is not already out there is that they are somewhat busy and are prioritizing different things at the moment. See link to this news.

http://store.steampowered.com/news/16000/

Why not stick to only a leap motion? Because it has a limited field of view.

Bang for Buck:

The exoskeleton glove will primarily focus on the index finger as I believe this will give the most bang for buck. Once the system works for the index finger, the other fingers should be straightforward (thumb will still need work). The project will hopefully cost less than 250$. Note that a braking mechanism as used in some commercial devices cannot account for all interaction dynamics. I hope to support open source work on impedance control with a UX approach.

Secret Sauce:

A first attempt will also be made to address the locomotion problem.

Misc:

These controllers will be portable enough that you could carry it in your backpack or suitcase if you wanted to.

I welcome feedback and suggestions. I am sure there are many among the community who have experience in using these systems or sensors and may be able to suggest cheaper and simpler alternatives. I am sure this type of input will be appreciated by other novices like me.

License:

This project is shared under: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Feel free to tweak this project and your improvements ! We want to make VR as amazing as it can be!