Brief introduction to the project:

The motion capture of arm is realized based on the hardware solution of "three MPU6050+MCU". The data transmission mode is transparently transmitted by using an onboard USB or Bluetooth serial port.

Solution Description

1.      Directly call built-in DMP driver of MPU6050 to capture the posture of each sensor (quaternion format) and send it to host computer.

2.      Bind the sensor to the bone model in the 3D environment by DYNAMIC INITIALIZATION to determine the posture deviation between the sensor and the corresponding bone model.

3.      Combine inverse kinematics to calculate bone posture in real time to achieve motion capture.

Hardware Frame

As shown in the figure below, MCU uses TI's MSP430F5259, and the communication with the host computer can be selected in USB or BLE. Three MPU6050 are respectively worn on the back of the hand, the lower arm near the wrist and the upper arm near the elbow, which is corresponding to the calculation of the movement posture of the hand, the lower arm and the upper arm. IMU that captures the lower arm posture is placed directly on the motherboard of MCU, and the remaining two IMUs are two separate modules.

Description of Advantages and Disadvantages of the Solution

Advantages of the solution:

1.     The hardware does not use a magnetic field sensor. It can be “plug-and-play”, which effectively avoids problems such as repeated calibration of magnetic field sensors and magnetic interference etc.

2.     The same initialization mode can be combined with other devices with posture data. Take Google Daydream's remote controller as an example which participates in a video demonstration later.

3.     Directly call built-in DMP driver of MPU6050 to reduce the calculation pressure of MCU and further cut the cost.

Disadvantages of the solution:

1.      The startup process of DMP driver is slow, and the posture of sensor can be stabilized after about 15 seconds.

2.      The posture calculated by DMP driver is easy to drift, and the effect is not as good as that of Mahony or Kalman filtering.

3.      DMP uses FIFO mode to cache the results of calculating. Improper handling can easily cause data overflow.

4.      Without fusion magnetic field sensor, the phenomenon of drift will exist no matter which posture calculation mode is adopted. However, when the drift is obvious, it can be reinitialized.

Project Demonstration

• Motion capture demonstration in the Unity3D environment on the PC terminal

PC is connected to a USB BLE module and set to be HOST mode. It mainly demonstrates the motion capture effect of dynamic initialization and that after the initialization. The hardware sends the posture of each IMU to the PC in real time; the deviation relationship between the coordinate system of each IMU and the coordinate system of the corresponding arm skeleton model can be quickly calculated through a DYNAMIC INITIALIZATION; combined with the initialization calculation result and the real-time posture of IMU, the skeletal posture of the arm model can be calculated, and the motion capture of the arm can be realized.

You can use the "USB Arm" engineering in the Unity3D folder in the provided resources.

• Motion capture and Google Daydream fusion demonstration

Develop Daydream app in Unity, and directly call the data provided by the Google VR service through the official SDK, such as the trigger event of the remote controller. Here, the posture of the remote controller and the posture of the motion capture device are directly called. Fusion is not a simple way to bind the remote controller to the arm model, but rather to the different postures of the synchronizing remote controller...