Close
0%
0%

Open Muscle Finger Tracking Sensor

Forearm Muscle-Based Finger Tracking Device

Public Chat
Similar projects worth following
OpenMuscle is an innovative open-source and open-hardware project designed to improve the lives of people in need of prosthetic hands. This project focuses on creating a cost-effective, customizable, and easily accessible platform to control prosthetic hands using forearm muscle movement detection.

The OpenMuscle system is based on pressure sensors placed around the forearm, which measure muscle activity and predict finger movement. By utilizing the natural muscle contractions in the forearm, this groundbreaking technology translates the user's intent into precise, controlled finger movements for the prosthetic hand.

As an open-source and open-hardware platform, OpenMuscle encourages collaboration and creativity within the assistive tech community. This project aims to provide a foundation for researchers, engineers, and hobbyists to further develop and refine the technology, making it accessible to a wider audience.

Open Muscle – The Open-Source Forearm Muscle-Based Finger Tracking Device

It uses custom built pressure sensors made from hall effect sensors, springs, and magnets. The current version has 12 sensors placed radially around the forearm and can detect different finger movements.

To train open muscle we used our LASK System that applies the labels for the feature data that the open muscle band records. Both are open source and open hardware.

Preliminary Machine Learning Testing

It was exciting to get our first machine learning results and they were better than expected! Live model predictions were done and showed the ability for the model to learn some of the finger presses. The results varied given the tightness of the band, the rotation of the sensor band, keeping the open muscle band at the same location on the forearm for both training capture and live prediction sessions.

Cyan is the actual signal from the LASK
Yellow is the predicted output from the ML Model

Bottom 12 are the open muscle sensors

The is much work to do but each mini success adds to the greater goal of a prosthetic sensor hardware, software, and machine learning suite.

All data to train the model and code available on Github: https://github.com/turfptax/openmuscle


Our first model training: Note that these included the time domain for each data-point.

Preliminary ML Test Results:
predictions_vs_actuals2-Ordered-1
Download
predictions_vs_actuals2-1Download
predictions_vs_actuals2-ArangeDownload


We have just received our first results from the training model and have a lot more work to do. If you would like to be a part of the team please reach out right away! Mostly it has just been me working on this for the last 9 months. -Turfp

OpenMuscle Detecting Finger movements with pressure sensors and machine learning-01.jpg

Detecting Finger Movements with a pressure sensor bracelet Open Muscle and machine learning.

JPEG Image - 176.54 kB - 03/26/2023 at 23:36

Preview
Download

Comma-Separated Values - 1.99 MB - 03/26/2023 at 19:30

Download

OpenMuscleV530Pinout copy-01.jpg

Pinout and Wiring Diagram for the Open Muscle V 5.3.0

JPEG Image - 2.42 MB - 12/02/2022 at 17:18

Preview
Download

OpenMuscle5-F_Mask.zip

OpenMuscle Gerber file for Version 5.3.0

x-zip-compressed - 66.98 kB - 10/23/2022 at 02:47

Download

  • 12 × 3mm x 12mm Clevis Pins
  • 12 × 20g MX Cherry Springs
  • 1 × Battery Charging & Supply Module
  • 1 × On/Off Switch
  • 1 × ESP32-S2 mini by wemos

View all 8 components

  • The Journey of Open Muscle: Democratizing Assistive Technology

    TURFPTAx07/13/2023 at 13:36 0 comments

    One day, while scrolling through the internet, I came across a video that stopped me in my tracks. It was about a man with no arms who had undergone surgery to move nerves into his pectoral muscles. This transformation enabled him to control two prosthetic limbs. But there was a catch – he couldn’t take these arms home. Why? Because the technology and code that powered them were proprietary. This man was denied the freedom to fully embrace these life-altering tools because of a man-made barrier. The story was not just sad; it was infuriating.

    This experience became the spark that ignited the creation of Open Muscle. I saw the potential of machine learning and believed that it could provide a solution. I had an idea: a matrix of pressure sensors sampling forearm muscle movements and using a force detection labeling system to measure finger movements. In theory, AI or machine learning could predict the finger movements based on the pressure sensors’ data.

    It was an ambitious plan, but I was ready to face the challenges head-on. The journey was not without its difficulties. For instance, finding the right springs for the pressure sensors was a task that felt like an endless Goldilocks conundrum. After much trial and error, the solution came in the form of a larger pressure pad, which allowed us to use firmer springs.

    Another challenge was designing a system that could be adjusted and upgraded if needed. The solution was a radial design, which captured sufficient data from the muscles and was a perfect fit for machine learning.

  • Open Muscle Video Update AI Prosthetic Sensor

    TURFPTAx05/02/2023 at 16:06 0 comments

  • Finger Movement Recognition Using Radial Pressure Sensors on the Forearm and Machine Learning Techniques

    TURFPTAx04/16/2023 at 16:14 0 comments

    In this study, we present a novel and cost-effective method for detecting individual finger movements using a custom pressure sensor array based on Hall effect sensors and magnets. Twelve sensors were placed radially around the forearm, and machine learning techniques, specifically a random forest regressor, were employed to distinguish between finger curls. With only 5 minutes of training data, our approach achieved a mean absolute error (MAE) of 35, demonstrating its potential as a viable and inexpensive sensor solution for finger movement and force detection. This forearm armlet-based system has applications in fields such as prosthetics, robotics, and human-computer interaction, offering a promising alternative to traditional sensor technologies.

    Introduction

    Hand amputations can significantly impact an individual’s quality of life and ability to perform daily tasks. Despite losing their hand, many people with amputations retain an intact forearm with functional muscle movement. However, due to the disconnected tendons, detecting and interpreting finger movements becomes challenging. In this study, we sought to develop an innovative and cost-effective sensor solution that could discern individual finger movements by sampling the force exerted by muscle contractions in the forearm. One of our main challenges was finding a low-pressure sensor system with minimal friction. We explored different designs and ultimately used a spring and clevis pin system for the best results, while also considering rubber foam for a more scalable solution in future designs.

    We hypothesized that with minimal training data, our approach could provide an inexpensive and accessible solution for a wide range of individuals who could potentially benefit from this technology. To test our hypothesis, we conducted experiments on able-bodied individuals with the intention of validating the efficacy of our approach and warranting further research in the context of hand amputees. Our work was conducted using open-source and open hardware principles to contribute to the scientific community with a non-proprietary alternative to existing forearm sensor technologies. This open approach aims to foster collaboration, innovation, and widespread adoption of our proposed forearm bracelet system.

    In this paper, we describe the design and implementation of our custom pressure sensor array, the machine learning techniques employed to recognize finger movements, and the performance of our system in detecting individual finger curls. We also discuss potential applications of our technology in prosthetics, robotics, and human-computer interaction, as well as future directions for research and development.

    Materials & Methods

    Pressure Sensors:

    Custom pressure sensors were designed using Hall Effect sensors mounted on printed circuit boards (PCBs). These PCBs were attached to PLA 3D printed clevis pin holders, which featured a hole for the clevis pin piston positioned 7mm above the Hall Effect sensor. A nail head “foot” was secured to the piston, and a 5mm diameter earth magnet was glued to the foot. The piston was fitted into the clevis pin holder, and a spring and clip were attached to the opposite side. The feet were designed to make contact with the skin, and their surface area could be adjusted to modify the sensor’s overall amplitude.

    A second version of the sensor utilized the same PCB but incorporated a rubber foam spring system, a rubber sheet, and a magnet. This alternative design yielded promising results and is recommended for further investigation, as its construction requires less time.

    Sensor Array and Data Acquisition: The magnet array was wired together and connected to an ESP32-S2 microcontroller, which utilized MicroPython firmware and its 20 13-bit ADCs to measure the Hall sensor values.

    Prototype Version 5.3.0 with the Clevis Pin Piston. Sensor consist of a hall effect sensor on a PCB, a clevis pin holder made of PLA....
    Read more »

  • More Live Prediction Tests

    TURFPTAx04/04/2023 at 12:56 0 comments

    The field of wearable technology and human-machine interfaces has made significant strides over the past few years. The development of ‘Open Muscle,’ an innovative system that utilizes 12 pressure sensors built with hall effect sensors and magnets with springs in between, has proven to be a game-changer in accurately predicting finger movements. This article explores the benefits of using a random forest regressor to enhance the capabilities of the ‘Open Muscle’ system, the advantages of the LASK system, and the potential for future hardware optimizations.

    The Power of Random Forest Regressors

    Random forest regressors, an ensemble learning technique, have been shown to yield exceptional results in a wide range of applications, from medical diagnosis to financial forecasting. In the context of predicting finger movements based on pressure sensor samples around the forearm, random forest regressors offer several key benefits:

    1. Robustness to noise: Given that the initial tests of the ‘Open Muscle’ system were conducted without applying any filters or hardware optimizations, the data collected might be noisy. Random forest regressors are known for their resilience to noise, which makes them an ideal choice for this application.
    2. Handling high-dimensional data: With 12 pressure sensors producing a large volume of data, random forest regressors can manage this high-dimensionality without compromising prediction accuracy.
    3. Reduced overfitting: By averaging multiple decision trees, random forest regressors are less prone to overfitting and can generalize better to new data.
    4. Feature importance: Random forests can rank the importance of features, helping researchers identify the most relevant pressure sensors and refine the hardware configuration accordingly.

    The LASK System: Enhancing Labeling and Efficiency

    The LASK system, which applies the labels to the pressure sensors, plays a crucial role in improving the ‘Open Muscle’ system’s performance. By using force measurements instead of movement alone, the LASK system provides a more reliable and accurate output for the machine learning model. This approach not only speeds up the training process but also enhances the overall efficiency of the system.

    Exploring Future Optimizations

    While the ‘Open Muscle’ system has shown promising initial results, there is significant potential for future improvements through hardware optimizations and the exploration of alternative machine learning algorithms. Some possible avenues for enhancement include:

    1. Implementing filters to minimize noise and improve signal quality.
    2. Optimizing hardware configurations based on feature importance from the random forest regressor.
    3. Comparing random forest regressors with other machine learning algorithms, such as support vector machines or neural networks, to evaluate performance and prediction accuracy.

  • Detecting Finger Movement Live with only 5 min of trainig data

    TURFPTAx03/29/2023 at 00:36 0 comments

    I was able to get live predictions with only 5 min of training data with the sensor system.

    The accuracy is lower but I just wanted to make sure that I had done everything correct.

  • Detecting Finger Movements with bracelet and Machine Learning

    TURFPTAx03/26/2023 at 23:38 0 comments

    We are excited to share our latest findings in predicting finger movement and pressure using machine learning. The results show that our model is capable of predicting the finger movement within a Mean Absolute Error (MAE) of 25, which is a sufficient level of accuracy for detecting both the finger movement and the pressure applied.

    Taken from the models prediction. 3-26-2023

    These screenshots showcase a portion of the data file available for download, which contains the actual and predicted finger movement and pressure values. Our model not only indicates that a finger is moving but also estimates the amount of pressure being applied, providing valuable insights into the intricacies of finger movements.

    This achievement opens up new possibilities for applications that require precise finger movement and pressure detection, such as in rehabilitation therapy, robotics, and gesture-based user interfaces.


    We invite you to download the full data file and explore the results in more detail. As we continue to refine our model and improve its accuracy, we look forward to discovering new ways to utilize this technology for the betterment of various fields and industries.

    All data to train the model and code available on our Github: https://github.com/turfptax/openmuscle

  • Random Forest Regressor Model Predicted Well

    TURFPTAx03/26/2023 at 22:08 0 comments

    The trained model had a Mean Average Error (MAE) value of approximately 25, equating to a 12% error rate. Despite the challenges, the model was able to make real-time predictions, demonstrating its potential for a significant impact on the prosthetic industry.

    Predicted Vs Actual from finger 1 or LASK1. The index finger was the most predicted pattern from the open muscle band. Here you see the Actual data from LASK1 in blue and the predicted output of the LASK1 from the model in orange. These are just preliminary data. Everything is published live on the gGitHub The model is 1.7GB so we are not providing the model but all the code is available to train the model with this exact dataset on the gGitHub From attachment predictions_vs_actuals2.csv

    GitHub Link for Open Muscle: https://github.com/turfptax/openmuscle

    predictions_vs_actuals2Download

  • OM12 Updated: Replaced Clevis Pins with Foam!

    TURFPTAx03/14/2023 at 18:04 0 comments

    I recently replaced all of the springs and clevis pins with a foam solution that produces similar results. A lot of the force goes into the elastic band and the depth of the ADCs is very important. With the ESP32-S2 I am able to obtain 1200 samples per second across all sensors or 100 samples per second for each sensor.

    The resulting signals are not as 'loud' as I wished them to be but they are reading the movement of the muscles underneath the skin. Will be posting a video update with the pros/cons of this system for use in biometric muscle contraction detection of forearm muscles.

View all 8 project logs

Enjoy this project?

Share

Discussions

TURFPTAx wrote 05/05/2023 at 16:12 point

We can accurately predict fingers with 20 min of training data. This is very exciting!!!

  Are you sure? yes | no

TURFPTAx wrote 03/26/2023 at 23:40 point

Excited to train the first model. It ended up being 1.7 GB in size but I was able to get decent timing on predicting new inputs. 
All the files to train your own model are updated on the github and will be here on hackaday shortly

  Are you sure? yes | no

yOyOeK1 wrote 03/16/2023 at 19:37 point

Can I come join to make some nice telemetry / control panel for it for tablet / phone.

  Are you sure? yes | no

TURFPTAx wrote 03/16/2023 at 20:45 point

Yes you can. The next iteration has this feature as the primary objective actually. Should be able to have users provide the biometric data via phone/tablet.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates