Close
0%
0%

LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

Similar projects worth following
Touchscreen devices have become a staple piece of technology in this day and age. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. We are releasing all of our work open-source, to make the Lipsync a solution that can be made at the community level for less than $300.

OVERVIEW

Smartphones are becoming a staple piece of technology for many citizens in North America and around the World. Smartphones and touchscreen devices enable users to navigate around their city better, to communicate with others more freely, and to operate applications which can promote happy and productive lives. Currently, there exist a limited number of practical devices for people with limited hand function such as quadriplegics to use touchscreen devices - this is where we step in.

The LipSync is a mouth-operated alternative input device that enables people with limited hand function to use smartphones, tablets, and computers. It is primarily intended to be a low-cost, portable option for enabling access to devices with a touchscreen. The device is operated by moving the mouthpiece joystick with the lips or teeth and  Moving the mouthpiece joystick with the lips or teeth will move a mouse cursor and applying sips or puffs on the mouthpiece straw will input a left click, right click, or tap.

COMPATIBILITY AND VARIANTS

The Lipsync will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB) or a Bluetooth connection. Specifically, the LipSync is compatible with smartphones (Android 4.0+, iOS 13+, Windows), tablets (Android 4.0+, iOS 13+, Windows) and computers (Windows, MacOS, Linux). There are now several versions available:

  • The Lipsync emulates a USB mouse.
  • The LipSync Wireless emulates a Bluetooth mouse.
  • The Lipsync Gaming emulates a USB joystick or USB gamepad. It is compatible with the Xbox Adaptive Controller and any desktop or laptop.
  • The LipSync Macro emulates a USB keyboard. It can be used with iOS 12 and older devices to provide switch access through the accessibility features. 

The LipSync Switch Input Module can be used to modify one of the above versions so that a user can use switches instead of sip and puff. This can be a useful option for those unable to form their lips around mouthpiece or apply the positive / negative pressure to generate sips and puffs.  The Switch Input module enables two assistive switches to be connected via 3.5 mm jacks to use the sip and puff functions.

GETTING A LIPSYNC

The LipSync is an open-source hardware project where all of the 3D printer files, component lists and microcontroller code are public. In the spirit of accessibility, the housing can be 3D printed, the electronic components are readily available, and the assembly is as straightforward as possible.

There are a couple of options to obtain a LipSync:

1. You are free to download the files and build a LipSync for yourself or someone you know. 

2. You can submit a build request through the Makers Making Change website. In this case, we'll try to pair you with a volunteer maker to complete the build, and you will reimburse them for the cost of materials. 

LIPSYNC PROJECT GOALS

  • Oct-Dec 2016: refine designs and usability with user testing - DONE
  • Jan-Mar 2017: host a “buildathon” where students, makers and volunteer teams will form around people with disabilities to build and mount LipSyncs - DONE
  • Apr-Dec 2017: Support a network of makers to connect directly with people with disabilities and creating 150 LipSyncs through the Pacific Northwest - DONE
  • Jan-Dec 2018: Scale maker-disability matchmaking service across North America, to support 1,500 LipSyncs being created at the local level, as well as developing and applying other assistive technologies. - DONE
  • In addition, the Neil Squire Society has been working on a service delivery model for the technology. It will soon be launching the Makers Making Change initiative. Makers Make Change has the goal of creating one-on-one relationships between makers and people with disabilities. The Makers Making Change initiative will provide the infrastructure to connect makers, open assistive technologies, and people with disabilities...
Read more »

  • LipSync V3.0

    Makers Making Change09/02/2021 at 20:58 0 comments

    Over the past few months, we've been working on improving the LipSync firmware and incorporating a number of other smaller changes. We've made a number of changes and incorporated feedback from preliminary testing with a small group of LipSync users and we're now ready to share this update with the community in an open Beta.

    You can find the updated files on Github here.

    Here are a few of the big changes:
    1. Reduced cursor drift

    2. Application Programming Interface

    3. Additional Mounting Options

    1. Reduced cursor drift.

    The force sensitive resistors (FSRs) that make up the joystick are notoriously finicky and the reported values shift over time which can cause the cursor to move unexpectedly. We've added some code to better differentiate purposeful movement from this sensor drift so the cursor should only move when you want it to.

    2. Application Programming Interface (API)

    We've added an API so that majority of settings can now be changed through a serial interface rather than requiring a change to the code. We've had preliminary success with a proof of concept Android App and ultimately this should allow a user to change many of the LipSync settings independently. For now, the API can be accessed by manually sending serial commands to the LipSync. The addition of the API also has the added benefit of being able to access real-time sensor values so we can diagnose problems and perform a variety of testing that wasn't possible before.

    3. Additional Mounting Options

    We normally refer to the top of the LipSync as the portion with the LED and the mounting connection on the bottom. Moving the mouthpiece towards the top moves the cursor up on the screen. This provides an intuitive mapping (i.e., a match) between the motion of the mouthpiece and the motion of the cursor if both the LipSync and the screen are mounted with their tops up (i.e., 0 Degrees). In some cases, it is more convenient to mount the LipSync in a different orientation (e.g. upside down, 180 degrees). 

    Previously, mounting the LipSync in a different orientation would cause a confusing mapping between mouthpiece motion and cursor motion since the code continued to treat movement towards the top of the LipSync as up. It was possible to manually change the code to correct the mapping, but this requires someone with the skill to do so and access to a computer.

    Now, the mounting orientation can be changed by changing a single entry in the code, or using the API. This will make it much easier to use and change between different mounting orientations, which will provide additional mounting options.

    Let us know if you test out the updated code and have any feedback. You can reach out here or on our forum

  • Entering in 2020 Hackaday

    Makers Making Change08/31/2020 at 19:30 0 comments

    Entering in this year's UCPLA contest thread of the Hackaday contest. We are also very excited to see other entries in looking at access technology for people with disabilities. LipSync is still a major device for us, but now at www.makersmakingchange.com we have over 90 open source hardware projects on our site, and our forum has many other design challenges submitted by real users with disabilities looking for access solutions. So thank you to all you socially conscious hackers! #UCPLA

  • Updates on LipSync code and resources

    Makers Making Change01/14/2020 at 23:09 0 comments

    Hi everyone, 

    While it has been a while since we have updated our hackaday page, the LipSync lives on. There is some exciting pieces for you, as the LipSync now has a few variants:

    • Original LipSync - emulates a USB mouse
    • LipSync Wireless - emulates a wireless bluetooth mouse
    • LipSync Gaming - emulates a USB joystick / gamepad. This is compatible with XBOX Adaptive Controller and any deskop/laptop.
    • LipSync Macro - emulates a USB keyboard, useful with older versions of iOS that do not have mouse support to provide switch access through the accessibility features.

    All of our resources are on our website, Makers Making Change:

    https://www.makersmakingchange.com/project/lipsync/

    All of our code, 3d files, bill of materials, are all available though that link, as well as our GivHub Repo:

    https://github.com/makersmakingchange

  • Bring Makers Making Change to Your Community

    Makers Making Change02/04/2019 at 23:59 0 comments

    You can bring Makers Making Change to your community in two ways: by becoming a Community Champion or by starting a Community Chapter.

    Sign up as a Community Champion

    • Raise awareness through hosting a presentation at your workplace or in your community
    • Distribute our pamphlet to maker and disability groups in your region
    • Help identify interested parties and user needs
    • Express interest in being a Community Champion here

    Sign up to start a Community Chapter

    • Use your makerspace, classroom or club as a hub for 3D printing and/or building of projects requested from our open source Projects Library
    • Lead events and workshops: engage makers and/or disability professionals in fun meetups that help identify device needs, teach maker skills and build community between these groups
    • Express interest in starting a Community Chapter here

    Our Low-Tech AT Kit, which includes (clockwise from top right): bottle opener, palm pen holder, ball pen holder, key turner, nail clipper

    Shortly after we review your application, we'll send you some documents to read over followed by a quiz. Once you successfully complete the quiz, you will be given access to marketing materials, template emails, presentations, and outreach materials to start making connections in your community.

    Questions? Please reach out to Zee Kesler, Makers Making Change Project Manager, at zeek@neilsquire.ca.

    We appreciate all the help! We couldn't do what we do without volunteers who contribute their time and skills.

  • Switch Survey

    Makers Making Change02/04/2019 at 23:56 0 comments

    Do you use switches or care for someone who does? Or are you a disability professional who works with or prescribes switches? We’d love your feedback on what type of switches we should be working on. We want to know what is important to you in terms of using switches, and what you are looking for in custom switch designs. A variety of open source and commercial switches. Please fill out our Switch Survey: https://tinyurl.com/MMCSurveySwitch. Thank you for your help!

  • Upcoming Events in the US

    Makers Making Change09/20/2018 at 16:47 0 comments

    Over the next couple weeks, we'll be hosting and attending a range of buildathons, maker faires, and workshops in the United States. Each of them offers an opportunity to have fun and learn new things while working together on solutions to benefit people with disabilities.

    Students solderingStudent makers working on LipSyncs at a prior buildathon

    We hope you’ll join us at an event or two if you’re in the area. Click on each event link to learn more and register:

    We look forward to seeing you!

  • AT Makers' Fair

    Makers Making Change09/12/2018 at 22:03 0 comments

    On September 29th, Makers Making Change will be in Concord, NH for the AT Makers’ Fair. This AT-dedicated fair brings together makers of all abilities to share ideas, develop new skills, and enhance innovations for people with disabilities.

    The fair will feature talks by AT maker movement leaders, hands-on workshops, an AT Invention contest, a Make AT Café makerspace, and more.

    Makers working on toy adaptations

    The Make AT Café is a particularly cool space. Participants can make and take home a variety of AT devices using various plastics, wire, specialty tapes, and basic hand tools. Many of the devices address specific challenges; these include low vision devices, holding and gripping devices, toy adaptations, communication access devices, and more.

    In addition, Chad Leaman, Director of Innovation, will be facilitating a workshop called Connecting to Makers and Quality Open AT Projects. The workshop will explore a range of freely available open-source assistive devices and resources, as well as community groups that can help in designing and making assistive devices.

    Register now for the AT Makers’ Fair. The deadline is September 14th. We look forward to seeing you there!

  • We're Hiring a Regional Coordinator in Calgary

    Makers Making Change09/04/2018 at 17:46 0 comments

    Are you actively involved in the maker community in Calgary?

    Are you an electronics and 3D printing whiz?

    We are looking for a Regional Coordinator who will work to integrate Makers Making Change within the community. You’ll love this role if you are a maker who is actively involved in maker culture – someone who is passionate about creating in all aspects of life. You'll interact and work with diverse groups of people, including people with disabilities, organizations, and disability professionals.

    Makers working on LipSync parts

    Makers at a prior LipSync Buildathon, soldering away

    As a Regional Coordinator, you'll help integrate our work into makerspaces and hackspaces, whether at libraries, universities, for-profit or community-run spaces, or schools. In addition, you'll have the opportunity to develop curriculum for middle and high school students, utilizing “design thinking” methodologies.

    Read more about the opening. We look forward to connecting with you!

  • Maker Faire New York

    Makers Making Change08/23/2018 at 20:39 0 comments

    Exactly a month from now, we will be at World Maker Faire New York. You can meet us at our booth throughout the weekend of September 22–23. Learn more and get your Maker Faire tickets here: https://makerfaire.com/new-york/

    Maker Faire New York -- See us there!

    At Maker Faire, I will also be on a panel centred on making assistive technologies for people with disabilities. I'll be joining these brilliant makers:

    • Jon Schull of e-NABLE, a network of volunteers creating free open-source 3D printed prosthetics for children and underserved populations
    • Holly Cohen of DIYAbility, which empowers people to not only to make access solutions for people with disabilities, but to also help people with disabilities be makers
    • Liz Arum of Ultimaker, who is 3D printing resources for blind students

    We’re interested in organizing a small LipSync Buildathon or two in the week after Maker Faire in the New York area and on the East Coast. If you’d like to be involved, please email me at chadl@neilsquire.ca.

    We look forward to seeing you there!

  • Testing, Testing...

    Makers Making Change08/09/2018 at 22:17 0 comments

    We're having a busy and fun time doing a lot of backend work here at Makers Making Change. One of our projects: testing and fixing a whole bunch of LipSyncs to make sure they are functioning brilliantly before being sent to people who need them.

    Completed LipSyncs in a box

    We are glad to have Fawzan’s help in testing these LipSyncs. As you may remember, Fawzan is a high school student who, along with his student group, 3D-printed assistive devices from our library for people with disabilities. Read more about Fawzan’s work.

    You won't hear from us as often in the next few weeks while we work on these projects. In the meantime, you can always give us a shout or get involved on our website. Have a bright rest-of-the-summer!

View all 81 project logs

Enjoy this project?

Share

Discussions

Mark Brady Ingle wrote 07/02/2017 at 22:52 point

is anyone still working on this project?  I need some help and ordering information on the mouth piece parts

  Are you sure? yes | no

Makers Making Change wrote 07/08/2017 at 16:14 point

hey Mark glad we caight up on email..sorry not logging into hackaday as often as I should. 

  Are you sure? yes | no

Mark Brady Ingle wrote 06/27/2017 at 02:50 point

I am testing on Mac and need the instructions for the RED/Green flash sequence.  I have a bluetooth connection established but I am not able to go any further.  Help would be appreci

  Are you sure? yes | no

Trent Robertson wrote 05/30/2017 at 16:00 point

Thank you for sharing this project. One of my best friends has ALS and would benefit from this. I have already set up voice recognition features in her apartment to allow her to control lights, air conditioning, and the TV, but she still needs someone to hold and control a tablet for her if she wants to surf the internet. I hope to find the time to build this soon.

  Are you sure? yes | no

Makers Making Change wrote 07/08/2017 at 16:13 point

hey Trent let us know if we can send you some parts / the main boards!

  Are you sure? yes | no

Mark Brady Ingle wrote 05/09/2017 at 20:53 point

Hello team!  Are the current instructions online the most current?  I have enough parts to get started so I wanted to make sure. 

  Are you sure? yes | no

Makers Making Change wrote 05/11/2017 at 20:20 point

Hi Mark, we have added the most recent as of now.  Let us know if you got questions with the build and in particular ping @Ivan.Gee if you need in particular details

  Are you sure? yes | no

eroy wrote 01/02/2017 at 23:34 point

Thank you for this project! I am endlessly disappointed in the cost of assistive technology. To make something like this open source is a gesture rarely seen. 

  Are you sure? yes | no

Charles G wrote 02/26/2017 at 06:49 point

Thank you for the kind words eroy! Stay tuned in the next couple weeks as we launch a new and improved version of the LipSync at an even lower price point!

  Are you sure? yes | no

Manuel Z wrote 10/10/2016 at 19:18 point

this is an amazing project! congratulations!!!! 

  Are you sure? yes | no

Charles G wrote 10/10/2016 at 23:14 point

Thanks, Manuel!

  Are you sure? yes | no

Sadiq Mohamed wrote 09/22/2016 at 12:02 point

A very worthwhile project. I had a friend with MS who would have benefited from this. Unfortunately he died some years ago. I wish you the best of luck with your testing and look forward to the results.

  Are you sure? yes | no

Ivan wrote 09/22/2016 at 17:55 point

My condolences about your friend @Sadiq Mohamed . I am glad you believe in our project, we are working our tails off over here so we can start getting makers and users together to start building these and help increase people's quality on life. Just as a side note, our plans are to continue compiling and refining open-source assistive technologies, if you have any ideas I would love to hear them.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates