Close

Lesson Learned : Keep Your IDE Up to Date

A project log for LipSync

An assistive tech which allows quadriplegics to use touchscreen mobile devices using a mouth-operated joystick with sip and puff controls

ivanIvan 09/28/2016 at 22:430 Comments

Hey everyone,

So I was working on the microcontroller code and doing some smartphone testing.

I wrote my microcontroller code, for the sake of maintaining my sanity, with two different functions for manipulating the HID mouse. One function is used when the LipSync is connected to a device via Bluetooth because the Bluetooth HID protocol requires specific bytes of data for the HID mouse profile. The other function is used when the LipSync is connected to a device via USB. In this instance, the HID mouse functions integrated into the Arduino Micro are used because they are straight-forward and easy to use.

That being said, I had used the LipSync with my smartphone (Samsung S5) via Bluetooth to open the messaging app and type out text messages. Two days ago I tried the same operation with the LipSync but connected to my smartphone via USB. I opened the messaging app and tried to type a text message but the on-screen keyboard was not popping up?

I checked my "Language and Input" settings on my smartphone and lone behold, my phone was recognizing the LipSync, but more specifically, the Arduino Micro as an external keyboard...

Things were starting to make sense now. My phone - for good reason - was not opening the on-screen keyboard because it "views" an external keyboard connected to it so the text can be typed out there. Okay cool.

I knew that the Arduino Micro can be used as a HID mouse or HID keyboard but presently I only wanted the microcontroller to be a HID mouse. After doing some research I found out that you cannot suppress the HID keyboard descriptor without changing the Arduino HID library for the Arduino Micro - or so I thought...

I successfully found a workaround for suppressing the HID keyboard functionality of the Arduino Micro but adding two lines of code and commenting out one line in the Arduino Leonardo/Micro HID library. This would be a cumbersome task not for changing the code but if makers in the future had to do this I suspect it would take them more time to find the correct library and install an appropriate text editor than to actually make the changes to the library which is just a c++ file (HID.cpp to be exact).

THANKFULLY my team member @Milad heard about this and looked at my microcontroller code and asked why I wasn't using #include <Mouse.h> in my code? My code seems to work without it, so I tried to compile and run my microcontroller code with the header file included - the IDE threw errors...

I tried to do the compile on an Arduino Uno as well just to double check - errors were thrown. So naturally I reversed my modifications to the HID library and tried to replicate the bug. I did and I even recorded it...

I also recorded what happens with the modifications to the HID library...

Well our problem was fixed with 2 lines of code and 1 commented out line of code...OR SO I THOUGHT!!!

The next day I met with @Milad and we looked over the code. His code compiles with the Mouse.h header file but mine doesn't, why? BECAUSE MY IDE WAS WAYYY OUT OF DATE!!! I was using Arduino IDE 1.6.5...Milad was using 1.6.12 - the most up to date version.

Long story short, I added one line to my microcontroller code "#include <Mouse.h>" and now while the LipSync is connected to my phone via USB, the mouse cursor works, the on-screen keyboard can be activated and my phone does not recognize the microcontroller as an external keyboard - SUCCESS!

As a side note - Milad and I discussed plans for more advanced functions with the LipSync, specifically Android keyboard shortcuts that may be able to be executed through the Arduino Micro (volume, power, etc) but alas when you add "#include <Keyboard.h>" my phone, again with good reason, recognizes the microcontroller as an external keyboard and the on-screen keyboard fails to activate.

Overall, I am glad we caught this potential problem before other makers started making these LipSyncs.

I have committed the revised microcontroller code with modified cursor speeds based on our first user test earlier this week.

Bye for now!

-Ivan

Discussions