Close
0%
0%

Less than $100, high FOV, mobile AR glasses

Augmented Reality glasses using modified lowcost Android phones

Similar projects worth following

[Also be sure to check out www.tiltfive.com AR glasses with huge potential from Jeri Elsworth!]

[Update 28 april: I'm willing to opensource the entire framework (tracking, warping, etc.) but it's a whole lot of complicated undocumented code, so I'm afraid it won't help. If anyone is truly interested in producing glasses I'm happy to continue work and transfer my knowledge. So far the off the shelf costs seem to be $110 for the phone (VKWorld S8), the glasses frame $35,  and some production (2 solder points on phone for camera flash, perhaps 3 solder points to reroute battery to external and extracting front camera from case or purchase it extra for $15~). Mass production would be cheaper I assume. Not looking for funding, but am looking to keep it opensource if its the case. Contact at kajos.net]

This is a project to achieve very low cost Augmented Reality glasses.

Some unique characteristics for this project are:

- Custom positional tracking using reflective markers

      - that allows for long tracking range above what is possible using QR codes

      - that is extremely lightweight, and runs on low performance processors such as MediaTek

      - works exceptionally well in low light conditions

      - allows for low battery drain

      - reflective QR codes allow for data to be attached to a marker. When a reflective blob is detected, a high resolution photo is taken so that the QR data and orientation can be read from a large distance.

- Using cheap, off the shelf hardware; namely Chinese imported MIX phones.

     - which cost currently $70 and are readily available

     - with a range of alternatives available

     - in the future, other phones can be supported. MIX phones are used solely for their long flex cables leading to the front camera.

     - alterations to the phone are simple and low cost

     - the frame for the AR glasses are Chinese imported for a cost of $30~. The FOV is about the same as the camera, 60-70 degrees.

- Tracking is implemented using an Android service. There is no need to build the tracker with AR apps. There is seamless transition between apps because of this as well. The Android tracker service will also have the ability to put the device in auto standby on no input, or wake the device up on movement or marker detection.

- There is no need to root the Android device. The requirements for Android level are low and should work with any Android phones available.

- Unity3D support is available and working smoothly.

- Maintains 60fps

  • 1 × MIX smartphone MIX smartphones have the front camera at the bottom, with a long flex cable going to it. Currently I'm using the VKWorld S8, which is a little more than $100. I have also the Leagoo KICAA Mix, which is $70, but haven't tested it yet.
  • 1 × 1 or 3 watt LED Currently using a white LED and a colored marker. Going to switch to a colored (green or blue) LED and colorless marker.
  • 1 × Reflective marker Again, needs to be colored if the flash is white (preferably at least).
  • 1 × AR glasses frame https://nl.aliexpress.com/item/Holografische-Effecten-Smart-AR-Doos-Augmented-Reality-Bril-Helm-3D-Virtuele-Comfortabele/32951237253.html
  • 1 × Resistor for coupling with the LED I'm using 300ohm with (I think) a 3 watt LED.

  • Fisheye lens

    kvtoet02/18/2019 at 16:13 0 comments

    Feeling sick today, but found some time to make a bracket to attach a fisheye lens.

    The way the glasses are designed there's only two simple ways to mount the camera: straight forward or in a 45 degree angle. Since the marker is usually on the floor or a table, it is best to have the camera aimed slightly downward. Although 45 degrees is too much, this would be improved in a custom frame. Ofcourse also the design would be improved to merge the lens with the headset more.

    Haven't yet integrated it in software, but this won't be much of a problem and I'll probably get around to it quickly.

    What does it mean? Fisheye cameras will allow for a much wider tracking camera close to 180 degrees. However I'm not sure QR code reading will go fluently without distortion correction, which could be a hit on performance. So I'm contemplating foregoing QR code support (for now). Next to that, the increased FOV will mean the marker will show up occupying less pixels, so either I need to increase the size of the marker, or increase the camera resolution (which would also be a performance hit).

  • First concept model

    kvtoet02/18/2019 at 11:53 0 comments

    A little update; currently working on the BOM and for that I needed to build a 3D model to estimate costs for the glasses frame.

    My plan is to go for a more VR goggle style design. They are easy to manufacture and have proven their ergonomic comfort to be reliable. The current glasses I use now miss the nose bridge and tend to slide down.

    This is a rough sketch of the design I'm aiming for:

    Note: the lenses are there for demonstration purposes but are not part of the design.

    Some changes will be made; such as the side supports will be rounded to block out less of the users view, the eye holes will be joined, the nose bridge will be shaped better and more. This is a rough sketch of the design.

    I've also found a supplier for the lenses that allows me to remain in budget.

  • Todo

    kvtoet02/10/2019 at 19:20 0 comments

    Hardware:

    - Create birdbath AR lenses for a good price (hard)

       - Perhaps using a dental vacuum form machine

    - Design and lasercut a frame for the lenses

    - Find a place to create flex ribbon extenders for a good price (hard)

    - Create reflective QR codes

    Software

    - Take high-res picture on blob find and read QR code

    - Create homescreen in Unity to start apps

    - Create some demos in Unity

    - Support for 3Dof controller - or 3Dof controller using additional phone

    - More testing and tweaking in high light conditions

    - Write and tweak vertex warping shader

  • Intro to reflective marker tracking

    kvtoet02/09/2019 at 16:31 0 comments

    A short explanation of the positional tracker that I've made and that is used in these glasses.

    The technique used are reflective markers with low exposure camera setting. 

    Basic steps are these:

    - Calibrate the phone once by shining the LED (which is positioned next to the camera) on a piece of retro reflective material close up to the camera. This will put the camera in low exposure setting when auto-exposure is enabled. Then lock the exposure.

    Some phones allow the exposure to be controlled directly without the use of auto-exposure. However this has become default only on later, higher end models of phones.

    This calibration needs to be done once every reset, or after the camera has been released from the exposure lock.

    - The camera feed is picked up from an OpenGL context, using a camera texture (GLES11 extension, available on all phones). A shader is ran over the camera input, comparing the colors in the camera texture with the color of the flash LED (or marker color). All other colors are filtered out. What remains are a few blobs. 

    - The remaining image is a black and white binary image, which is bitpacked using another shader, in a smaller image. Finally this much smaller image is downloaded to CPU with GLReadPixels.

    - From these blobs, some might be static part of the scenery, such as a lamp or other object. The tracker remembers all objects, then flashes the LED briefly and identifies which objects disappeared and which stayed present. There are additional filters, but this is it in a nutshell.

    - The blobs that disappeared are reflective blobs, and thus have a high likelihood of being retro reflective markers.

    - The distance is taken from the radius of the marker circle, in a nutshell. 

    - Lastly a high resolution photo is taken so the QR code printed on the marker can be read out and the orientation of the marker can be determined. This step isn't fully integrated yet. (The QR code reader is implemented, and the POSIT algorithm for the orientation has been written and tested, but not yet all brought together in a working package). Right now, the object orientation is taken from the gyroscope and accelerometer.

    Some clear upsides of this type of tracking are:

    - Very minimal impact on performance. The biggest performance hit is a single iteration of the binary camera image on CPU, to do blob finding. 

    - Process doesn't get much heavier with more blobs. Scales well.

    - Latency is low as the algorithm finishes quickly.

    - Works exceptionally well in low light conditions, as reflective markers show up brightly and environment dimly. High light environments still need some extra work, but achievable (depends on brightness of LED too).

    - No camera calibration required.

    - Works well with low resolution cameras.

    - Long range tracking. Longer range can be achieved by increasing the marker size too. Much better than sole QR code tracking.

    - Solid tracking; it can stand shaking the camera (low impact by smearing) and finds the marker in <200ms from a dead drop point.

    - The use of markers works well with multiplayer AR. SLAM has difficulty with this.

  • Bringing the price down to $50

    kvtoet02/09/2019 at 11:48 0 comments

    Given the cost of the AR glasses can be brought down to below $20, there is still the phone hardware left. Currently the phone costs at least $70 (for the Leagoo KICAA MIX).

    However there are phones on Aliexpress that cost $30.

    The differences between these price ranges can be seen in:

    - the resolution ($70 gets you 1920x1080 or similar, $30 is 960x480~)

    - the processor (Octacore 1.5ghz or Quadcore 1.3ghz, both MediaTek).

    The cheaper ones have benefits:

    - the screen is 4-5 inch. Currently the AR glasses don't make use of the entire screen with the bigger, expensive phones. Keeping a larger screen on costs extra battery. With custom lenses a higher resolution phone could have the benefit in pixel/degree, but currently not very much the case.

    - a smaller screen allows for smaller AR glasses.

    - weight is much lower. A 3000mah battery adds a lot of weight.

    - the quad core CPU is probably more efficient. I think they're both on the same nm die size.

    I'm fairly certain I can make the positional tracker work fine on the quad core CPU, perhaps with a somewhat lower camera resolution if needed.

    What is left is that the cheaper phones have regular front cameras. The phones that are used now have long flex ribbons leading to the camera by design; the $30 phones do not. Thus a flex ribbon extender needs to be made. In my research I found this can be a hassle, due to custom connectors being used which can be hard to source, more so out of China. If anyone has any ideas, I'd like to hear them.

  • AR frame

    kvtoet02/09/2019 at 11:38 0 comments

    Creating cheaper and better AR glasses frames

    The AR frames I'm using at the moment are these from Aliexpress. There are a number of downsides to these glasses:

    - The place where the camera is mounted is at the wrong angle. A small bracket correcting the angle needs to be placed on top so that the camera has better overlap with the screen FOV.

    - The glasses do not rest on the nose. This is a missed opportunity, as this would allow the glasses to be mounted much more easily. Right now finding the sweet spot is a hassle to new users. Also resting the glasses on the nose allows the weight to be distributed better. 

    - The glasses are quite bulky. 

    - The price is dropping but could be cheaper.

    - The glasses are sensitive to head shaking, because there is no nose rest and the strap is thin.

    I've been thinking about building my own glasses in low quantity. My current idea is to:

    - laser cut the frame, and aim for a more VR glasses design.

        - the glasses rest on the nose, and a wide strap for the head is used so that head shaking is not a problem.

        - this would be cheap to about $5 per frame

    - for the lenses it's trickier. I'm currently contacting Chinese vacuum forming services to quote prices, but I predict this might still be pricey in low quantities. I'm thinking about trying to use a dental vacuum form device to manufacture small lenses, although these would be missing the coating, which would lessen stray light. This would come at $1 per lens/eye, so $2 total, and no costs in setup.

View all 6 project logs

  • 1
    Finding and modifying a suitable phone

    Required is a phone with a long flex cable leading to one of the cameras. In my research Chinese MIX smartphones are most suited. These have flex cables of ~10cm to the front camera (which is located at the bottom of the phone, instead of the top). You might have to look if spare camera parts are available, as it may be easier to buy a $10 spare camera than to disassemble the phone (more than necessary) and risk damaging it.

    My first successful test has been done with the VKWorld S8. This phone has a rather large battery and places a significant weight on your head, but does provide for a long battery lifetime. I have ordered a spare front camera so I didn't need to unearth the original front camera from below the battery pack, saving some hassle.

    My second test will be with the Leagoo KICAA MIX. This phone has the front camera flex cable running over the top of the battery, making it easy to detach the front camera without taking the battery out.

    Lastly the postional tracker I have developed uses the camera flash to detect reflective markers. Thus the camera flash LED needs to be ran outside the phone to be positioned next to the external camera. Some soldering is required. The original SMD LED needs to be desoldered and two long wires need to be attached to the points. A 1 or 3 watt wide angle LED is used and attached to the wires. I have placed a 300ohm resistor in series to dim the light as to be not too bright.

  • 2
    Ordering parts

    Needed are:

    - Chinese AR glasses Link

    - A 1 or 3 watt LED (note a colored LED works well with clear reflective markers, otherwise a white LED with colored markers works also)

    - A MIX phone with long flex cable to the front camera For example this one

    - Reflective markers Such as these to test with

        - In the future custom reflective QR markers will be used

  • 3
    Calibrating

    Software and hardware needs to be calibrated:

    - The correct camera resolution needs to be specified

    - The correct camera angle in relation to the display needs to be specified

    - The phone angle in relation to the eyes needs to be specified

    - The display angle needs to be specified

    Once thats done (or done for you), a short calibration step needs to be done in order to put the camera into low-exposure mode. This needs to be done at least once every phone-reset. Currently the tracker service auto calibrates to the marker/LED color and light levels. It also times the intervals of the flash LED toggling - as this is not immediate and can vary per phone.

View all 3 instructions

Enjoy this project?

Share

Discussions

kvtoet wrote 04/07/2022 at 15:21 point

I'm looking for Penelope!

  Are you sure? yes | no

tjf1257 wrote 05/25/2019 at 23:40 point

Would you be willing / able to share the CAD file for the preliminary design?  Trying a project of my own and the geometry of the exterior reflector would be a major help.  Best of luck with your project, it's super cool!

  Are you sure? yes | no

kvtoet wrote 08/11/2019 at 19:59 point

Sorry for this late reply, but hackaday emails end up in my spam apparently, I have to fix that. I wouldn't work of that CAD, it was really a quick mashup and the ratios are probably wrong. On the better news; I found out that the seller of the glasses is open to selling the lenses separately, at lower price. I think it was $15 or so. I would recommend that over making your own

  Are you sure? yes | no

tjf1257 wrote 08/11/2019 at 21:42 point

Thanks so much for the info, that's super helpful!

  Are you sure? yes | no

kvtoet wrote 08/12/2019 at 08:53 point

Followup to my previous comment; the lenses that come with these glasses are screwed in place with two screws - and no glue. So if you have trouble buying the lenses separately (because of quantity perhaps) you can buy the entire glasses and detach the lenses easily. Good luck!

  Are you sure? yes | no

emadukhan wrote 02/18/2019 at 21:20 point

Do you have a link to the GitHub for the Unity project used for this ? I want to try this without an android phone. 

  Are you sure? yes | no

kvtoet wrote 02/21/2019 at 14:06 point

@emadukhan My current project doesn't use the Unity VR functionality (and gets matrices from android service), as I find to be a bit broken for usage with Cardboard and the warping being done is inefficient for AR glasses. I digged up an older project and uploaded that though: https://github.com/Kjos/CardboardARGlassesUnity . This one did use the Unity VR functionality (with Cardboard SDK). Haven't tested with iOS, but could work. QR codes that I created using the online profile generator are in the root. This is a video of it working from a while back: https://www.youtube.com/watch?v=h8LderdiN80 I stripped some of the assets though, for copyright reasons. The project is a bit messy, was just a prototype.  

Edit: Posted at wrong place, don't think you received my reply before

  Are you sure? yes | no

emadukhan wrote 02/21/2019 at 14:34 point

Got it, Thanks :) 

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates