Project Logs

Collapse

• Audio input for 20 cents USD

  Jon Nordby • 02/25/2024 at 15:40 • 0 comments

  TLDR: Using analog MEMS microphone with an analog opamp amplifier, it is possible to add audio processing to our sensor.
  The added BOM cost for audio input is estimated to be 20 cents USD.
  A two-stage amplifier with software selectable high/low gain is used to get the most of the internal microcontroller ADC.
  The quality is not expected to be Hi-Fi, but should be enough for many practical Audio Machine Learning tasks.

  Ultra low cost microphones

  The go-to options for a microphone for a microcontroller based system are digital MEMS (PDM/I2S/TDM protocl), analog MEMS, or analog elecret microphone.

  The ultra low cost microcontrollers we have found, do not have pheripherals for decoding I2S or PDM. It is sometimes possible to decode I2S or PDM using fast interrupts/timers or a SPI pheriperal, but usually at quite some difficulty and CPU usage. Furthermore, the cheapest digital MEMS microphone we were able to find cost 66 cents. This is too large part of our 100 cent budget, so a digital MEMS microphone is ruled out.

  Below are some examples of analog microphones that could be used. All prices are in quantity 1k, from LCSC.

  MEMS analog. SMD mount

  • LinkMEMS LMA2718T421-OA5   0.06 USD
  • LinkMEMS LMA2718T421-OA1    0.08 USD
  • Goertek S15OT421-005              0.09 USD
  • CUI CMM-2718AT-42316-TR        0.47 USD

  Analog elecret. Capsule

  • INGHAi GMI6050  0.09 USD
  • INGHAi GMI9767  0.09 USD

  So there looks to be multiple options within our budget.

  
  The sensitivity of the MEMS microphones are typically -38 dBV to -42 dBV, and have noise floors of around 30-39 dB(A) SPL.

  Analog pre-amplifier

  Any analog microphone will need to have an external pre-amplifier
  to bring the output up to a suitable level for the ADC of the microcontroller.

  An opamp based pre-amplifier is the go-to solution for this. The requirements for a suitable opamp can be found using the guide in Analog Devices AN-1165, Op Amps for MEMS Microphone Preamp Circuits.
  

  The key criteria, and their implications on opamp specifications, are as follows:

  • achieve the neccessary gain (Gain Bandwidth Product) - not introduce noise (Input Noise Density)
  • flat frequency response (Gain Bandwidth Product)
  • not introducing too much distortion (Slew Rate, THD)

  Furthermore, it must work at the voltages available in the system, typically 3.3V from a regulator, or 3.0-4.2V from Li-ion battery.

  ADC considerations

  The standard bit-depth for audio is 16 bit, or 24 bits for high-end audio. To cover the full audible range, the samplerate should be 44.1/48 kHz. However, for many Machine Learning tasks 16 kHz is sufficient. Speech is sometimes processed at just 8 kHz, so this can also be used.

  Puya PY32V003 datasheet says specify power consumption at 750k samples per second. However, ADC conversion takes 12 cycles, and the ADC clock is only guaranteed to be 1 Mhz (typical is 4-8 Mhz). That would leave 83k samples per second in the worst case, which is sufficient for audio. In fact, we could use an oversampling ratio of 4x or more - if we have enough CPU capacity.

  The ADC resolution is specified as 12 bits. This means a theoretical max dynamic range of 72 dB. However, some of the lower bits will be noise, reducing the effective bit-depth. Realistically, we are probably looking at an effective bitrate between 10 bit (60 dB) and 8 bit (42 dB). Practical sound levels at a microphone input vary quite a lot in practice. The sound sources of interest may vary a lot in loudness, and the distance from source to sensor also has a large influence. Especially for low dynamic range, this is a challenge: If the input signal is low, we will a have poor Signal to Noise Ratio, due to quantization and ADC noise. Or, if the input signal is high, we risk clipping due to maxing out the ADC.

  Finding the gain

  The gain is a critical...

  Read more »

• Board bringup of first prototypes successful

  Jon Nordby • 02/25/2024 at 12:25 • 0 comments

  First prototype boards arrived this week. 

  In the weekend I did basic tests of all the subsystems:

  • Charger/regulator. Voltage levels
  • Microphone+amp. Gain, noise
  • Microcontroller. Flashing, toggle GPIO pin - BLE module. I2C communication.
  • Accelerometer. I2C communication
  • LEDs. Blink

  
  
  As always with a first revision, there are some issues here and there. But thankfully all of them have usable workarounds. So we can develop with this board.

  Examples of issues identified:

  • LEDs are mounted the wrong way. Flip them or use external pins on header
  • Battery charger 4.2v is too high for BLE module and MEMS mic. Use external 3.3v regulator
  • MEMS mic did not work. Use external elecret mic

  Next step will be to write some more firmware to validate more in detail that the board is functional. This includes:

  • Driver for Holtek BC7161 BLE module (I2C)
  • Driver for ST LIS3DH accelerometer (I2C)
  • ADC readout for audio input

• Development board sent to production

  Jon Nordby • 02/11/2024 at 11:09 • 0 comments

  I made an initial development board. This supports both sound-based and accelerometer-based ML tasks. As well as using the LEDs as a color detector. So this is intended to be used to develop and validate the tech stack. And then further cost-optimization will happen with later revisions.
  
  These are the key components
  

  • Microcontroller. Puya PY32F003
  • BLE beacon transmitter
  • Accelerometer. ST LIS3DH
  • Microphones. Top-port or bottom-port MEMS, or external electret capsule
  • Battery charger for LiPo/Li-Ion cells
  • USB Type A connector for power/charge

  Using a pre-built and FCC certified module for Bluetooth Low Energy, the Holtek BM7161.
  This is a simple module based around the low cost BC7161 chip.

  An initial batch of 10 boards have been ordered from JLCPCB.

  
  Also did a check of the BOM costs. At 200 boards, the components except for passives cost

  • With microphone:      0.66 USD per board
  • With accelerometer:  0.825 USD per board

  Additionally, there are around 20 capacitors, 1 small inductor, and 20 resistors needed.
  This is estimated to be between 0.15 - 0.20 USD per board.
  So it looks feasible to get below the 1 USD target BOM, for as low as 200 boards.

  Also designed a small 3d-printed case, with holes for the microphone and LEDs / light sensor.
  

• Activity Recognition using accelerometer with tree-based ML models

  Jon Nordby • 01/21/2024 at 22:33 • 0 comments

  Summary/TLDR

  This looks to just-barely-doable on the chosen microcontroller (4 kB RAM and 32 kB FLASH).
  Expected RAM usage is 0.5 kB to 3.0 kB, and FLASH between 10 kB to 32 kB FLASH.
  There are accelerometers available that add 20 to 30 cents USD to the Bill of Materials.
  Random Forest on time-domain features can do a good job at Activity Recognition.
  The open-source library emlearn has efficient Random Forest implementation for microcontrollers.
  

  Applications of Activity Recognition

  The most common sub-task for Activity Recognition using accelerometers is Human Activity Recognition (HAR). It can be used for Activities of Daily Living (ADL) recognition such as walking, sitting/standing, running, biking etc. This is now a standard feature on fitness watches and smartphones etc.

  But there are ranges of other use-cases that are more specialized. For example:
  

  • Tracking sleep quality (calm vs restless motion during sleep)
  • Detecting exercise type counting repetitions
  • Tracking activities of free-roaming domestic animals
  • Fall detection etc as alerting system in elderly care

  And many, many more. So this would be a good task to be able to do.
  

  Ultralow cost accelerometers

  To have a sub 1 USD sensor that can perform this task, we naturally need a very low cost accelerometer.

  Looking at LCSC (in January 2024), we can find:

  • Silan SC7A20  0.18 USD @ 1k
  • ST LIS2DH12   0.26 USD @ 1k
  • ST LIS3DH       0.26 USD @ 1k
  • ST LIS2DW12   0.29 @ 1k

  The Silan SC7A20 chip is said to be a clone of LIS2DH.
  

  So there looks to be several options in the 20-30 cent USD range.
  Combined with a 20 cent microcontroller, we are still below 50% of our 1 dollar budget.

  Resource constraints

  It seems that our project will have a 32-bit microcontroller with around 4 kB RAM and 32 kB FLASH (such as the Puya PY32F003x6). This sets the constraints that our entire firmware needs to fit inside. The firmware needs to collect data from the sensors, process the sensor data, run the Machine Learning model, and then transmit (or store) the output data. Would like to use under 50% of RAM and FLASH for buffers and for model combined, so under 2 kB RAM and under 16 kB FLASH.

  Overall system architecture

  We are considering an ML architecture where accelerometer samples are collected into fixed-length windows (typically a few seconds long) that are classified independently. Simple features are extracted from each of the windows, and a Random Forest is used for classification. The entire flow is illustrated in the following image, which is from A systematic review of smartphone-based human activity recognition methods for health research.
  
  

  This kind of architecture was used for in the paper Are Microcontrollers Ready for Deep Learning-Based Human Activity Recognition? The paper shows that it is possible to perform similarly to a deep-learning approach, but with resource usage that are 10x to 100x better. They were able to run on Cortex-M3, Cortex-M4F and Cortex M7 microcontrollers with at least 96 kB RAM and 512 kB FLASH. But we need to fit into 5% of that resource budget...

  RAM needs for data buffers

  The input buffers, intermediate buffers, tends to take up a considerable amount of RAM.  So an appropriate tradeoff between sampling rate, precision (bit width) and length (in time) needs to be found. Because we are continiously sampling and also processing the data on-the-run, double-buffering may be needed. In the following table, we can see the RAM usage for input buffers to hold the sensor data from an accelerometer. The first two configurations were used in the previously mentioned paper:
  

  					samples	size	percent
  buffers	channels	bits	samplerate	duration
  2.00	3	16	100	1.28	128	1536	37.5%
  				2.56	256	3072	75.0%
  		8	50	1.28	64	384	9.4%
  				2.56	128	768	18.8%
  1.25	3	8	50	2.56	128	480	11.7%

  16 bit is the typical full range of accelerometers, so it preserves all the data. It may be possible to reduce this down to 8 bit with sacrificing much performance....

  Read more »

• Ultra low cost microcontrollers

  Jon Nordby • 01/20/2024 at 01:01 • 0 comments

  If the complete BOM for sensor is to be under 1 USD, the microcontroller needs to be way below this. Preferably below 25% in order to leave budget for sensors, power and communication.
  

  Thankfully, there have been a lot of improvements in this area over the last years. Looking at LCSC.com, we can find some interesting candidates:

  • ST STM32G030F6P6. 32 kB FLASH / 8 kB RAM. `0.30 USD @ 1k`
  • WCH CH32V003. RISC-V. 16KB FLASH / 2KB RAM.  48MHz QFN-20. `0.15 USD @ 1k`
  • Puya PY32F003x6. 4 kB RAM / 32 kB FLASH. `0.13 USD @ 1k`.
  • Puya PY32F002. Cortex M0+. 20 Kb FLASH / 3 kB RAM. 24 Mhz `0.10 USD @ 1k`
  • Padauk PFS154. 2 kB FLASH / 128 bytes of RAM, `0.06 USD @ 1k`
  • Fremont Micro Devices FMD FT60F011A-RB. 1kB FLASH / 64 bytes RAM. `0.06 USD @ 1k`.

  There are also a very few sub-1 USD microcontrollers that have integrated wireless connectivity.

  • WCH CH582F. Bluetooth Low Energy. `0.68 USD @ 1k`

  Implications for 1 dollar TinyML project

  It looks like if we budget 10-20 cents USD to the microcontroller, then we get around:

  • 16-20 kB FLASH
  • 2-4 kB RAM
  • 24-48 Mhz clock speed
  • 32 bit CPU
  • No floating-point unit (FPU)

  At this price point the WCH CH32V003 or the Puya PY32F003x6 look like the most attractive options. Both have decent support in the open community. WCH CH32 can be targetted with cnlohr/ch32v003fun and Puya with py32f0-template.

  What kind of ML tasks can we manage to perform on such a small CPU? That is the topic for the next steps.
  

View all 5 project logs