Close
0%
0%

dsp arduino code for filtering analog enviroments

get guaranteed accuracy from a sensor in a noisy environment quickly and consistently from hybrid filtering bandpass example for VL53L0

Public Chat
Similar projects worth following
I have included code for arduino and python that show basics for DSP filtering in code, and how to not only reduce noise, but increase response of filtering. for example if change of single is greater than what is in noise range, then we can adjust to those levels faster. at the same time we can look at wave from different sample points and determine if it is leveling out to within a set percentage and if so determine that the signal is stabilized and ready to reliably sample. the code i provide is more for sampling such as from a Time of Flight Sensor, or any sensor that sees the same data repeatably. it is normal for such devices to have lots of filtering, but this comes at a cost of slow response times. filters in this article speed up samples. there still can be drifts +/- 0.5mm to +/-1mm but it was over 8hr period, so it would need to be over hours or days. i'm looking into that as well as i find that temp being the cause of variation to be unlikely. data in another article!

while working with the VL53L0x i have been dealing with issues with noise and wanted to have a cleaner signal for several measurements at different distances. it was taking up to a minute for the signal to stabilize to a near flat value and repeat values from using a default low pass digital filter

i have included code for the arduino that has a hybrid filter that changes its amount of filtering depending on the rate of change in the signal.  it does not us RC or π math, just averaging and some scaling, and some conversion of floats to longs. there is a function called filternoisereductionbox( ) that can take a noisy signal and output a clean one, it also does not take forever to adjust to change.

the arduino code, can do a few things. it can filter analog inputs, it can respond to changes quickly and when near wave peak it can provide more filtering to reduce noise further, and it can let you know when the signal is stable and no longer changing within a set amount. this is useful for measuring devices, and analog signals where there is a lot of noise, or where accurate measurement of values is needed over a shorter time.

this is really just a fancy band pass filter with a sharp cut off. with measurements there still could be slow drifts that are not accounted for, temp, air moisture level, surrounding static charge, magnetism, moon gravitational pull. just a few out of a million things that effect analog devices. for the TOF flight sensor I am using for my projects i want it to have an accuracy of +/-0.5mm. since it is only able to measure 1mm differences, the noise in the environment will help with fuzzy readings to get +/-0.5mm. 

included below is a program called lowpassfilter.py it does a few things. first it includes actual random noise from a sample of random.org of lightning strike data. it is as random as you can get. noise has been reduced to be less than 10% of signal. in this example filtering is done from a low pass filter that

just takes samples and averages them by 10 samples on green line and by 100 samples on yellow line.

the low pass filter is often used to reduce noise, but at a cost of response times. the more samples are averaged, details are lost if looking for the same speed of measurement. you would normally need twice as many samples as averaged for reliable data such as range finders, distance, or a dc analog value to be at a value that could be accurate. this graphing shows frequency with over 7 times increase in rate with same samples of averaging 10 and samples of 100. you can still see noise samples in the sample of 10.

 samplehold=(samplehold*99 +ref_from_signal*1)/100 even simpler is 

samplehold=samplehold*0.99+ref_from_signal*0.01

now if we look at data, we can also look at rate of change of signal by taking

signalA=signalA*0.49 +ref_input*0.51 and taking signalB=signalB*0.51+ref_input*0.49 

and then taking difference is= (signalA -signalB) *2 *10 to get change in signal up or down

we then take the absolute value and we come up with data that can represent the real time rate of change

it does a few things. 1 it shows us the rate of data change with above graph. rate of change increases as difference in signal changes. also these are peak to peak waves. when taking measurements changes can be smaller than lowest and highest values

and it allows us to see a few patterns. when the change rate is low, the signal is stable, and we know when the signal is not stable, it has a higher rate of change. the 100x filtering signal will almost never become stable in the times given in these examples. but it also is the most filtered and least noise signal.

here is a graph from program called bandpassfilterWdetectionOfStability.py that shows how detecting rate of change can show you a stability in signal for a time to measure. but in 100x sample rate of change is still too slow. 

the green line slashes are areas that the...

Read more »

x-python-script - 12.05 kB - 01/19/2021 at 12:23

Download

filter_sensor.zip

arduino code

Zip Archive - 13.79 kB - 01/19/2021 at 11:54

Download

bandpassfilterWdetectionOfStability.py

this is similar file, it just shows ares of stability for a measurement to take place. however with 100x filtering stable does not mean accurate at least in this code

x-python-script - 11.82 kB - 01/18/2021 at 08:50

Download

bandpassfilter.py

bandpass filter example showing noise and filtering of 10x and filtering of 100x along with showing amount of change in signal

x-python-script - 11.18 kB - 01/18/2021 at 08:36

Download

  • arduino code is loaded and includes a hybrid filter for low noise fast response

    jamesdanielv01/19/2021 at 11:53 0 comments

    included files for filter_Sensor. includes example vl53l0x and my own noisefilterbox.h that has a function that reduces noise. this is from actual data from a lidar sensor VL53L0x with noise +/- 0.5mm at 30-50 samples a second. no tweaking the registers of the device yet.

    this is using filter in Filtermode==3 and has an stable time of down to 4 seconds for a change of 150mm

    here is same device at a different time without any filtering

    this filter adjusts quickly to change. 

  • measure of an analog environment with noise and filtering.

    jamesdanielv01/18/2021 at 23:40 0 comments

    for the arduino code, there is a noisefilterbox.h file, that has a function called

    filterNoiseReductionBox(float AnalogfloatSignalInput,int FilterMode, bool isItStable) it does a few things

    AnalogfloatSignalInput is the signal to input into it.

    FilterMode can be 0,1,2,3

    FilterMode==0 is no filter so you can see default noise.

    FilterMode==1 is 10x filter. it is still somewhat fast, but you can see ripples. 

    FilterMode==2 is 100x filter. it is very slow to respond, but noise free.

    FilterMode==3 this is hybrid mode filter, it is quite fast and noise free.

    here is an actual VL53L0x sensor with a case enclosure for sensor and sampling about 30 to 50 times a second. the plotter of Arduino outputs the data. i used a baud of 19200. you can see the noise is +/-6mm

    here is what it looks like using FilterMode==1 a 10x sample filter

    signal is still noisy but it is now a smaller scale, it is +/- 2

    here is FilterMode==2 

    with 100x filtering noise is reduced to +/- 0.5mm but the slew rate, or rate of change is really slow. with 30-50 samples a second, it takes at least 1300 samples to  become stable so between 26 to 43 seconds per 

    measurement.

  • how to get useful information out of filtered data and noise reduction

    jamesdanielv01/18/2021 at 06:41 1 comment

    when using the lowpassfilter.py code you can see from the pulsed data, and increase in frequency that there is a limit to how good of filtering we can do and get a quick enough response for data to be useful.

    here is a second set of python code called bandpassfilterwdetectionofstability.py that can detect when a signal is stable and not changing more than a set percentage. from this data graph it can be seen that when a lot of filtering is done, even though there is no noise, it takes a long long time for sample to stabilize, and in the 100x filtering the signal rarely stabilizes within the times allowed for in simulation.

View all 3 project logs

  • 1
    Step 1

    use a VL53L0x, or any device with analog noise to filter out

    then download arduino code or play with the python file to simulate what you are doing.

View all instructions

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates