Close

Cochlea Prototype!

A project log for NeuroBytes

Build your own nervous system!

zakqwyzakqwy 04/13/2017 at 16:104 Comments

I posted these shots without much context to the NeuroTinker Instagram account. Hopefully this post fills in a few gaps.

Above: sketchy prototyping techniques featuring a Teensy 3.5 and an Adafruit MAX4466 electret mic board. Advantage: I was able to cobble this together from stuff I had on hand in a few hours. Disadvantage: it's quite delicate (especially the LEDs).

The cochlea is a spiral hollow structure in the inner ear that does the dirty work of converting sound into neural impulses. I won't get into detail on its workings here -- it's a fascinating system that is worthy of deep study -- beyond the parts relevant to this build.

Above: a diagram of the cochlea from here.

The cochlea is filled with a fluid that moves in response to sound entering the ear; the fluid in turn moves thousands of hair cells, each of which triggers nerve cells that send electrical signals to the brain. Due to their location along the fluid path (along with the varied stiffness of the basilar membrane), the hair cells respond to different frequencies.

There are many ways we could simulate the ear, the simplest being an sound-pressure-level-to-firing-rate converter of some sort. Conversely, a faithful full-scale reproduction of the organ would have thousands of outputs tuned to distinct frequencies, along with an input method to simulate the extra hairs used to 'tune' the mechanical preamplification system that exists in the real ear. We opted for something in between: some frequency selectivity but hopefully not too much bulk or cost.

Above: kinda hard to see, and the harmonica skills are severely lacking. But it works.

The prototype makes liberal use of @Paul Stoffregen's excellent #Teensy Audio Library -- in particular, the 1024-bucket FFT function. The code, shown below, doesn't do much beyond (a) grab specific buckets, (b) scale the FFT values, and (c) map those values to LED brightness. The output ports don't do anything yet, and eventually the brightness values will be converted into firing rates... but you get the idea.

#include <Audio.h>
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <SerialFlash.h>

// GUItool: begin automatically generated code
AudioInputAnalog         adc1;           //xy=227,187
AudioAnalyzeFFT1024      fft1024_1;      //xy=480,289
AudioConnection          patchCord1(adc1, fft1024_1);
// GUItool: end automatically generated code

//  LED pin identities
int pinLED0 = 35;
int pinLED1 = 36;
int pinLED2 = 37;
int pinLED3 = 20;
int pinLED4 = 21;
int pinLED5 = 22;
int pinLED6 = 23;

//  FFT reading array
float input_array[7] = {0,0,0,0,0,0,0};

//  FFT max value array
float max_array[7] = {0.01,0.01,0.01,0.01,0.01,0.01,0.01};

//  FFT value scaling array
float scale_array[7] = {0.08, 0.10, 0.14, 0.3624, 0.3068, 0.46, 0.7201};

//  LED output array
int led_array[7] = {0,0,0,0,0,0,0};

void setup() {
  AudioMemory(12);
  fft1024_1.windowFunction(AudioWindowNuttall1024);
  
  pinMode(pinLED0, OUTPUT);
  pinMode(pinLED1, OUTPUT);
  pinMode(pinLED2, OUTPUT);
  pinMode(pinLED3, OUTPUT);
  pinMode(pinLED4, OUTPUT);
  pinMode(pinLED5, OUTPUT);
  pinMode(pinLED6, OUTPUT);

}

void getSamples() {
  //  fills input_array[7] with FFT readings
    input_array[0] = fft1024_1.read(10,12);
    input_array[1] = fft1024_1.read(13,15);
    input_array[2] = fft1024_1.read(16,19);
    input_array[3] = fft1024_1.read(22,25);
    input_array[4] = fft1024_1.read(28,31);
    input_array[5] = fft1024_1.read(35,37);
    input_array[6] = fft1024_1.read(47,49);  
}

void applyMinimum(float minimum) {
  //  cuts off the lowest part of the FFT result (noise)
  int i;
  for (i=0;i<7;i++) {
    if (input_array[i] < minimum) {
      input_array[i] = 0; 
    }
  }
}

void keepMaximum() {
  //  keeps the maximum FFT values from input_array in max_array
  int i;
  for (i=0;i<7;i++) {
    if (input_array[i] > max_array[i]) {
      max_array[i] = input_array[i];
    }
  }
}

void scaleOutput() {
  //  scales led_array (0-255) based on scale_array
  int i;
  for (i=0;i<7;i++) {
    led_array[i] = (input_array[i] / scale_array[i]) * 255;
    if (led_array[i] > 255) {
      led_array[i] = 255;
    }
  }
}

void updateLEDs() {
  //  updates the LEDs with the current led_array values
  analogWrite(pinLED0, led_array[0]);
  analogWrite(pinLED1, led_array[1]);
  analogWrite(pinLED2, led_array[2]);
  analogWrite(pinLED3, led_array[3]);
  analogWrite(pinLED4, led_array[4]);
  analogWrite(pinLED5, led_array[5]);
  analogWrite(pinLED6, led_array[6]);  
}

void loop() {
  int i;
  if (fft1024_1.available()) {
    getSamples();
    applyMinimum(0.01);
    keepMaximum();
    scaleOutput();
    updateLEDs();
    Serial.print("FFT raw output: ");
    for (i=0;i<7;i++) {
      Serial.print(input_array[i],4);
      Serial.print("\t");
      Serial.print(led_array[i]);
      Serial.print("\t\t");
    }
    Serial.println();
  }
}

Discussions

Paul Stoffregen wrote 11/24/2017 at 21:36 point

Going to blog about this soon & give your Kickstarter campaign a shout-out.  If there's any more info or progress, or a grand plans how this will work with the neuron simulator system, or anything else we should mention on the blog article, please let me know?

  Are you sure? yes | no

zakqwy wrote 11/25/2017 at 00:16 point

hey Paul, thanks for the shout-out! we tabled the cochlea module for the time being (we need to get some basic NeuroBytes kits on the market first) but are still planning it as a future product, maybe summer '18. of note, I also prototyped a sound output module (Teensy 3.2 + audio board) that acts as a cochlea counterpart -- there's an Instagram video of a basic NeuroBytes-powered beat machine here: https://www.instagram.com/p/BYb7ggPDOx2/

our hope is to use both modules as novel input/output devices to give users more options for interacting with the system. the cochlea will have 7(ish) outputs tuned to different frequency bands to roughly emulate the actual organ, each of which can be freely fed into downstream NeuroBytes devices.

  Are you sure? yes | no

Paul Stoffregen wrote 11/26/2017 at 00:25 point

Robin put it on the PJRC home page / blog just now.

Tomorrow I'll give you a shout-out on social networking.  Hope it helps.

  Are you sure? yes | no

Eric Hertz wrote 04/16/2017 at 10:57 point

cool!

  Are you sure? yes | no