Close

More thoughts on v0.5

A project log for NeuroBytes

Build your own nervous system!

zakqwyzakqwy 12/21/2014 at 22:288 Comments

WARNING: This is a long post related to future planning and likely will not be acted on for some time. Most of the ideas below haven't been thoroughly thought through (wowsers), so read at your own risk.

I haven't abandoned v0.4; I'm roughly 60% through the production run and as I covered in the previous post, no game-ending issues have popped up. I'm going to keep using the current iteration until... well, who knows? Until its constraints get too frustrating. More likely than not, I won't consider actively developing a new iteration until ~120 Neurons is no longer enough to hold my interest. Having said that, I couldn't sleep last night and did a bit of 1:00 AM whiteboard brainstorming which I'll decipher in this log update.

I'm going to go through the ideas here (I had enough presence of mind to number them chronologically). This post will serve as a reminder, more than anything--future Zach, make sure you think about this stuff before ordering the next generation of PCBs. Apologies, as always, for the poor quality of the whiteboard pictures...


  1. New PCB concept drawing
    1. ATtiny13A--the cheapest of the ATtiny series, and surprisingly it doesn't seem to be cheaper than the 44A in similar quantities. In any case--the idea is to back the I/O count down as much as possible. I suppose as pictured, I'd still need 8 I/O lines--three Dendrites (including the Learn Mode connector), two Axon options, and the RGB LED. Maybe the 44A is still the right call.
    2. Big deal here--each dendrite type has one connector rather than many. I'll get in to more details later, but this design includes some type of external splitter that allows multiple signals to feed a single dendrite.
    3. More on 'learn mode' later, too.
    4. The inhibitory axon is lower priority--that would complicate a lot of things but might prove interesting. Idea is--plug an Inhibitory Axon into an Excitatory Dendrite and it--does what? TBD, I suppose.
  2. Notes on the dendrite system
    1. So... the new dendrites will use the ATtiny platform's built-in 10-bit analog-to-digital capability. That means they'll be looking at voltage inputs between 0 and 5vdc.
    2. The analog front-end--that's the detail I'm conveniently skipping over here. This part of the system will convert simultaneous dendritic excitations into a proportional voltage response using some as-of-yet nonexistent circuit. One complicating factor--I still want exciters to work!
  3. Options for excitatory and inhibitory dendrites
    1. See drawing (1) on the right? Yup, this relates to the two dendrites on v0.5.
    2. Here's the idea--I want to be able to plug a single unbranched (/simple) axon into either of these dendrites and have it work, but I also want to be able to split the dendrite. Maybe call the split version a 'soma', since that is the cell wall that dendrites branch out from?
  4. Drawing conventions
    1. Lest anybody gets confused, these are the 'schematic symbols' I'm using tonight. To be clear--plugs live on the ends of wires, while headers are soldered to PCBs.
  5. Dendrite expander idea
    1. This part could connect to the inhibitory and excitatory dendrite headers on the board. See? 1-to-5! Maybe I could make a few different standard split ratios--1:3, 1:5, 1:8, etc?
    2. One key idea here--separate the splitter board from the wire/plug. Not drawn but indicated with an arrow--splice a plug onto both ends of the wire, then put an input header on the board? Might bring down overall mfr costs and make everything a bit more modular?
  6. Thinking about synapses and weighting
    1. Linked via twisty arrow to idea (2).
    2. More far-out ideas here--making the axon-dendrite interface (synapse) something other than a 'perfect' conductor.
    3. If 'weighting' can be done using resistance--maybe use CdS cells as passive weighting components? I suppose cadmium doesn't go with RoHS, huh? Would the LEDs from other Neurons throw this off?
    4. Ideally--do 'weighting' in software (learning mode) AND hardware. How?
  7. v0.5 dendrite circuit (see below)
  8. v0.5 axon terminal circuit (see below)
  9. v0.5 axon (board level) circuit
    1. Pretty basic block diagram of a potential inter-Neuron circuit.
    2. Circuit "A", in the Neuron dendrite system, would convert the number of simultaneous inputs (either signals or exciters) into a proportional voltage, as shown in the graph on the lower right.
    3. Circuit "C", in the Neuron axon system, would allow each Neuron to add to the total voltage produced by "A" when its Neuron fires.
    4. Circuit "B" would allow multiple Neurons to send signals to a single downstream Neuron. What to call this? 'Soma'?
  10. Dendrite clusters and axon terminals
    1. Basic terminology--Dendrite Clusters are the input splitters shown in idea (5), while Axon Terminals are standard axons pre-soldered in parallel.
  11. More on dendrite clusters
    1. Yup, just added on to idea (5) here. Not much beyond the headers being in parallel with passive components as necessary (no microcontroller!). I want these to be scalable as needed!
  12. More on axon terminals
    1. I described this under idea (10)--just illustrated here. Ideally, everything is passive and in parallel like it is today!
    2. Do I need both Axon Terminals AND Dendrite Clusters? What about a 'splitter' and an extending Axon?
  13. Overall goals for v0.5--big picture
    1. Every Neuron version has been cheaper than the previous. Well, except v0.1. In any case, I don't want to break that cycle--as such, recreating one of the v0.4 units should not be more than ~$3.50 or so.
    2. In ideas (7), (8), and (9) I showed a proposed block diagram of the new interconnection system. If I can make circuit "B" in (8) unnecessary, then the "dendrite clusters" and "axon terminals" will just be parallel circuits--that makes manufacturing really simple and cheap and (hopefully) scalable.
    3. Cheap passives--circuits "A" and "C" are additions to the v0.4 design! Gotta make 'em as cheap as possible! That means no extra microcontrollers!
    4. More flexibility with connections--duh. That's the whole point. Some Neurons only have one dendrite. Some have TONS of dendrites. I need both options!
    5. The bonus would just be cool. Some kind of modular circuit that would change the weighting of a given input (or inputs). That gets closer to emulating real-world synapses--as in, not every connection is made equal.
  14. Can they learn?
    1. You can't tell in this image (refer to the master whiteboard picture at the beginning of this log update), but this idea is waaaay over on the other side of the board.
    2. Learn Mode signal--the best comparison I can make is the HART protocol, since it overlays a digital signal over an analog signal. Same idea here--dendrites and axons behave as they did previously, but they can also send a coded message that downstream Neurons will recognize to switch modes.
  15. Don't forget to remove blue painter's tape
    1. This happened when I taped a piece of paper to my whiteboard a few months ago. Normally I just avoid the area, but I needed the whole sheet for this brain dump--as such, it's marked as the Whiteboard Damage Zone.
  16. Dendrite waveform
    1. Here is what you might see if you snoop a dendrite with a 'scope. Note that the voltage is quantized to a degree, allowing us to differentiate between the number of inputs sending active signals (exciters AND action potentials!).
  17. Learn mode signal
    1. Extended from the previous image--the 'fast tone' just needs to be unique. As in, something that wouldn't be recreated by normal Neuron activity--maybe ten state changes in a certain number of milliseconds or something.
    2. How many modes? Maybe two--Learn and Run. Two different tones for different modes, or a toggling set up?
  18. Learn mode signal networking
    1. Should Learn Mode (and Run Mode) signal propagation follow the standard network? As in, if a Neuron is placed in Learn Mode, all downstream Neurons are placed in the same mode?
    2. Seems like electrically, the Learn Mode signal should propagate as we want it to without any special software considerations--all downstream Neurons would get the signals, but since Axons don't listen any lateral Neurons wouldn't get the message.
    3. What happens during Learn Mode? Maybe it's identical to Run Mode, but input weighting changes based on action potential frequency? How can we adjust input weighting when dendrites are in parallel?
  19. Learn mode propagation diagram
    1. A bit blurry, oops. Black squares are Neurons, blue lines are standard Axon-->Dendrite connections, and green lines are Learn Mode signal propagation.
    2. New peripheral--the Learning Remote! Used to inject Learn Mode signals into a given Neuron.
  20. Learn mode features
    1. Also blurry--sorry.
    2. Describes what happens in Learn Mode--Neurons behave normally but would start increasing input weights.
    3. Again--how do we differentiate between individual Dendrite connections? Would this setup only work with unique inputs (as with v0.4)?
    4. Learn Mode Jumpers--connect standard signals to the Learn Mode input (see idea (1)) and ONLY Learn Mode signals will be 'heard'. Allows Learn Mode to propagate to non-connected network sections.
  21. Local/regional learning!
    1. This picture got a bit stretched and is super blurry. Sorry.
    2. This connects to the diagram in idea (19)--it just shows linking Learn Mode jumpers to a normally unconnected neural network.
    3. Seems like a 'bonus feature'.

Okay--back to v0.4 soon, I promise. With any luck, I'll reference these ideas when I take the next steps in v0.5 development. I think the analog input concept could improve network flexibility--it would be nice to have 10+ dendrite Neurons without specialized boards or programs. But for now, that's a solution lacking a problem.

Zach

Discussions

Stryker295 wrote 12/25/2014 at 07:32 point

A very great read. In a way this feels very similar, software-wise, to physics/fluid sims I've made in the olden days.

I wonder how hard it would be to implement this where each neuron is a pixel, on say a 64x64 LED matrix, and then simulate all the neurons in software, and put a photoresistor/button/some form of human input on each LED.... it'd be a giant interactive touch-LED-wall of neurons.

  Are you sure? yes | no

zakqwy wrote 12/26/2014 at 01:24 point

Sort of like an FEA system? That's fascinating--I played around briefly in school with a few of these tools but never really got much more than a demonstration of basic features. I think a predefined neural network has some significant advantages--lots of Neurons that are easy to define in different ways, easy I/O, and so forth. I'm not sure how easy they are to hack, but there are a number of MIDI pads out there that have large matrices of illuminated buttons; I think I've seen 16x16 before, but that might be a bit expensive. Either way, one of the standard 4x4 units could be used to get software bugs out before building a big array.

One constraint with a 2-D array is that it's not easy to visualize connections that aren't fairly regular (i.e. next to each other); you could program the display to respond as a complex network, but it wouldn't be physically intuitive on the display itself. Writing a neat touchscreen driver for the program could be neat--figure out an easy way to navigate around 3D space, give yourself the ability to drop 'Neuron' nodes, and include tools for connecting and configuring networks of various sizes.

Either way, it would be awesome (in my opinion) to connect a software-based or LED matrix neural network to a physical setup like I'm building here; might give you the best of all worlds--complex connections via the computer program, and easy reconfiguring using the actual boards.

Speaking of which--I should get back to focusing on v0.4! I think there is still a lot to be learned from this generation.

  Are you sure? yes | no

Jarrett wrote 12/22/2014 at 16:15 point
What requirements do you have for a micro?

How many channels (if any) of PWM, ADC, digital I/O ?


I ask because while ATMegas are great for people making the transition from Arduino, PIC is king where cost and configurability are concerned.


It looks like the ATTiny44A has 8 I/O, and at least some PWM and 10b A/D converters for $1.34 in 100-unit quantities on Digikey.
Using Microchip's amazing tool: http://www.microchip.com/ParamChartSearch/chart.aspx?branchID=1005

We can find PIC16F1503 that has all of the same features at $0.75 in 100-count quantities.


If you don't need all of those peripherals, we can go even lower: PIC16F54 has the same pin count without the PWM/ADC channels and is $0.54.

Anyway. You get the idea, if you're willing to do a little more work.

  Are you sure? yes | no

zakqwy wrote 12/22/2014 at 18:03 point
Wow, great call; I defaulted to ATtinys because that is what I've used in the past. I've never programmed PICs in the past but this might be a good excuse to get started. Any way I can use some form of C for the code, or would I need to go down the ASM path? Think I could bit-bang PWM and use the cheaper microcontroller?

  Are you sure? yes | no

Jarrett wrote 12/22/2014 at 18:17 point
PICs work fine with C. The workflow between ATMega and PIC is very similar, there just isn't as much community support (entirely because of the Arduino thing).

How much does accuracy matter?
As long as the micro isn't really doing anything, bitbanging it is worth a shot. Try it on one of the ATMegas you have kicking around, first.

  Are you sure? yes | no

zakqwy wrote 12/22/2014 at 20:17 point

I'm doing the 3 channels of PWM (for each Neuron's LED) manually on the ATtiny44A currently; it seems to work well enough, although I suspect I need to change timing parameters on one of the loops because I think it's getting slowed down. I'll keep your suggestion in the back of my mind--once I get closer to v0.5 I'll get my hands on a few PICs and start tinkering a bit. Saving BOM costs is critical, so this is quite helpful.

  Are you sure? yes | no

PointyOintment wrote 12/22/2014 at 09:49 point
For 2.2, I think you could use a resistor ladder DAC pretty easily. It would be R–R instead of the common R–2R. You could change some of the resistors or replace them with pots or photocells for variable weighting, resulting in R–R–xR. No idea how to control the weighting in software, though, if it just receives the one analog signal. It would have to communicate weights to the dendrite cluster somehow.

  Are you sure? yes | no

zakqwy wrote 12/22/2014 at 14:14 point
Thanks for the comment--points me in the right direction (http://www.allaboutcircuits.com/vol_4/chpt_13/3.html). It seems like you just use individual resistor values to change the weight of each bit, so the total input range spans the full scale of the analog input--correct? So, if the scale is 0-5vdc and you're dealing with four lines, the LSB voltage value would be 1/(2^4) * 5? I suppose if the resistor values are all identical, then each input will produce a unique voltage change? Binary math is pretty neat.

Why can't I directly drive the output from the resistor ladder--why is the operational amplifier necessary?

  Are you sure? yes | no