Close
0%
0%

Acoustic Camera

An acoustic phased array that is able to create images and videos from sound

Public Chat
Similar projects worth following
This was my Senior design project in ECE at Carnegie Mellon, and uses 96 MEMS microphones to create images and videos from sound. The project was mainly just a platform to mess around with beamforming and general phased array and signal processing math.
Videos of it operating: https://youtu.be/i6a2K4w8dn0 https://youtu.be/TViES6iZTKw
Project Blog (contains most details): https://jeduffy.site/?page_id=2
Final Project Presentation: https://youtu.be/G44M0HbIIBQ

V005.zip

The network driver code - takes the raw input from the array and converts it to ASCII hex for the Java to read. Also has a "Log" version just for logging data to a file instead of streaming into a fifo. You will also have to increase the default MTU (maximum transmission unit), which is 1500 or something by default. Meaning packets have to be less than 1500 bytes or they'll be dropped by the network driver or something. The array sends 2400 byte packets, so you have to increase this, I went up to 4000.

x-zip-compressed - 1.78 MB - 06/02/2020 at 01:46

Download

C009_MultiImages.zip

Processing sketch to do real-time imaging. Requires the network driver to be running at the time as well to work, OR, a logfile to be piped into the FIFO. You'll first have to make a fifo using mkfifo (I called mine /home/john/pipe, it's hadcoded near the end of the file). If you just run a file cat Logfile.log > /home/john/pipe while this is running this should work

x-zip-compressed - 97.71 kB - 06/02/2020 at 01:38

Download

PAR_03_ALLBOARDS.zip

the FPGA firmware

x-zip-compressed - 3.15 MB - 05/29/2020 at 23:33

Download

Reflection.mp4

Video showing acoustic reflection off of a piece of acrylic

MPEG-4 Video - 34.65 MB - 05/29/2020 at 23:19

Download

odds_medium_moving.avi

Moving speaker in front of the array

Audio Video Interleave (AVI) - 17.69 MB - 05/29/2020 at 23:17

Download

View all 10 files

Enjoy this project?

Share

Discussions

Adam Quantrill wrote 08/11/2020 at 16:44 point

John, now you have the imaging device, it would be a simple matter to build a telescope, your choice of Newtonian, or refractor (get some bin bags full of heavy gases and make them into lens shapes)

  Are you sure? yes | no

TheBestJohn wrote 08/10/2020 at 19:01 point

Could this be used to get the projected location of the sound in 3d space relative to the receiver? I've had an idea for a self pointing light source that responds to finger snaps and have thought about a phased array before but I haven't had a large amount of time to investigate them other than in a precursory look

  Are you sure? yes | no

John Duffy wrote 08/11/2020 at 04:07 point

Im going to have to disagree with your username :), but yeah, it should be able to do basic angle-of-arrival.  But, you could probably do that with a lot fewer microphones, like is done in an amazon echo. I know seeed has (or at least had) an array for that kind of direction finding. 

You only need this many to get the gain and bandwidth to do imaging over many frequencies, with reasonably non-blurry images.

  Are you sure? yes | no

Elliot Williams wrote 06/16/2020 at 10:09 point

"mainly just a platform to mess around with beamforming and general phased array and signal processing math"

No further justification necessary, your honor!

Marvelous project. 

Can you see reflections off of walls from the point sources?  It would be so cool to see how an impulse bounces around in a room.

  Are you sure? yes | no

John Duffy wrote 08/09/2020 at 18:27 point

Thanks!

Yes! reflections were one of the coolest things to see with this, pointing it at a corner of my room even acted like a corner reflector!  In some of the later tests you could even see sound going through things, and diffracting around edges.

I eventually added some processing to take barker coded impulses and do 3D mapping, but still needs a LOT of work to get the processing to produce useful images/videos.  Havent had much time the last couple months, just moved twice and started a new job, but planning to get back on this at some point.

  Are you sure? yes | no

RyanABahr wrote 06/03/2020 at 04:42 point

Most impressive project I've seen in a while. Great video. 

  Are you sure? yes | no

John Duffy wrote 08/09/2020 at 18:28 point

Thanks!

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates