Deep Fake Birdsong, 2020

Artist and analog electrical engineer: Kelly Heaton
Software engineer: Johann Diedrick

Project Summary

Deep Fake Birdsong unites art with science to inquire about the electrical nature of lifeforms and to push the envelope of artificial intelligence. The project examines the art of Kelly Heaton, who builds circuits with lifelike qualities in order to question the nature of consciousness. For this collaborative work with software engineer Johann Diedrick, Heaton built an interactive analog electronic circuit that generates birdsongs. In essence, Heaton's electronic sculpture is a minimal interface analog audio synthesizer. The circuit's simplicity and low-component count is conceptually important to the experiment, which endeavors to show the wide range of lifelike behaviors that can manifest from analog electronic hardware. As it turns out, the expressive power of analog electronics is tremendous because a small quantity of oscillators can, when combined, create a surprisingly rich diversity of sound. We demonstrate that slight alterations in the base resistance of five interconnected oscillators will result in a wide variety of unique bird-like sounds. In this regard, Heaton's work has much in common with BEAM robotics (Mark W. Tilden et al.) In a second step of the project, Heaton and Diedrick recorded audio from her birdsong generating circuit and ran it through software that is designed to identify biological species of birds. Diedricks's Flights of Fancy is a software system that parses an audio recording of arbitrary length into a series of spectrograms with one second duration. These incremental signals are then compared against a database of actual recorded birdsong and assigned a species match along with a percent confidence rating. Of the 122 generated spectrograms parsed and analyzed by Flights of Fancy, four different species of bird were identified with match confidence percentages ranging from 48% to 96%. The four species of identified birds are:

Silky-tailed Nightjar: 47 matches with an average percent confidence of 93.147

Grey-breasted Seedsnipe: 16 matches with an average percent confidence of 26.515

Paramillo Tapaculo: 54 matches an average percent confidence of 48.991

Olive-striped Flycatcher: 5 matches with an average percent confidence of 47.938

The Experiment

Heaton approached Diedrick with the following experimental idea: what would his bird identification software say about an analog electronic circuit that generates bird-like sounds (as opposed to recordings of actual birdsong)? Would the software detect one or more bird species and, if so, with what degree of confidence? Based on a 2:02 minute recording (one spectrogram per second, omitting segments that contained no audio), Diedrick’s software system generated 122 spectrograms. From there, we passed these images to our trained model to see what bird species it predicted the spectrogram to come from, based on its training against the BirdCLEF dataset. From our experiments, the system identified four bird species that it predicted the spectrograms belonged to from our recording. The system produces a “match confidence” score which tells us that, across the 1,500 bird species it can predict, which one of these species does it have the highest confidence to which the spectrogram belongs. The match confidence percentages of our experiment ranged from 19% to 96%. Notably, 47 of the 122 spectrograms were matched with Antrostomus sericocaudatus (Silky-tailed nightjar) with an average percent confidence match rating (across all associated spectrograms) of 93%. If we were scientists instead of experimental artists, we would confidently conclude that the predominant bird species in our recording is the Silky-tailed nightjar.

Technical Details / Hardware

The "singing" electronic bird sculpture is centered on a discrete component (analog electronic) circuit. This circuit consists of five oscillators (astable...