Close
0%
0%

VoiceBox

Adding a Flask web interface (and other stuff) to the Google AIY Voice Kit.

tmTM
Similar projects worth following
Adding a few bells and whistles to the Google AIY Voice kit.
Hardware: Pimoroni Blinkt RGB strip, Nokia 5110 display, 5V cooling fan and esp8266.
Software: the Flask Python web library, MPD (Music Player Daemon).

In this project we use a 3D printer (optional), the Python Flask web interface library, the Linux Music Player Daemon, and a variety of cheap hardware to build on the Google AIY Voice kit. We use this platform to develop a range of mini hardware and software projects, tied together by Google voice-recognition.

(OpenSCAD model of 3D printed enclosure: see "Replacing the Cardboard Box" below.)

See this post for an index of log posts so far.

Wi8x8_sketch_and_library.zip

Wi8x8.ino, esp8266 web-control of an LED matrix. Modified version of Adafruit_LED_Backpack (1.1.6)

x-zip-compressed - 8.02 kB - 08/25/2018 at 00:21

Download

WiLight.ino

Arduino IDE script to program ESP-01 module to control a Neopixel ring.

ino - 15.93 kB - 08/22/2018 at 22:17

Download

playlists.zip

Collection of .m3u playlist files, each containing the url of a single radio stream. [Directory: ~/.mpd/playlists]

x-zip-compressed - 3.51 kB - 08/15/2018 at 20:12

Download

VoiceBox.zip

Main Google Assistant + Flask server. [Directory:~/AIY-projects-python/src/VoiceBox]

application/x-zip-compressed - 400.24 kB - 08/05/2018 at 20:18

Download

Blinkter.zip

Web server controlling Blinkt RGB LED array. [Directory: ~/Python/Blinkter]

application/x-zip-compressed - 1.54 MB - 08/02/2018 at 21:08

Download

View all 11 files

  • 1 × Google AIY Voice Kit (v1) Microsoft are selling the old version for $5 yay!
  • 1 × Nokia 5110 display $2 display from Aliexpress
  • 1 × Blinkt 8 Neopixel strip from Pimoroni (about $7)
  • 1 × Miniature 5V cooling fan about $2 from Aliexpress
  • 12 × 2x6mm self-tapping screws with washer https://www.newegg.com/Product/Product.aspx?Item=9SIA4SR79W3388

View all 8 components

  • Tabula Not So Rasa

    TM09/02/2018 at 18:28 0 comments

    My original plan was to download the latest Raspbian image and build the AIY environment on top of it. So I downloaded 2018-06-27-raspbian-stretch.zip from the Pi Foundation and then tried to follow the instructions in the AIY github repository (HACKING.md). And then I started going around in circles. The current branch "aiyprojects" supports my v1 VoiceHat, but the instructions are all based on the v2 Pi Zero "bonnet". I saw no virtue in blindly installing drivers for "MCU" (the SAMD microcontroller on the bonnet?) or "Myriad" (the FPGA on the Vision bonnet?).

    Fortunately, while poking around the repository looking for clues I noticed that an updated SD card image was posted to the releases page on August 3rd. I think this is based on the 6/27 Raspbian image, and hence - unlike the previous (4/13) AIY image - should support Raspberry Pi 3+. (After finding the 4/13 image didn't work on it, I moved my only 3+ to my OSMC box and don't want to move it back.)

    So far, getting VoiceBox up and running has been very straight-forward. I'll write up the installation steps in an "Instructions" post after I've replicated them on another fresh SD card.

    The biggest holdup was a bug in the new version of aiy.audio.say() which led me on a wild goose chase looking for bugs in the VoiceHat drivers or my PulseAudio configuration. I'll write-up a work-around when I've decided between a quick fix (removing the volume level modifier in tts.py), and taking the opportunity to improve the quality of the local text-to-speech output.

  • Wrestling With Python​

    TM08/29/2018 at 18:59 0 comments

    TM Wrestling With Python [Dramatized]

    I decided to port everything to Python3 before starting the Raspbian image rebuild (so I only have to concentrate on one set of Python libraries). Thankfully, it took next to no time - a sad reflection on the lack of ambition in my Python coding.

    sysinfo

    Apart from adding brackets to print statements the only change required was to decode the output of subprocess.check_output() calls:

    eg_string = subprocess.check_output("<shell commands>", shell=True).decode('utf-8').replace('\"', '')

     Interestingly, I only needed to decode() calls with shell=True.

    I also added a signature line to the top of the source (sysinfo.py),

    #!/usr/bin/env python3

    and changed the systemd service file (sysinfo.service) to use python3.

    ExecStart=/usr/bin/python3 /home/pi/Python/sysinfo/sysinfo.py
    

    Blinkter

    dict_keys() and dict_values() return "views" instead of lists in  Python3, so I had to change the already ugly line:

        pattern_index = PATTERN_NAMES.keys()[PATTERN_NAMES.values().index(request.form['pattern_name'])]
    

    to the shameful:

        pattern_index = list(PATTERN_NAMES.keys())[list(PATTERN_NAMES.values()).index(request.form['pattern_name'])]
    

    Maybe closer to the truth.

  • You Are Here

    TM08/28/2018 at 22:26 0 comments

    I've made a lot of posts in this project and the Hackaday.io inteface doesn't seem to allow creation of an index. So here's a summary of logs so far:

    In each case you should look at the logs before and after that linked for context and follow-up.

    I must admit that I'm stalling a bit. I have lots of ideas for extending the project, but all the worthwhile ones are contingent on building a fresh Raspbian image with fully up-to-date Google API libraries and Numpy. I should also make sure that everything runs on Python3 (before f-strings break library compatibility again). 

    The next post will probably be a long, rambling, open-ended log of my steps in building the new image, I'll try to tidy and summarize in a later post, once everything's (hopefully) working.

    Python 2 vs. Python 3 [Dramatized]

  • esp8266 + I2C

    TM08/24/2018 at 23:25 0 comments

    To talk to our LED backpack we'll need to install a couple of Adafruit Arduino libraries:

    <Sketch><Include Library><Manage Libraries>

    Install  "Adafruit_LED_Backpack" (I used v1.1.6) and "Adafruit_GFX_Library" (I used 1.2.9).

    Then take a few minutes to read drmpf's excellent tutorial on using ESP-01 pins. I decided to use his technique for driving I2C over the TX and RX pins, leaving GPIO0 and GPIO2 unused.

    This gives me the option of maybe controlling a relay (or two) in addition to the I2C devices.

    Because we'll be using TX (pin1) as SDA and RX (pin3) as SCL, rather than the esp8266 defaults of GPIO0 (pin0) and GPIO2(pin2), we'll need to make a slight modification to the Adafruit backpack library, to use the more general Wire.begin(sda, scl) from the esp8266 version of the Wire library, rather than the standard Arduino version - Wire.begin().

    Find the source file for the library, on my Windows installation, it's at 

    C:\Users\TM\Documents\Arduino\libraries\Adafruit_LED_Backpack_Library

    We need to add a signature for the new generalized begin() to the file Adafruit_LED_Backpack_Library.h

      void begin(uint8_t _addr, int sda, int scl);

    (Leave the old version without sda & scl for backwards compatibility.)

    We then modify the definition of begin() in  Adafruit_LED_Backpack_Library.cpp.

    void Adafruit_LEDBackpack::begin(uint8_t _addr = 0x70, int sda, int scl)
    {
      i2c_addr = _addr;
    #if defined(ESP8266)
      Wire.begin(sda, scl); // esp8266 specific generalized version
    #else
      Wire.begin();         // default version
    #endif  Wire.beginTransmission(i2c_addr);
      Wire.write(0x21);     // turn on oscillator
      Wire.endTransmission();
      blinkRate(HT16K33_BLINK_OFF);
    
      setBrightness(15);    // max brightness
    }
    
    void Adafruit_LEDBackpack::begin(uint8_t _addr = 0x70) {begin(_addr, SDA, SCL);}

    So when existing code  calls begin() without I2C parameters, the defaults (SDA & SCL), will be used. If the target board isn't an esp8266 (ESP8266 is not defined), the standard, parameterless version of Wire.begin() is used (sda & scl arguments are ignored).

    After all that, the actual sketch is a bit of an anti-climax (see "Wi8x8.ino" in the files section).

    I haven't really decided what I'm going to do with Wi8x8 yet. You'll see from the code that I've added "ear" and "cloud" patterns, so maybe VoiceBox can display its state on the remote LED matrix - in addition to the flashing button and LCD display. Another possibility is is to have scrolling text messages or a stock price ticker, but the former would need more chained matrices, and the latter works better on a multi-color matrix.

  • Wi8x8

    TM08/24/2018 at 15:30 0 comments

    A new bag of goodies, including some AMS1117 5V to 3.3V converters, arrived from China. Time for another accessory, so I dusted off an Adafruit 8x8 LED matrix with I2C backpack, and hooked it up to an ESP-01 module to make "Wi8x8".

    (Details and code to follow.)

  • WiLight

    TM08/22/2018 at 22:15 0 comments

    Now that we have our Arduino IDE configured to program esp8266's and the module's wired up, we can knock together a simple program to provide a web control interface for a Neopixel ring.

    The program is structured in much the same way as Blinkter, with a web interface to select the current pattern and a program loop that displays a 'frame' of the pattern's animation on each iteration.

    Here we see the "spin" pattern in default, rainbow, colors:

    The "solid" pattern sets all 12 Neopixels to the same color, which can be chosen by clicking on the color rectangle and using your browser's color-picker. (This screen shot shows the Chrome color-picker and the default color, which comes out as a not-too-bluey-white on the Neopixels.)

    VoiceBox supports two verbal commands, "lamp on" which sets the WiLight's pattern to "solid", and "lamp off" which sets the pattern to "none":

    def lamp(pattern_name):
        assistant.stop_conversation()
        try:
            response = requests.get('http://lamp/pattern?pattern_name='+pattern_name, timeout=5)
        except ConnectionError:
            aiy.audio.say("Couldn't connect to lamp.")
    ...
    def process_event(assistant, event):
    ...
        elif event.type == EventType.ON_RECOGNIZING_SPEECH_FINISHED and event.args:
    ...
            if 'lamp on' in text:
                lamp('solid')
            elif 'lamp off' in text:
                lamp('none')
    

  • GIMP Animation Package

    TM08/21/2018 at 16:48 0 comments

    I haven't used GAP, the GIMP Animation Package, to make a .gif in years. The OpenSCAD animation in "Replacing the Cardboard Box" was made by saving animation frames in OpenSCAD, then importing the frames as layers into GIMP. To split part of a video into .gif-able frames in GIMP requires the GAP add-in. Anyhow, the ancient GAP v2.6 still (sort of) works in the latest version of GIMP. Very simple Youtube instructions here.

  • Hooking Up the ESP-01

    TM08/20/2018 at 15:26 0 comments

    (ESP-01 diagram from Wikipedia.)

    ESP-01 connections:

    ESP-01 PinConnects to:
    VCC+3.3V supply
    RSTReset Dupont lead
    CH_PD+3.3V via 10kOhm pull-up resistor
    TX"FTDI" Adapter RXD
    RX"FTDI" Adapter TXD
    GPIO-0GPIO-0 Dupont lead
    GPIO-2Neopixel Data Input
    GNDPower supply ground

    "FTDI" USB Serial Adapter connections:

    "FTDI" Adapter PinConnects to:
    GNDPower supply ground
    TXDESP-01 RX
    RXDESP-01 TX

    Neopixel Ring Connections

    Neopixel Ring PinConnects to:
    GNDPower supply ground
    Power+5V Power supply
    Data InputESP-01 GPIO-2

    You'll need a power supply that can deliver +5V (for the Neopixels) and +3.3V (for the ESP-01). I've been waiting for over a month for a delivery of AMS1117 5V to 3.3V step-down converters, in  the meantime, I'm using a +5V supply connected to an NTE1904 LDO voltage regulator circuit. 

    To program the esp8266, connect the GPIO-0 Dupont lead to the common ground and then touch the Reset Dupont cable to ground to boot the module. Because GPIO-0 is grounded, the module will boot into programming mode. You can now press the upload button on the Arduino IDE at your leisure. When programming is complete, the module should reset itself and run the program (even though GPIO-0 is still pulled low). Disconnect GPIO-0 from ground before manually booting or resetting, or you'll end up in programming mode again.

  • Not So Fast...

    TM08/19/2018 at 02:10 0 comments

    I usually use a Linux laptop for programming Arduinos, Teensies and esp8266s so it's several years since I last set things up on a Windows machine. I'd forgotten all about FTDI-gate and FTDI-gate2.

    TL;DR Chinese manufacturers of knock-off copies of FTDI's serial USB chips use the same USB identifiers (VID & PID) as the original chips, so Windows will use FTDI's drivers to communicate with them. In 2014 FTDI released a new driver (via Windows Update) that deliberately soft-bricked clone chips. After being forced to retract the update, FTDI doubled down in 2016 by releasing a new driver that sends “NON GENUINE DEVICE FOUND!” to ersatz chips.

    Why am I telling you all this? If you're a cheapskate leech like me, you'll probably be using a knock-off "FTDI" adapter to program your esp8266 module. If so, you'll have to install an old driver.  From the FTDI driver download page, scroll all the way down and download and unzip the 2012-04-13 2.08.24 driver. 

    Plug in your USB adapter, and when a "driver not found" dialog pops up, select manual install and browse to the directory containing the unzipped driver software. After installing and rebooting, you may have to fire up Device Manager, click "Ports (COM & LPT)" (or "USB Serial Adapter"), and "USB Serial Port" to repeat the manual driver install. Reboot again and you should be able to select the new COM port in the Arduino IDE (<Tools><Port:>).

  • Accessorize!

    TM08/18/2018 at 16:17 0 comments

    I've really enjoyed having VoiceBox on my desk. I've had RaspberryPis configured for specific tasks, like OctoPrint or OSMC, running for years, but this is the most use I've ever got out of a Pi as a development platform. One drawback of the enclosure is that the GPIO pins are hard to get to, making the addition of new gadgets fiddly: I'm running out of side-wall real estate anyway.

    My solution is to connect the peripherals to esp8266 modules and to have VoiceBox control them via a web interface. My first Hackaday project, ESP8266 Retro Browser, used the ESP-01 module as a wireless modem, controlled by an Arduino Mega 2560 sending AT commends over a serial link. Thankfully, things have moved on since 2014, and we no longer have to slave an 80MHz 32-bit microcontroller to a 16MHz 8-bit AVR.

    To program the esp8266 directly using the Arduino IDE (Integrated Development Environment) we first have to download an up-to-date copy of the Arduino software, run the downloaded installer (arduino-1.8.5-windows.exe in my case), click yes (repeatedly) to install USB and COM port divers, then yes again when prompted by Windows Firewall, to allow access to the internet. 

    We then need to add support for the esp8266 to our newly installed Arduino IDE. Following the instructions on the ESP8266 Core project's github page, fire up the Arduino IDE using the desktop link. Select <file><preferences> and under "Additional Boards Manager URLs" enter the address of the esp8266 configuration file: http://arduino.esp8266.com/stable/package_esp8266com_index.json.

    The above step just tells the IDE where to look for additional, non-Arduino, configuration files. To actually add support for the board, go to <Tools><Board:><Boards Manager>, scroll down and select "esp8266 by ESP8266 Community" and click <Install>.

    One more thing we'll need to add to the IDE for wiLight, the Adafruit Neopixel library. Go to <Sketch><Include Library><Manage Libraries..>. Enter Neopixel in the search box and select "Adafruit Neopixel by Adafruit". Click <Install> to add the library.

    I'll make sure my newly installed IDE works and use it to tart up the code for my first peripheral: "wiLight", before publishing a write-up.

View all 26 project logs

Enjoy this project?

Share

Discussions

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates