Our first look will be at getting a Blinky app running using the esp-open-sdk. This is the result of an incredible amount of work by a bunch of very dedicated people who are trying to make the ESP8266 toolchain as open-source as possible. This toolchain is designed to work with Linux and Mac. Windows users will have to use a Linux VM, but I've provided a pre-configured one. Linux and Mac users can also use the VM if they want. I'm a lazy hacker sometimes too, though I strongly recommend setting it up locally where possible. It will likely be less prone to problems.

Linux Instructions

I run Ubuntu Budgie 18.04 on my daily driver development laptop. (Say that three times fast!) In order for this toolchain to compile, it requires a few dependencies to be installed. In my case, I already had all but two, but just to make sure, you may as well run the entire list:

sudo apt-get install make unrar-free autoconf automake libtool libtool-bin gcc g++ gperf flex bison texinfo gawk ncurses-dev libexpat-dev python-dev python python-serial sed git unzip bash help2man wget bzip2

If you are running a version of Ubuntu older than 16.04, remove "libtool-bin" from that if you get a package not found error. The above command should work on any recent version of Debian or Ubuntu, or distros which use those repositories (Mint, etc). If you are unable to find a package, use your package manager's search function to try and find the correct package. Use keywords; for example, on Fedora, if I got an error that it couldn't find libexpat-dev, I might issue

sudo dnf search libexpat

Take a good look at the results. Usually the correct package to install is obvious. Now it's a good idea to create a directory for your toolchain. I was trained to always create a directory in my home directory called workspace, and to do all my development work in there. You can also keep project-specific SDKs in there, which is what we will do. My ESP8266 projects ended up in:

mkdir -vp ~/workspace/ESP8266/projects

And we will clone the project into ~/workspace/ESP8266. This keeps the code and the projects segregated from each other.

Ready to clone!

Alright. Now we are ready to clone the project and build! This is actually the easy part. Move into the directory we created (something like ~/workspace/ESP8266) Just clone the project with:

As of September 2022, pfalcon seems to have stopped maintaining their repo. Another user has stepped up to maintain it, and so the link here has been updated.

git clone --recursive http://github.com/esp-open-sdk/esp-open-sdk.git

Note the --recursive flag! The project will not build successfully if you leave this out. This is because this project contains pointers to other projects, which need to be recursively cloned.

Before we apply the patch below, we need to do a full build first. Change into the esp-open-sdk directory, then into the sdk directory below that, and run

make

The default settings will create many libraries which are unnecessary and when we build our code later on, it won't fit into the RAM on the ESP8266. So we must apply a patch to slim down the libraries that are created.

If you get error: too few arguments to function '_PyImport_FixupBuiltin' during build, there are two options to fix it. You can either apply the patch given here (make sure you are in the esp-open-sdk directory), or temporarily remove your /bin/python symlink, and replace it with a symlink to /bin/python2. If you do the second option, make sure to replace the symlink back to its original state after the build is finished!

The original patch no longer works due to changes to the Makefile in the git repository. The original command below has been updated, but the original patch is available to view at https://patch-diff.githubusercontent.com/raw/pfalcon/esp-open-sdk/pull/282.patch

Now grab the patch, and apply it with these commands:

wget -O strip.patch https://gitlab.com/snippets/1833087/raw
patch < strip.patch

You will see a message showing you that the patch was successful. If you get errors saying it cannot find the files to patch, make sure you are in the esp-open-sdk/sdk directory.

Now building it is as easy as running make again. Do not use a -j option!

make

If you have any errors, make sure to read the error log carefully. It will usually give hints as to what to do. Skip ahead to step 4 after you get a clean build.

Mac Instructions

Update 2022: It's almost certain that these instructions are outdated now. The git repo that is currently maintained is at https://github.com/esp-open-sdk/esp-open-sdk and may contain instructions for Mac.

For Mac, the process is a bit more complex. I have never used a Mac, so I am just copying the build instructions provided by the project. If there are errors in this, let me know and I will correct them immediately: 

brew tap homebrew/cask
brew install binutils coreutils automake wget gawk libtool help2man gperf gnu-sed --with-default-names grep
export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"

Apparently Mac file systems aren't case-sensitive (which seems absolutely crazy to me!), so you need to create a virtual, case-sensitive file system. To do so, enter these commands (maybe use a name like "ESP-dev-volume" or something similar instead of "case-sensitive"): 

sudo hdiutil create ~/Documents/case-sensitive.dmg -volname "case-sensitive" -size 10g -fs "Case-sensitive HFS+"
sudo hdiutil mount ~/Documents/case-sensitive.dmg
cd /Volumes/case-sensitive

So now we are in our new volume. It's time to clone the Git repository and build it! 

git clone --recursive https://github.com/pfalcon/esp-open-sdk.git

Don't forget the recursive flag! That ensures that the sub-projects are also cloned. Move into the new directory that was created when cloning:

cd esp-open-sdk

We're going to patch the Makefile and add a couple other files to make the compiled code smaller later on. We can get the patch and apply it with these two commands:

The original patch no longer works due to changes to the Makefile in the git repository. The original command below has been updated, but the original patch is available to view at https://patch-diff.githubusercontent.com/raw/pfalcon/esp-open-sdk/pull/282.patch
Also, please see the associated comments in the Linux section of this guide if you encounter any other errors.

wget -O strip.patch https://gitlab.com/snippets/1833087/raw
patch < strip.patch

Once the patch has applied successfully, we can now build the toolchain. This might take a while, depending on your hardware. On my 2015 laptop, it takes about 20 minutes. Do not use a -j option!

make

 Once this process is complete, the toolchain is built! Skip ahead to step 4 to do the post-build steps.

Windows/VM Instructions:

As with the Linux and Mac instructions, these instructions are probably not quite correct anymore in 2022. The Lubuntu image is out of date and is no longer available, but if you download any modern Linux distro and create a VM with VirtualBox or VMWare, you can then follow the updated instructions in the Linux section above! If you need help with this, feel free to contact me or leave a comment on this page.

Espressif recommend using a VM with Linux to do ESP8266 development on Windows. I highly recommend creating a virtual machine from scratch, and following along with the tutorial for Linux. This way, you will learn a lot more, and you will understand what to do if things go wrong. To make this even easier for you, I have created a VM image of Lubuntu 18.04 pre-installed with all the tools you will need to develop applications for the ESP8266. You could use this image on Linux or Mac as well if you wanted to keep your development environment self-contained. Start by downloading VirtualBox. Pick the platform package that matches the host system, i.e., if you are on Windows, pick the Windows Platform Package. Also, download the VirtualBox Extension Pack. Go through the setup process for VirtualBox. Once that is complete, double-click on the Extension Pack file. This will open VirtualBox if it isn't already open and prompt you to install the extension. Go ahead and install it. Now we are ready to download and import our development image.

[original links removed -- please see note at the top of this section!]

Once the file has downloaded, you can open VirtualBox, go to File->Import Applicance... and then locate the .ova file. A new virtual machine should appear in the list after this is done! You might want to tweak a few settings depending on how much RAM your system has. If you need help on this topic, there are many excellent tutorials out there on how to use VirtualBox.

A few quick notes: the password for the user esp is hackaday. You'll need this when running commands prefixed with sudo. I have created shortcuts to the terminal and to emacs on the desktop. The VM already has the toolchain installed, with the patches and changes made that we outline in the next sections. In the Settings page for the VM, you need to pass your USB device through to the VM. You do this by going to the USB page, and clicking the green plus sign at the right. Choose the device you want the VM to use, and then click OK. Restart the VM, and the device will show up inside the VM! Note that while the VM is capturing that device, the host system can no longer see it. The VM contains a udev rule for the Feather HUZZAH, all you need to do is edit the serial number to match yours. Read the next section on udev rules if you want to create one for your device!

If you get any errors during the build, be sure to carefully read them. They will almost always tell you exactly what was wrong. In my case, I had forgotten to install libtool-bin and this was causing an error. Likely you will just be missing a library or something else easy to fix. If it builds successfully, great! If you need help, check the very end of this tutorial for places to ask for help. Now we have all our standard gcc-based compilation tools in our SDK folder. This is where things get a little tricky if you are a newbie to command-line compilation. In every operating system, when you type a command into the terminal, the OS needs to know where to look to find that program. If it had to search the entire filesystem each time you typed a command, everything would be painfully slow. So there is an environment variable which tells the OS where to look. In Linux and MacOS, it's called PATH. If we were to just type the name of our new compiler into the terminal right now, we'd see:

amr@amrLaptop:~/workspace$ xtensa-lx106-elf-gcc
xtensa-lx106-elf-gcc: command not found

The OS has no idea where this binary is located. So, we need to add the directory which contains these binaries into our PATH. To add or modify environment variables, we use a command called export:

export PATH=$PATH:/home/amr/workspace/ESP8266/esp-open-sdk/xtensa-lx106-elf/bin/

Of course, replace the path with the location that you compiled the toolchain in. I don't want to spend too much time on this concept, but I'll quickly break it down. We are setting the PATH variable to a brand new value. Because we don't want to delete the existing PATH variable, we add it at the front with the dollar sign, which means "replace this with the current value of PATH". The colon adds a new entry to the PATH variable, and then we specify the exact location of the binaries. If you followed my example, insert your home directory in place of mine. You never want to use relative directories in PATH; make sure to always type out the full path starting from the root directory. If we want to make these changes permanent (and you likely do), then edit your ~/.bashrc and ~/.bash_profile file with whatever editor you prefer. In both ~/.bash_profile and ~/.bashrc, simply add

PATH=$PATH:/home/esp/workspace/ESP8266/esp-open-sdk/xtensa-lx106-elf/bin/

to the very bottom of the file. In ~/.bash_profile you need to add a second line after that one: 

export PATH

This will add that directory to your PATH every time you start the terminal, so you don't have to do it by hand.

Update September 2022: I haven't had time to verify if this following patch is still needed, so try running esptool.py first and see if you get errors. If you do, then run the command below; otherwise, just leave esptool.py as is!

One last step before we are ready to move on. We need to make two small changes to the esptool.py file. I made a little script that will do this for us. All you need to do is make sure you are still in the esp-open-sdk/ directory, and then copy and paste this command:

wget -O fix.sh https://gitlab.com/snippets/1738300/raw && chmod +x fix.sh && ./fix.sh

This will make sure that esptool.py runs without errors, and it also reduces the amount of time to program the ESP8266 substantially. Updated August 2020 to fix the permissions bug!

Note to Mac users: Because your development environment is on a virtual drive, you must remember to remount it after rebooting your machine. Run the command "sudo hdiutil mount ~/Documents/case-
sensitive.dmg" replacing case-sensitive.dmg with whatever you named your virtual drive.
You could also add the created .dmg file to your Login Items in Systems & Preferences > Users and Groups > Your Username > Login Items. This will automatically mount it on login!

Preparing our Workspace

Now that we have the toolchain installed, we need to do a few more things to make our workspace easy to use. We are going to create a udev rule to make our ESP8266 easier to find; we will erase the flash and then program default values; and then we will write a basic Makefile that can be used for any ESP8266 project. These few preparations will save a lot of time later in the development cycle.

Creating udev Rules

First, we are going to create a udev rule for your ESP8266 board. Udev is a daemon running in Linux that assists the kernel in managing hotplug devices. These include things like USB hard drive and USB serial devices, which is what we'll be working with today. You will be able to create a udev rule no matter how your ESP8266 is connected, as long as you follow these steps. Unplug all USB devices related to development (logic analyzers, oscilloscopes, Bus Pirates, programmers, etc.). Then plug in your ESP8266; it might be a development board like the Feather HUZZAH with on-board USB, or it might be an ESP8266 module that you've soldered a USB-to-serial board onto. Once it's plugged in, take a look at the devices in the /dev folder. Almost all USB-to-serial devices (including the Feather, the SparkFun Thing, and any generic FTDI/CP2104/CH340 USB-to-serial bridges) will show up as /dev/ttyUSBx, where x is a digit that represents the order it was enumerated in. So if it is the only device plugged in, it will likely show up as /dev/ttyUSB0. Now we need to find the information needed to create a udev rule. We will need the VID and PID of the device, as well as the serial number. This can easily be dumped with a tool that is a part of udev, called udevadm. Give this a try, modifying /dev/ttyUSB0 for whatever your device shows up as (it could be ttyAMA0):

udevadm info -a -p $(udevadm info -q path -n /dev/ttyUSB0)

This will give you a bunch of output. Scroll up to the top of the output; all we're interested in are the first few blocks.

Take a look at that image. Note down the attributes that are highlighted in that image. The first two are just to make sure we have the correct device. The listing in the picture is for one of my Feather HUZZAH boards. The name of your device might be different, depending on the manufacturer. With this information in hand, we can create the udev rule file. In your favourite editor, create a new file in /etc/udev/rules.d/:

sudo emacs /etc/udev/rules.d/51-feather.rules

You can change the name after 51- to whatever you'd like. In that file, we are going to enter the following:

KERNEL=="ttyUSB?", ATTRS{idProduct}=="ea60", ATTRS{idVendor}=="10c4", ATTRS{serial}=="0138BEFE", MODE="0766", SYMLINK+="feather0"

Replace the idProduct, idVendor, and serial number with the values we got from udevadm. You can name the symlink whatever you like, just make it something that is easy to remember. Once you've saved the file, we need to restart udev with:

sudo systemctl restart udev

We also need to make sure your user is part of the dialout group. Run the following command:

sudo usermod -a -G dialout esp

Replace esp with your username. Now you can unplug and replug your device, and if you take a look under /dev you'll see your new device symlink! I create udev rules for any USB device that I use in development. For example, I created another rule for my Bus Pirate. If you go through the process above, you can create rules for any USB device! Pretty handy.

Prepare the ESP8266

Now that we have our shiny new udev rule, we can start preparing our ESP8266 board. One of the first things we need to do is erase the flash and program in the default values that come with the SDK. If we fail to do this step, our programs just won't run. The first step is very easy. All we are doing is erasing the entire flash chip:

esptool.py --port /dev/feather0 erase_flash

• If you get an error saying esptool.py isn't a file or command, go back and make sure you added the esp-open-sdk bin folder to your PATH.
• If you get an error saying the device can't be found, make sure it is present in /dev/ and that it isn't being used by another process.
• On modern Linux systems, /bin/python is a symlink to /bin/python3, not python2. However, esptool.py is written in python2, and this often causes issues. The easiest way to fix it is to edit esptool.py, which can be found in

  ~/workspace/ESP8266/esp-open-sdk/xtensa-lx106-elf/bin/

  and change the top line to

  #!/usr/bin/env python2

  You'll notice the comment below this line, talking about why this change hasn't been made permanent -- the Python guidelines still say that python should be a symlink to python2; however almost all Linux distros have made it a symlink to python3 as version 2 is being phased out. What a pain!

In most cases it will be one of these simple errors. The erasing process should take about 10 seconds. Once the flash is successfully erased we are ready to program the default values. Assuming we are still inside the esp-open-sdk/ folder, run the following command:

esptool.py --port /dev/feather0 write_flash 0x3FC000 ./ESP8266_NONOS_SDK-2.1.0-18-g61248df/bin/esp_init_data_default.bin

This should only take a second or two. The address 0x3FC000 is assuming you have a 32Mbit flash chip, like the Adafruit Feather HUZZAH does. Check the below chart if you have a different sized flash. Note that the version info after ESP8266_NONOS_SDK will be different if you are using a newer version of the SDK. If all went smoothly, then congratulations! We can move on to the penultimate step before writing code!

I don't believe ESP8266 come with a flash smaller than 4 megabit. If you have a different size flash, simply take the very top address of your flash, and subtract 0x3FFF from it -- this is where the defaults should be programmed.

Because we won't be using a typical IDE to develop code for the ESP8266, we'll have to take another quick detour and introduce you to make. Make is an extremely powerful and versatile tool, and it's certainly not limited to just compiling code. Make and Makefiles are a huge subject, and we are just going to cover the very basics here. If you're interested, there are many wonderful resources out there to help you learn more. Make is a program that is developed by the Free Software Foundation as part of the GNU project. It's freely available, and is an integral part of any development system. It takes files called Makefile as input, and executes the instructions inside those files to compile, test, debug, and program your code to the target. They are similar to bash scripts in that they execute lists of instructions, but make is much smarter and is optimized for compilation. It works especially well with C, which is perfect for us. Let's take a look at a basic Makefile:

P=main
CC=xtensa-lx106-elf-gcc
LDLIBS=-nostdlib -Wl,-Map=output.map -Wl,--start-group -lc -lhal -lpp -llwip -lphy -lnet80211 -lwpa -lmain -Wl,--end-group -lgcc
CFLAGS= -I. -mlongcalls
LDFLAGS=-Teagle.app.v6.ld

all: $(P)

$(P)-0x00000.bin: $(P)
    esptool.py elf2image $^

$(P): $(P).o

$(P).o: $(P).c

flash: $(P)-0x00000.bin
    esptool.py --port /dev/feather0 write_flash 0 $(P)-0x00000.bin 0x10000 $(P)-0x10000.bin

clean:
    rm -f $(P) $(P).o $(P)-0x00000.bin $(P)-0x10000.bin

The top section is where we declare all the variables we'll need. Many of these have standard names, especially CC, LDLIBS, CFLAGS, and LDFLAGS. These represent the name of the C Compiler, the libraries to pass to the linker, the flags to pass to the C Compiler, and the flags to pass to the linker. Because make is so optimized for use with C, you'll notice that we don't even have to use these variables in the body of the file. Make just assumes that because we declared these variables, we must want to use them when compiling C code. 

The rest of the file contains sections called recipes in make lingo. These are separated into one or more lines. The first line is the result we want, followed by a colon, and optional dependencies. The rest of the recipe is individual commands to run, each on a newline starting with a tab. If no instructions are given, make will decide what to do based on the file type and what the dependencies are. In our Makefile, we can see that $(P) depends on $(P).o, which itself depends on $(P).c. This tells make it needs to first compile $(P).c into an object file $(P).o, and then link that object file with all the libraries we specify in LDLIBS. There's also a recipe to automate uploading our code to the target device. We'll see how this works shortly.

Now that we have a Makefile, let's take a look at our first Blinky program!

After all that work, we are finally ready to compile and program our first application! Let's blink an LED! Various ESP boards have LEDs on different pins. The Feather HUZZAH has one on GPIO0, and another on GPIO2. The SparkFun Thing has one on GPIO5. NodeMCU has a red LED on GPIO0. And most ESP modules have one on GPIO2 or GPIO12. In my example below, we'll be blinking the LED on GPIO0, but it's very easy to change this to whatever GPIO you like!

Alright, let's start by taking a look at the code:

#include "ets_sys.h"
#include "osapi.h"
#include "gpio.h"
#include "os_type.h"

/*
*  Define the LED as being on GPIO0.
*  You will need to change this if your LED is on another pin.
*  Almost all ESP8266 modules have an LED on BIT2. Check the datasheet for your module!
*/
#define LED BIT0 

static os_timer_t ledTimer;

void ICACHE_FLASH_ATTR ledTimer_cb(void *arg) {
  static int ledStatus = 0;
  if (!ledStatus) {
    gpio_output_set(LED, 0, LED, 0);
    ledStatus = 1;
  } else {
    gpio_output_set(0, LED, LED, 0);
    ledStatus = 0;
  }
}

void ICACHE_FLASH_ATTR sdk_init_done_cb(void) {
  os_timer_setfn(&ledTimer, (os_timer_func_t *)ledTimer_cb, NULL);
  os_timer_arm(&ledTimer, 500 , 1);
}

void ICACHE_FLASH_ATTR user_init() {
  gpio_init();
  system_init_done_cb(sdk_init_done_cb);
}

Even to those experienced with C, this code might look a bit strange. Let's go through it from top to bottom so that you really understand what is going on here. The includes at the top are from the SDK we built. Without these, the code wouldn't work at all. Three of these, ets_sys.h, osapi.h, and os_type.h will be present in just about every single C source we'll be writing in these tutorials. They represent the most basic functions and types needed to run code on the ESP8266. In this example, we also have gpio.h because we want to be able to manipulate the GPIO. Documentation on almost every aspect of the SDK can be found in the official non-OS SDK documentation PDF. I recommend saving a copy of this, as their website can occasionally be slow or flaky. Every function call we make in this program has detailed information in that document.

We define the macro called "LED" to be the value of BIT0. This is just to make it easier to change which pin we are using. If you are using a different board, as outlined earlier, make sure to change this to the correct value. GPIO0 through 12 are simply BIT0 through 12. Super handy!

The ESP8266 can be a microcontroller slave, or, in our case, the master of the system

We then come across our first ESP8266-specific type, the os_timer_t type. This is a struct that holds information about a software timer, which is what we will use in this code to control the speed of the blinking. Because of some quirks in the way the SDK works, most of the code we will learn about will use software timers to execute our code.

And now our first function, ledTimer_cb! This is a callback function, which means this function is called by the SDK once a certain task is completed. In our case, it's called when the timer fires. Before the name of the function, we have a decorator. A decorator modifies something about the function, typically where it is stored or how it is linked. In our case, ICACHE_FLASH_ATTR is telling the linker to store and run this function in flash, instead of copying it into RAM to execute. This helps save RAM for more important things. The rest of this function should be pretty easy to comprehend for anyone with experience with C. We created a variable to store the state of the LED. Every time this function is called, we check which state it is in (on or off) and then take certain actions. In our case, the action is a call to a function from gpio.h called gpio_output_set(). This function takes four arguments. The first two are masks that enable or disable the GPIO pin as output. When disabled, the pin becomes an input. The last two arguments are masks to set or clear the bit representing the output state. So gpio_output_set(BIT0, 0, BIT0, 0) will enable GPIO0 as an output and also set the output state to on. Conversely, gpio_output_set(0, BIT0, 0, BIT0) disables the pin as an output, and sets the state to off. Technically, we could just set the output bit and not bother disabling the GPIO.

The next two functions are like the setup() function in Arduino. sdk_init_done_cb is a function which is called once the system has finished setting itself up. This is where we set the initial state for the program. In our case, we are setting up the software timer to call the function ledTimer_cb when it is fired, and then enabling that timer with a period of 500ms. The 1 in os_timer_arm means the timer should run forever. Notice that we are passing the address of the ledTimer_cb function using the indirection operator, &. If you have no idea what that means, that's okay -- just remember to include it whenever you call these function! We also have to cast our function to (os_timer_func_t *). Again, if you don't know what that means, don't worry -- just remember to put this before the name of the function you want the timer to call. Fairly simple, right? 

The very last function, user_init(), simply initialises the GPIO and then tells the SDK to call sdk_init_done_cb when it is ... well, done initialising the SDK, of course! You'll notice almost all the function names and types have descriptive names that give us a good idea of what they are doing. This is good programming practice, and something we will stick to through these tutorials.

Alright, save that code to a file called main.c. You should now have two files in the blinky/ directory, Makefile and main.c. Another quirk of the ESP8266 SDK is that it expects there to be a file called user_config.h in the project directory, even if it contains nothing. It will complain and stop the build if it isn't there, so create it using:

touch user_config.h

Now we are ready to run our first application! Run the command:

make flash

This will build the code, and then flash it to the ESP8266. Make sure that the ESP8266 is plugged in, and that the Makefile has the correct port. This is the step where any bugs, issues or errors will pop up, so don't be discouraged if it doesn't work. The error messages are usually descriptive enough to point you in the right direction. You can ask for help at the ESP8266.com forums, or you can contact me by joining the channel ##esp8266 on Freenode IRC. Yes, with two #'s. It's the unofficial ESP support channel!

Whew! That was a lot of work! Pat yourself on the back for getting this far. If you have any issues or questions, don't hesitate to contact me. You can message me here on Hackaday, you can catch me on Freenode as mentioned above, or you can leave a comment on this tutorial. I can't promise to fix every issue, but I should be able to steer you in the right direction. There is a wonderful community that support each other. Finding help shouldn't be too hard!

Now that we have the toolchain set up and your first app running, we have a base that we can build on in future tutorials.