Close
0%
0%

plotly + ESP8266

(Plotly has stopped support) A $6 Wireless Real-time Data Logger/Grapher

Similar projects worth following
NOTE: Plotly no longer supports streaming on their free cloud service. This project will no longer work :(.

I was recently asked to create a wireless sensor node for which it's data can be real-time graphed. I came up with several different low cost solutions, all with pros and cons, and better suiting for different applications. Although some attempts weren't suitable for the task at hand, they were very suitable for other projects. So I have included this solution here.

For this project I'm using the nodeMCU development kit (NodeMCU devKit Github) to develope my code. The development kit contains a ESP8266 WiFi module, and I'm using this to wirelessly send data to the plotly streaming server. Plotly is a online browser based tool for visualizing data, which also has a streaming feature for displaying your data in real-time. Plotly is great because any device that has a browser and is connected to the internet (smartphone, PC, tablet, etc) can see the real-time data.

The nodeMCU devkit contains a ESP8266-12 WiFi module, a 3.3V regulator, two buttons (flash and user) and a USB to Serial converter. I purchased my board from aliexpress for around $6US. I chose to use this board because it contains all that is needed to develop with the ESP8266.

I have erased the nodeMCU firmware and downloaded my custom Arduino code using the Arduino IDE 1.6.5 and the ESP8266 Board Package installed (Arduino-Compatible IDE for ESP8266 Github page).

The Arduino Code

The Arduino code:

#include <ESP8266WiFi.h>
#define ssid       "YourSSID"
#define pass       "YourWiFiPassword"
#define userName   "YourPlotlyUserName"
#define APIKey     "xxxxxxxxxx"
#define fileName   "test"
#define fileopt    "overwrite"
#define nTraces    1
#define maxpoints  "30"
#define world_readable true
#define convertTimestamp true
#define timezone "Australia/Melbourne"
char *tokens[nTraces] = {"xxxxxxxxxx"};
char stream_site[25] = {0};
WiFiClient client;
/* -------------- Setup ------------------ */
void setup() {
  // Setup Serial
  Serial.begin(9600);
  delay(2000);
  Serial.println();
  Serial.println();
  ESP8266_Init();
  ESP8266_Connect("plot.ly", 80);
  Serial.println("Initializing plot with Plot.ly server...");
  plotly_init();
  Serial.println("Making sure disconnected...");
  client.stop();
  Serial.println("Done!");
  ESP8266_Connect("arduino.plot.ly", 80);
}
/* ------------- Loop -------------------- */
void loop() {
  int val = analogRead(A0);
  plotly_plot(millis(),val,tokens[0]);
}
/* --------------- Functions -------------------- */
void ESP8266_Init(){
  Serial.println("-------------------------------------");
  Serial.println("           INITIALIZING...");
  Serial.println("-------------------------------------\r\n");
  // Put WiFi into Station mode
  Serial.println("1. Putting WiFi into station mode...");
  WiFi.mode(WIFI_STA);
  Serial.println("  Done!\r\n");
  // Connect to WiFi
  Serial.print("2. Connecting to: \"");
  Serial.print(ssid);
  Serial.print("\",\"");
  Serial.print(pass);
  Serial.println("\"...");
  WiFi.begin(ssid, pass);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println();
  Serial.println("  WiFi connected!");  
  Serial.print("  IP address: ");
  Serial.println(WiFi.localIP());
  Serial.println();
  Serial.println("-------------------------------------");
  Serial.println("      INITIALIZATION COMPLETE!");
  Serial.println("-------------------------------------\r\n");
}
void ESP8266_Connect(char* url, int port){
  Serial.print("Connecting to: \"");
  Serial.print(url);
  Serial.println("\" server...");
  if (!client.connect(url, port)) {
    Serial.println("*Connection Failed!*\r\n");
    while(1){};
  }
  else{Serial.println("Connected!");}
}
void plotly_init(){
  unsigned int i = 0;
  char charbuffer;
  // Calculate content length
  unsigned int contentLength = 126 + strlen(userName) + strlen(fileopt) + nTraces*(87+strlen(maxpoints)) + (nTraces - 1)*2 + strlen(fileName);
  if(world_readable){
    contentLength += 4;
  }
  else{
    contentLength += 5;
  }
  String contentLengthString = String(contentLength);
  const char* contentLengthConstString = contentLengthString.c_str();
  unsigned int postLength = contentLength + 94 + strlen(contentLengthConstString);
  // Send Post and initalization data
  client.print(
    String("POST /clientresp HTTP/1.1\r\n") +
    "Host: plot.ly:80\r\n" +
    "User-Agent: Arduino/0.6.0\r\n" +
    "Content-Type: application/x-www-form-urlencoded\r\n" +
    "Content-Length: " + String(contentLength) + "\r\n\r\n" +
    "version=2.3&origin=plot&platform=arduino&un="...
Read more »

  • 1 × NodeMCU Development Kit
  • 1 × Micro USB Cable
  • 3 × Female to Female Jumper Wire
  • 1 × LDR (Light Dependent Resistor) or Potentiometer + Wire
  • 1 × Small Lipo Battery (soldered on female header)

  • plotly's API Has Changed

    Johnny06/09/2016 at 14:13 0 comments

    I've updated the code to work with the current API (as of 11:42PM 9th June 2016).

    The code required "Content-Type: application/x-www-form-urlencoded\r\n" be placed in the POST header.

    POST /clientresp HTTP/1.1
    Host: plot.ly:80
    User-Agent: Arduino/0.6.0
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 241
    
    version=2.3&origin=plot&platform=arduino&un=johnnyizz&key=xxxxxxxxxx&args=[{"y": [], "x": [], "type": "scatter", "stream": {"token": "xxxxxxxxxx", "maxpoints": 30}}]&kwargs={"fileopt": "overwrite", "filename": "test", "world_readable": true}
    

    The rest of the code remains untouched.

    The API has change since I first wrote my code, it is much more flexible now, so some other modifications can be made to make the code more simple.

    This type POST will now work: (Notice Content-Length doesn't need calculation!)

    POST /clientresp HTTP/1.1
    Host: plot.ly
    Content-Type: application/x-www-form-urlencoded
    
    version=2.3&origin=plot&platform=arduino&un=johnnyizz&key=xxxxxxxxxx&args=[{"y": [], "x": [], "type": "scatter", "stream": {"token": "xxxxxxxxxx", "maxpoints": 30}}]&kwargs={"fileopt": "overwrite", "filename": "test", "world_readable": true}
    

    Data can now also be sent as: (Notice JSON string length doesn't need calculating!)

    POST  HTTP/1.1
    Host: arduino.plot.ly
    
    {"x": 1234, "y": 1234, "streamtoken": "xxxxxxxxxx"}
    
    or
    POST  HTTP/1.1
    Host: stream.plot.ly
    plotly-streamtoken: xxxxxxxxxx
    
    {"x": 1234, "y": 1234}
    

    A following "\r\n0\r\n\r\n" (as seen in the my code) will close the connection (you will need to resend the header) or 60 seconds of the server not receiving a '\n'. If the connection doesn't close, subsequent data (e.g. "{"x": 1234, "y": 1234}\n") can be sent.

  • City Strings Accepted Following "plotly-convertTimestamp:"

    Johnny06/09/2016 at 13:41 0 comments

    https://github.com/plotly/arduino-api/blob/master/Accepted%20Timezone%20Strings.txt

    "Africa/Abidjan",
    "Africa/Accra",
    "Africa/Addis_Ababa",
    "Africa/Algiers",
    "Africa/Asmara",
    "Africa/Bamako",
    "Africa/Bangui",
    "Africa/Banjul",
    "Africa/Bissau",
    "Africa/Blantyre",
    "Africa/Brazzaville",
    "Africa/Bujumbura",
    "Africa/Cairo",
    "Africa/Casablanca",
    "Africa/Ceuta",
    "Africa/Conakry",
    "Africa/Dakar",
    "Africa/Dar_es_Salaam",
    "Africa/Djibouti",
    "Africa/Douala",
    "Africa/El_Aaiun",
    "Africa/Freetown",
    "Africa/Gaborone",
    "Africa/Harare",
    "Africa/Johannesburg",
    "Africa/Juba",
    "Africa/Kampala",
    "Africa/Khartoum",
    "Africa/Kigali",
    "Africa/Kinshasa",
    "Africa/Lagos",
    "Africa/Libreville",
    "Africa/Lome",
    "Africa/Luanda",
    "Africa/Lubumbashi",
    "Africa/Lusaka",
    "Africa/Malabo",
    "Africa/Maputo",
    "Africa/Maseru",
    "Africa/Mbabane",
    "Africa/Mogadishu",
    "Africa/Monrovia",
    "Africa/Nairobi",
    "Africa/Ndjamena",
    "Africa/Niamey",
    "Africa/Nouakchott",
    "Africa/Ouagadougou",
    "Africa/Porto-Novo",
    "Africa/Sao_Tome",
    "Africa/Tripoli",
    "Africa/Tunis",
    "Africa/Windhoek",
    "America/Adak",
    "America/Anchorage",
    "America/Anguilla",
    "America/Antigua",
    "America/Araguaina",
    "America/Argentina/Buenos_Aires",
    "America/Argentina/Catamarca",
    "America/Argentina/Cordoba",
    "America/Argentina/Jujuy",
    "America/Argentina/La_Rioja",
    "America/Argentina/Mendoza",
    "America/Argentina/Rio_Gallegos",
    "America/Argentina/Salta",
    "America/Argentina/San_Juan",
    "America/Argentina/San_Luis",
    "America/Argentina/Tucuman",
    "America/Argentina/Ushuaia",
    "America/Aruba",
    "America/Asuncion",
    "America/Atikokan",
    "America/Bahia",
    "America/Bahia_Banderas",
    "America/Barbados",
    "America/Belem",
    "America/Belize",
    "America/Blanc-Sablon",
    "America/Boa_Vista",
    "America/Bogota",
    "America/Boise",
    "America/Cambridge_Bay",
    "America/Campo_Grande",
    "America/Cancun",
    "America/Caracas",
    "America/Cayenne",
    "America/Cayman",
    "America/Chicago",
    "America/Chihuahua",
    "America/Costa_Rica",
    "America/Creston",
    "America/Cuiaba",
    "America/Curacao",
    "America/Danmarkshavn",
    "America/Dawson",
    "America/Dawson_Creek",
    "America/Denver",
    "America/Detroit",
    "America/Dominica",
    "America/Edmonton",
    "America/Eirunepe",
    "America/El_Salvador",
    "America/Fortaleza",
    "America/Glace_Bay",
    "America/Godthab",
    "America/Goose_Bay",
    "America/Grand_Turk",
    "America/Grenada",
    "America/Guadeloupe",
    "America/Guatemala",
    "America/Guayaquil",
    "America/Guyana",
    "America/Halifax",
    "America/Havana",
    "America/Hermosillo",
    "America/Indiana/Indianapolis",
    "America/Indiana/Knox",
    "America/Indiana/Marengo",
    "America/Indiana/Petersburg",
    "America/Indiana/Tell_City",
    "America/Indiana/Vevay",
    "America/Indiana/Vincennes",
    "America/Indiana/Winamac",
    "America/Inuvik",
    "America/Iqaluit",
    "America/Jamaica",
    "America/Juneau",
    "America/Kentucky/Louisville",
    "America/Kentucky/Monticello",
    "America/Kralendijk",
    "America/La_Paz",
    "America/Lima",
    "America/Los_Angeles",
    "America/Lower_Princes",
    "America/Maceio",
    "America/Managua",
    "America/Manaus",
    "America/Marigot",
    "America/Martinique",
    "America/Matamoros",
    "America/Mazatlan",
    "America/Menominee",
    "America/Merida",
    "America/Metlakatla",
    "America/Mexico_City",
    "America/Miquelon",
    "America/Moncton",
    "America/Monterrey",
    "America/Montevideo",
    "America/Montreal",
    "America/Montserrat",
    "America/Nassau",
    "America/New_York",
    "America/Nipigon",
    "America/Nome",
    "America/Noronha",
    "America/North_Dakota/Beulah",
    "America/North_Dakota/Center",
    "America/North_Dakota/New_Salem",
    "America/Ojinaga",
    "America/Panama",
    "America/Pangnirtung",
    "America/Paramaribo",
    "America/Phoenix",
    "America/Port-au-Prince",
    "America/Port_of_Spain",
    "America/Porto_Velho",
    "America/Puerto_Rico",
    "America/Rainy_River",
    "America/Rankin_Inlet",
    "America/Recife",
    "America/Regina",
    "America/Resolute",
    "America/Rio_Branco",
    "America/Santa_Isabel",
    "America/Santarem",
    "America/Santiago",
    "America/Santo_Domingo",
    "America/Sao_Paulo",
    "America/Scoresbysund",
    "America/Shiprock",
    "America/Sitka",
    "America/St_Barthelemy",
    "America/St_Johns",
    "America/St_Kitts",
    "America/St_Lucia",
    "America/St_Thomas",
    ...
    Read more »

  • Speaking to PLOTLY

    Johnny05/14/2015 at 04:10 0 comments

    The ESP8266 needs to talk to the "plot.ly" server to initialize and create your data file and prepare it for receiving data. To do this firstly you will need to connect to the "plot.ly" server on "port 80", and send a POST request similar to this:

    POST /clientresp HTTP/1.1
    Host: plot.ly:80
    User-Agent: Arduino/0.6.0
    Content-Type: application/x-www-form-urlencoded
    Content-Length: 241
    
    version=2.3&origin=plot&platform=arduino&un=johnnyizz&key=xxxxxxxxxx&args=[{"y": [], "x": [], "type": "scatter", "stream": {"token": "xxxxxxxxxx", "maxpoints": 30}}]&kwargs={"fileopt": "overwrite", "filename": "test", "world_readable": true}

    Then finally you disconnect.

    This has created your file.

    *note that your own user name (I used "johnnyizz"), api-key and token will be inserted here, and your own filename and graph settings. Therefore the calculated "Content-Length" will be different for you (the formula for calculating this can be seen in the arduino code below). Also multiple stream tokens can be used for more graphs.

    // Calculate content length
    unsigned int contentLength = 126 + strlen(userName) + strlen(fileopt) + nTraces*(87+strlen(maxpoints)) + (nTraces - 1)*2 + strlen(fileName);
    if(world_readable){
      contentLength += 4;
    }
    else{
      contentLength += 5;
    }

    Once the file has been created in your account, we connect to the "arduino.plot.ly" server which will receive our data in jason form. Notice that the data is linked to your stream token. The following is the POST requests for sending data:

    POST / HTTP/1.1
    Host: arduino.plot.ly
    User-Agent: Arduino
    Transfer-Encoding: chunked
    Connection: close
    plotly-convertTimestamp: "Australia/Melbourne"
    
    0x34
    {"x": 1234, "y": 1234, "streamtoken": "xxxxxxxxxx"}
    
    0
    
    

    Here we can choose the time-stamp for our location (mine is "Australia/Melbourne"), the x and y data that's being sent, and our stream token.

    Note that the "0x34" above the jason string represents the length of characters making the jason string in hexadecimal.

    The following string equals 44 characters:

    string = "{/"x/":  ,  /"y/":  ,  /"streamtoken/":  /"xxxxxxxxxx/"}/n";

    44 + length of 'x' data + length of 'y' data = 52. Converted to hex is 0x34.

    Each time you send data, you repeat this.

    Finally, don't forget to use this formula for calculating the post length:

    String xString = String(x);
    String yString = String(y);
      
    const char* xConstString = xString.c_str();
    const char* yConstString = yString.c_str();
      
    unsigned int jasonLength = 44 + strlen(xConstString) +  strlen(yConstString);
    String jasonLengthString = String(jasonLength, HEX);
      
    const char* ConstJasonLengthString = jasonLengthString.c_str();
    unsigned int postLength = 167 + strlen(ConstJasonLengthString) + jasonLength;

View all 3 project logs

  • 1
    Step 1

    Visit the Plotly website and sign up. After that, go to your settings page, and generate a streaming token. Now write down your Username, API Key, and Token.

  • 2
    Step 2

    Visit the ESP8266/Arduino github page. Here you will see instructions to install the ESP8266 Board Package into Arduino IDE 1.6.5 or newer.

  • 3
    Step 3

    Past in the code I have included in my project details section into the Arduino IDE (1.6.5 or above), add your own WiFi SSID and Password, plotly username, API key and token into the code and save.

View all 6 instructions

Enjoy this project?

Share

Discussions

fred raducki wrote 03/20/2018 at 22:24 point

hello
thanks a lot for sharing this project. 
i see you made fixes in 2016. But now comes a new noob interested in this topic in 2018 and stumbles upon a (new?) error. 
May i ask for help. i had read everything user "shyam.sunder91" wrote and my problem seems quite similar. 
i have the "ERROR" on a similar moment. 
sharing here the content of my serial 

-------------------------------------
           INITIALIZING...
-------------------------------------

1. Putting WiFi into station mode...
  Done!

2. Connecting to: "MYWIFIID","MYCODE"...
.......
  WiFi connected!
  IP address: 192.168.1.X

-------------------------------------
      INITIALIZATION COMPLETE!
-------------------------------------

Connecting to: "plot.ly" server...
Connected!
Initializing plot with Plot.ly server...
*ERROR!*

Soft WDT reset

ctx: cont 
sp: 3ffefed0 end: 3fff0110 offset: 01b0

>>>stack>>>
3fff0080:  5815be23 3fff009c 3ffeee58 40202625  
3fff0090:  00000000 00000000 00000000 00000000  
3fff00a0:  00000000 00000000 00000000 00000000  
3fff00b0:  00000000 00000000 00000000 00000000  
3fff00c0:  3fff0f4c 0000000f 00000003 40203818  
3fff00d0:  4020137a 3ffe8d30 3ffef0bc 3ffef0e8  
3fff00e0:  3fffdad0 00000000 3ffef0bc 40202697  
3fff00f0:  feefeffe feefeffe 3ffef0e0 40203ed0  
3fff0100:  feefeffe feefeffe 3ffef0f0 40100710  
<<<stack<<<
/⸮⸮⸮,⸮⸮⸮

Any ideas ?

  Are you sure? yes | no

Johnny wrote 03/28/2018 at 14:27 point

Just saw your comment. I don't get notifications for old projects for some reason. If I get time over Easter break, I'll take a look.

Johnny.

  Are you sure? yes | no

fred raducki wrote 03/30/2018 at 12:52 point

actually i ended up using thinger.io it worked like a charm with Arduino uno and Ethernet shield and also with NodemcuV3. so i'm leaving plotly aside ;-)

  Are you sure? yes | no

ru4mj12 wrote 07/19/2016 at 05:06 point

I'm getting the "All Streams Go" and the url to the chart, but the streaming data isn't plotting.   i.e.

34
{"x": 7645, "y": 5000, "streamtoken": "2m5iq5n4ve"}
I tried using wireshark to capture the traffic, but not very familiar with how to parse or see why it's not plotting 
https://www.cloudshark.org/captures/ba50fba66815   (192.168.137.40)
Any idea how to format the capture to see what is causing the issue?
Thanks!

  Are you sure? yes | no

shyam.sunder91 wrote 05/27/2016 at 13:55 point

really glad to see your project, when i tried to use it in as it is with the NodeMCU board and necessary steps , i still see the "ERROR" on the command console, the mistake with my modification lies at plotly_init, i have entered few fields in your code like wifi username, paswd, and the username of plotly, the API key generated from my account and the streaming char *tokens[nTraces] = {" i changed this to generated token "};, no other things are changed, i see a return value -1 and did not find the ~ , so even i replaced the  "~" with my username in client.find , still the error persists, registering primarily with plotly after my acc creation i have gone to settings and copied the API key and used it in code and then generated the token and used it in code i dont think any other modifications are needed with plotly, coming to the code debugging i have just put a copy of client. print in Serial.print format to see what am i exactly sending to client.print ?, 

it is as follows

POST /clientresp HTTP/1.1
Host: plot.ly:80
User-Agent: Arduino/0.6.0
Content-Length: 246

version=2.3&origin=plot&platform=arduino&un=shyam.sunder91&key=xxxxxxxxxxxx&args=[{"y": [], "x": [], "type": "scatter", "stream": {"token": "xxxxxxxxxxx", "maxpoints": 30}}]&kwargs={"fileopt": "overwrite", "filename": "test", "world_readable": true}

then comes the client.find("~") check where the program fails and returns ERROR, so to the commands plotyly did not respond with my url, which means there is error with plotly configuration that i have done, i have re-checked the credentials but still the error persists

where exactly i am wrong i am not able to trace out !

 i am really new to the arduino and esp thing, kindly help me in tracing out the bug

  Are you sure? yes | no

Johnny wrote 05/31/2016 at 05:33 point

Hey Shyam,

Thanks for the comment. Sorry I haven't looked at this in a while, so would take some time for me to get back up to speed on how it all worked. When I originally wrote this, it was working well. What could have happened is either plotly's API has changed or the firmware for the NodeMCU has changed. If I get the time, I'll take a look further into this.

  Are you sure? yes | no

Johnny wrote 06/08/2016 at 13:36 point

I did more poking around and it seems manually trying to connect to plotly's streaming server also doesn't work. After looking at the latest NODE.JS libraries, it seems as though they are now operating different? Maybe the Arduino code was never updated to the newer API. I'll investigate further. Shouldn't be too hard to fix.

John.

  Are you sure? yes | no

Johnny wrote 06/09/2016 at 14:19 point

Issues have been fixed. The API had changed since I first was playing around with this. I have tested the code and now works 100%. Give it another go.

Good luck!

Johnny.

  Are you sure? yes | no

Johnny wrote 10/31/2015 at 10:18 point

Thanks. Sweet, let me know how you go!

  Are you sure? yes | no

Esplod wrote 10/30/2015 at 07:14 point

Very nicely documented. I have an ESP8266 with sensor talking to thingspeak and look forward to trying to connect to Plot.ly over the weekend.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates