LowPowerLab Forum

Software support => Pi Gateway => Topic started by: Kylix on February 26, 2018, 01:05:41 PM

Title: Using request.post with different Thermostat [solution]
Post by: Kylix on February 26, 2018, 01:05:41 PM
I'm using for almost 2 years an ESP8266 based thermostat which already provides the state in Json format and I'm trying to use it with the Gateway.
I managed to add the dummy thermostat node and make it insert all the values in the DB (target temperature, actual temperature, ecc.)

(https://ibin.co/w800/3t3ui8dp9S7w.jpg)

I'm having problems changing the parameters: target temperature, operating mode...and so on.
The target temperature, for example, can be changed by sending a POST request to http://IPOFTHETHERMOSTAT/control/thermostat.cgi?param=thermostat_manualsetpoint and in the BODY, 2000 as value (it sets the target temperature to 20°C). I tested this request with Advanced REST Client and I was able to change the target temperature.

I have no idea what I'm doing wrong in metrics.js file. I tried this:

      HEAT17 : { states: [{ label:'Heat:17°', action:'', icon:'fa-fire',
                            serverExecute:function(node){
                              var targetNow=0, modeNow='';
                              //if (node.metrics['TSTATE']) modeNow = node.metrics['TSTATE'].value;
                              //if (node.metrics['TARGET']) targetNow = node.metrics['TARGET'].value;
                              //if (targetNow == 1700 && modeNow=='MANUAL') return;
                              var thejson = '1700';
      var param = 'thermostat_manualsetpoint';
                              exports.tstatRequest(param, thejson, node._id);
                            },
                         }],
                  breakAfter:true,
               },

...

exports.tstatRequest = function(param, thejson, nodeId) {
  //console.log('tstatRequest:' + JSON.stringify(thejson));
  db.findOne({ _id : nodeId }, function (err, dbNode) {
    var thermostatIP = dbNode.settings && dbNode.settings.ip ? dbNode.settings.ip : (exports.motes[dbNode.type].settings.ip || settings.radiothermostat.ip.value);
   //console.log('URL: ' + 'http://'+thermostatIP+'/control/thermostat.cgi?param=' + param);
    request.post({ headers: {'Content-Type':'application/javascript'}, url: 'http://'+thermostatIP+'/control/thermostat.cgi?param=' + param,
                          body: thejson},

              function(error,response,body){
                console.log('BODY: ' + JSON.stringify(body));
                if (error) console.log('ERROR in tstatRequest(): ' + JSON.stringify(thejson) + ' nodeId:' + nodeId + ' - ' + error);
                else exports.tstatPoll(nodeId); //now ask for a refresh of status from thermostat (HTTP GET)
              }
    );
  });
}


Any suggestion?
Title: Re: Different type of thermostat - request.post not working
Post by: Felix on February 26, 2018, 04:23:37 PM
I suggest going to your Pi console, start node, and import the request module as it's done in gateway.js, then test your actual requests with that module. Then once it's working there you can move that code in your metrics.
Any further errors should be logged in gateway.sys.log
Title: Re: Different type of thermostat - request.post not working
Post by: Kylix on February 27, 2018, 03:08:33 PM
I'm getting this:

(https://ibin.co/3tBUZ1DX9h3b.jpg)

I also watched the gateway.sys.log file but I have no error regarding the POST Request.

In Advanced REST Client, the request works well and is set like this (the body is plain text):

(https://ibin.co/w800/3tBps0sZwEg3.jpg)

And this is the actual (working) Javascript function that the thermostat has for changing various parameters:

function save(param, payload) {
doingsave=true;
    $.ajax({
        type: 'POST',
        url: "thermostat.cgi?param=" + param,
        data: payload,
dataType: 'text',
cache: false,
        async: true,
timeout: 3000,
tryCount : 0,
retryLimit : 3, success: function (data) {
statusMsg = false;
if(!connected) setStatus("Connected",2,0);
connected=true;
doingsave=false;
},
error : function(xhr, textStatus, errorThrown ) {
        if (textStatus == 'timeout') {
            this.tryCount++;
            if (this.tryCount <= this.retryLimit) {
                //try again
                $.ajax(this);
                return;
            }           
            return;
        }
if(connected) setStatus("No connection to server!",0,1);
connected=false;
doingsave=false;
}
    });
}
Title: Re: Different type of thermostat - request.post not working
Post by: Felix on February 28, 2018, 12:08:41 PM
You obviously did not use node console before :)
But hopefully you used javascript, the undefined is normal. It's the result of the statement you just typed.
Title: Re: Different type of thermostat - request.post not working
Post by: Kylix on February 28, 2018, 04:18:32 PM
You're right Felix, I have no experience with nodeJS or javascript.
I tried as you suggested but it doesn't give me a clue about any error at all:
> request.post({ headers: {'Content-Type':'application/x-www-form-urlencoded'}, url: 'http://192.168.0.250/control/thermostat.cgi?param=thermostat_manualsetpoint', body:'2000'}, function(error,response,body){if (error) console.log('ERROR:' + error);});
Request {
  domain:
   Domain {
     domain: null,
     _events: { error: [Function] },
     _eventsCount: 1,
     _maxListeners: undefined,
     members: [] },
  _events:
   { error: [Function: bound ],
     complete: [Function: bound ],
     pipe: [Function] },
  _eventsCount: 3,
  _maxListeners: undefined,
  headers:
   { 'Content-Type': 'application/x-www-form-urlencoded',
     host: '192.168.0.250',
     'content-length': 4 },
  body: '2000',
  callback: [Function],
  method: 'POST',
  readable: true,
  writable: true,
  explicitMethod: true,
  _qs:
   Querystring {
     request: [Circular],
     lib: { formats: [Object], parse: [Function], stringify: [Function] },
     useQuerystring: undefined,
     parseOptions: {},
     stringifyOptions: {} },
  _auth:
   Auth {
     request: [Circular],
     hasAuth: false,
     sentAuth: false,
     bearerToken: null,
     user: null,
     pass: null },
  _oauth: OAuth { request: [Circular], params: null },
  _multipart:
   Multipart {
     request: [Circular],
     boundary: 'd9039101-86f0-4bb7-8594-e3c5ba5d49f6',
     chunked: false,
     body: null },
  _redirect:
   Redirect {
     request: [Circular],
     followRedirect: true,
     followRedirects: true,
     followAllRedirects: false,
     followOriginalHttpMethod: false,
     allowRedirect: [Function],
     maxRedirects: 10,
     redirects: [],
     redirectsFollowed: 0,
     removeRefererHeader: false },
  _tunnel:
   Tunnel {
     request: [Circular],
     proxyHeaderWhiteList:
      [ 'accept',
        'accept-charset',
        'accept-encoding',
        'accept-language',
        'accept-ranges',
        'cache-control',
        'content-encoding',
        'content-language',
        'content-location',
        'content-md5',
        'content-range',
        'content-type',
        'connection',
        'date',
        'expect',
        'max-forwards',
        'pragma',
        'referer',
        'te',
        'user-agent',
        'via' ],
     proxyHeaderExclusiveList: [] },
  setHeader: [Function],
  hasHeader: [Function],
  getHeader: [Function],
  removeHeader: [Function],
  localAddress: undefined,
  pool: {},
  dests: [],
  __isRequestRequest: true,
  _callback: [Function],
  uri:
   Url {
     protocol: 'http:',
     slashes: true,
     auth: null,
     host: '192.168.0.250',
     port: 80,
     hostname: '192.168.0.250',
     hash: null,
     search: '?param=thermostat_manualsetpoint',
     query: 'param=thermostat_manualsetpoint',
     pathname: '/control/thermostat.cgi',
     path: '/control/thermostat.cgi?param=thermostat_manualsetpoint',
     href: 'http://192.168.0.250/control/thermostat.cgi?param=thermostat_manualsetpoint' },
  proxy: null,
  tunnel: false,
  setHost: true,
  originalCookieHeader: undefined,
  _disableCookies: true,
  _jar: undefined,
  port: 80,
  host: '192.168.0.250',
  path: '/control/thermostat.cgi?param=thermostat_manualsetpoint',
  httpModule:
   { IncomingMessage: { [Function: IncomingMessage] super_: [Object] },
     METHODS:
      [ 'ACL',
        'BIND',
        'CHECKOUT',
        'CONNECT',
        'COPY',
        'DELETE',
        'GET',
        'HEAD',
        'LINK',
        'LOCK',
        'M-SEARCH',
        'MERGE',
        'MKACTIVITY',
        'MKCALENDAR',
        'MKCOL',
        'MOVE',
        'NOTIFY',
        'OPTIONS',
        'PATCH',
        'POST',
        'PROPFIND',
        'PROPPATCH',
        'PURGE',
        'PUT',
        'REBIND',
        'REPORT',
        'SEARCH',
        'SUBSCRIBE',
        'TRACE',
        'UNBIND',
        'UNLINK',
        'UNLOCK',
        'UNSUBSCRIBE' ],
     OutgoingMessage: { [Function: OutgoingMessage] super_: [Object] },
     ServerResponse: { [Function: ServerResponse] super_: [Object] },
     STATUS_CODES:
      { '100': 'Continue',
        '101': 'Switching Protocols',
        '102': 'Processing',
        '200': 'OK',
        '201': 'Created',
        '202': 'Accepted',
        '203': 'Non-Authoritative Information',
        '204': 'No Content',
        '205': 'Reset Content',
        '206': 'Partial Content',
        '207': 'Multi-Status',
        '208': 'Already Reported',
        '226': 'IM Used',
        '300': 'Multiple Choices',
        '301': 'Moved Permanently',
        '302': 'Found',
        '303': 'See Other',
        '304': 'Not Modified',
        '305': 'Use Proxy',
        '307': 'Temporary Redirect',
        '308': 'Permanent Redirect',
        '400': 'Bad Request',
        '401': 'Unauthorized',
        '402': 'Payment Required',
        '403': 'Forbidden',
        '404': 'Not Found',
        '405': 'Method Not Allowed',
        '406': 'Not Acceptable',
        '407': 'Proxy Authentication Required',
        '408': 'Request Timeout',
        '409': 'Conflict',
        '410': 'Gone',
        '411': 'Length Required',
        '412': 'Precondition Failed',
        '413': 'Payload Too Large',
        '414': 'URI Too Long',
        '415': 'Unsupported Media Type',
        '416': 'Range Not Satisfiable',
        '417': 'Expectation Failed',
        '418': 'I\'m a teapot',
        '421': 'Misdirected Request',
        '422': 'Unprocessable Entity',
        '423': 'Locked',
        '424': 'Failed Dependency',
        '425': 'Unordered Collection',
        '426': 'Upgrade Required',
        '428': 'Precondition Required',
        '429': 'Too Many Requests',
        '431': 'Request Header Fields Too Large',
        '451': 'Unavailable For Legal Reasons',
        '500': 'Internal Server Error',
        '501': 'Not Implemented',
        '502': 'Bad Gateway',
        '503': 'Service Unavailable',
        '504': 'Gateway Timeout',
        '505': 'HTTP Version Not Supported',
        '506': 'Variant Also Negotiates',
        '507': 'Insufficient Storage',
        '508': 'Loop Detected',
        '509': 'Bandwidth Limit Exceeded',
        '510': 'Not Extended',
        '511': 'Network Authentication Required' },
     Agent: { [Function: Agent] super_: [Object], defaultMaxSockets: Infinity },
     globalAgent:
      Agent {
        domain: [Object],
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        defaultPort: 80,
        protocol: 'http:',
        options: [Object],
        requests: {},
        sockets: {},
        freeSockets: {},
        keepAliveMsecs: 1000,
        keepAlive: false,
        maxSockets: Infinity,
        maxFreeSockets: 256 },
     ClientRequest: { [Function: ClientRequest] super_: [Object] },
     request: [Function],
     get: [Function],
     _connectionListener: [Function: connectionListener],
     Server: { [Function: Server] super_: [Object] },
     createServer: [Function],
     Client: [Function: deprecated],
     createClient: [Function: deprecated] },
  agentClass:
   { [Function: Agent]
     super_:
      { [Function: EventEmitter]
        EventEmitter: [Circular],
        usingDomains: true,
        defaultMaxListeners: 10,
        init: [Function],
        listenerCount: [Function] },
     defaultMaxSockets: Infinity },
  agent:
   Agent {
     domain:
      Domain {
        domain: null,
        _events: [Object],
        _eventsCount: 1,
        _maxListeners: undefined,
        members: [] },
     _events: { free: [Function] },
     _eventsCount: 1,
     _maxListeners: undefined,
     defaultPort: 80,
     protocol: 'http:',
     options: { path: null },
     requests: {},
     sockets: {},
     freeSockets: {},
     keepAliveMsecs: 1000,
     keepAlive: false,
     maxSockets: Infinity,
     maxFreeSockets: 256 } }
>
Title: Re: Different type of thermostat - request.post not working
Post by: Felix on March 01, 2018, 02:21:25 PM
Take incremental steps. Read the request module documentation (https://github.com/request/request), and build simple requests, so you understand what is going on.
You can also use curl in the Pi console to run http requests.
Title: Re: Different type of thermostat - request.post not working
Post by: Kylix on March 01, 2018, 05:15:29 PM
I've just tried with curl in Pi's console and this works:

curl -d "2000" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://192.168.0.250/control/thermostat.cgi?param=thermostat_manualsetpoint

How to convert it for using in metrics.js file?
Title: Re: Different type of thermostat - request.post not working
Post by: Felix on March 02, 2018, 08:22:23 AM
I don't have the thermostat you have so I can't try it.

Did you see the request examples?

https://github.com/request/request#forms
Also see:
https://stackoverflow.com/questions/6432693/post-data-with-request-module-on-node-js
Title: Re: Different type of thermostat - request.post not working
Post by: Kylix on March 03, 2018, 05:48:16 AM
Hi Felix!

I already tried both options.
The thermostat I have, if you send a GET request like this: http://192.168.0.250/control/thermostat.cgi?param=thermostat_manualsetpoint , returns the value of the target temperature

If I send a POST request to http://192.168.0.250/control/thermostat.cgi?param=thermostat_manualsetpoint with 2000 as value, it sets the target temperature to 20°C

Somehow, the nodejs POST request doesn't get well interpreted and it returns the target temperature instead of changing it to the desired POST value. The thermostat surely accepts the POST request (and it corectly changes the target temperature) if I send it with curl, as you suggested:
curl -d "2000" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://192.168.0.250/control/thermostat.cgi?param=thermostat_manualsetpoint

The thermostat C code that manages the HTTP requests is this (I removed non relevant parts of the code):

int ICACHE_FLASH_ATTR cgiThermostat(HttpdConnData *connData) {
char buff[2048];
char temp[128];
char humi[32];
int len=0;

httpdStartResponse(connData, 200);
httpdHeader(connData, "Content-Type", "application/json");
httpdHeader(connData, "Access-Control-Allow-Origin", "*");
httpdEndHeaders(connData);

if (connData->conn==NULL) {
//Connection aborted. Clean up.
return HTTPD_CGI_DONE;
}

os_strcpy(buff, "Unknown");
os_strcpy(temp, "N/A");
os_strcpy(humi, "N/A");

len=httpdFindArg(connData->getArgs, "param", buff, sizeof(buff));
if (len>0) {


if(os_strcmp(buff,"thermostat_manualsetpoint")==0) {
if(connData->post->len>0) {
sysCfg.thermostat1manualsetpoint=(int)atoi(connData->post->buff);
CFG_Save();
os_printf("Handle thermostat manual setpoint save (%d)\n",(int)sysCfg.thermostat1manualsetpoint);
} else {
os_sprintf(buff, "%d", (int)sysCfg.thermostat1manualsetpoint);
}
}

}

httpdSend(connData, buff, -1);
return HTTPD_CGI_DONE;
}
Title: [SOLVED] Different type of thermostat - request.post not working
Post by: Kylix on March 03, 2018, 01:01:35 PM
Solved by using curlrequest  :D :

var curl = require('curlrequest');
var options = {
    url: 'http://192.168.0.250/control/thermostat.cgi?param=' + param,
    method: 'POST',
    data: thejson
};

curl.request(options, function (err, response) {
        if (err) console.log('ERROR in tstatRequest(): ' + JSON.stringify(thejson) + ' nodeId:' + nodeId + ' - ' + error);
     else exports.tstatPoll(nodeId); //now ask for a refresh of status from thermostat (HTTP GET)
});
Title: Re: Using request.post with different Thermostat [solution]
Post by: Felix on March 03, 2018, 07:19:28 PM
Great, congratulations!
I don't know request by heart and while it's pretty extensive and full of options there may be simpler options out there for simple requests.
Title: Re: Using request.post with different Thermostat [solution]
Post by: Kylix on March 04, 2018, 03:37:11 AM
Curl "request" uses almost the same code as request.post ... I have no idea why it didn't work. I spent 3 days trying all the solutions found on the web and nothing....

In case somebody is interested: I'm using a simple and cheap DIY thermostat, based on ESP8266 (Olimex relay board: https://www.olimex.com/Products/IoT/ESP8266-EVB/open-source-hardware) + DS18B20 with a Moteino + RFM69HW connected through serial. A remote Moteino + RFM69HW node + BME680 node, sends: temperature, pressure, humidity, battery level (not absolutely necessary. I'm using it because I have the main unit connected to the gas boiler, outside the house and I needed the inside temperature).

The thermostat project is not mine (I only slightly changed it in order to have the remote node parameters displayed on the main page) and can be found here: https://github.com/mharizanov/ESP8266_Relay_Board

(https://community.openenergymonitor.org/uploads/default/optimized/2X/0/0cd67d55a715e793de627caf7e99141fe29a66ed_1_322x500.png)

It also sends the sensor data to Thingspeak:

(https://community.openenergymonitor.org/uploads/default/original/2X/8/83c1f23029ec99f3a89eedc01223f1a51f39d3e1.png)
Title: Re: Using request.post with different Thermostat [solution]
Post by: LukaQ on March 04, 2018, 04:44:56 AM
Cool, I use the same type of graph. Would you share that code, I don't think I got more than two different Y axis working
Title: Re: Using request.post with different Thermostat [solution]
Post by: Kylix on March 04, 2018, 05:06:05 AM
This is the thingspeak.js file:

   
      // Webpage Javascript to chart multiple ThingSpeak channels on two axis with navigator, load historical data, and export cvs data.
      // Public Domain, by turgo.
      //  The charting library is called HighStock.  It is awesome!  HighSoft, the owners say,
      //   "Do you want to use Highstock for a personal or non-profit project? Then you can use Highchcarts for
      //   free under the  Creative Commons Attribution-NonCommercial 3.0 License. "
var dynamicChart;
var channelsLoaded = 0;
// put your ThingSpeak Channel#, Channel Name, and API keys here.
// fieldList shows which field you want to load, and which axis to display that field on,
// the 'T' Temperature left axis, or the 'O' Other right axis.
var channelKeys =[];
channelKeys.push({channelNumber: channel , name:'ESP8266',key: ro_apikey,
                  fieldList:[{field:1,axis:'R'},{field:2,axis:'T'},{field:3,axis:'H'},{field:4,axis:'T'},{field:5,axis:'P'},{field:6,axis:'S'}]});
 

/*channelKeys.push({channelNumber:5683, name:'Sam Rogers Solar System',key:'XXXXXXXXXXXXXXXX',
   fieldList:[{field:1,axis:'T'},{field:2,axis:'T'},{field:3,axis:'T'},{field:4,axis:'T'},{field:5,axis:'T'},{field:6,axis:'T'}]});
channelKeys.push({channelNumber:9354, name:'CRYSTAL CITY WEATHER Fault',key:'XXXXXXXXXXXXXXXX',
   fieldList:[{field:1,axis:'T'},{field:2,axis:'O'},{field:3,axis:'O'},{field:4,axis:'O'},{field:5,axis:'O'},{field:6,axis:'O'}]});
  */
 
// user's timezone offset
var myOffset = new Date().getTimezoneOffset();

// converts date format from JSON
function getChartDate(d) {
    // get the data using javascript's date object (year, month, day, hour, minute, second)
    // months in javascript start at 0, so remember to subtract 1 when specifying the month
    // offset in minutes is converted to milliseconds and subtracted so that chart's x-axis is correct
    return Date.UTC(d.substring(0,4), d.substring(5,7)-1, d.substring(8,10), d.substring(11,13), d.substring(14,16), d.substring(17,19)) - (myOffset * 60000);
}

      // Hide all series, via 'Hide All' button.  Then user can click on serries name in legent to show series of interest.     
function HideAll(){
  for (var index=0; index<dynamicChart.series.length; index++)  // iterate through each series
  {
    if (dynamicChart.series[index].name == 'Navigator')
      continue;
    dynamicChart.series[index].hide();
    //window.console && console.log('Series Number:',index,' Name:',dynamicChart.series[index].name);
  }
  //});
           
}
     
      //  This is where the chart is generated.
$(document).ready(function()
{
//Add Channel Load Menu
var menu=document.getElementById("Channel Select");
for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
{
   window.console && console.log('Name',channelKeys[channelIndex].name);
   var menuOption =new Option(channelKeys[channelIndex].name,channelIndex);
   menu.options.add(menuOption,channelIndex);
}
var last_date; // variable for the last date added to the chart
window.console && console.log('Testing console');
//make series numbers for each field
var seriesCounter=0
for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
  {
    for (var fieldIndex=0; fieldIndex<channelKeys[channelIndex].fieldList.length; fieldIndex++)  // iterate through each channel
      {
        channelKeys[channelIndex].fieldList[fieldIndex].series = seriesCounter;
        seriesCounter++;
      }
  }
//make calls to load data from each channel into channelKeys array now
// draw the chart when all the data arrives, later asyncronously add history
for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
  {
    channelKeys[channelIndex].loaded = false; 
    loadThingSpeakChannel(channelIndex,channelKeys[channelIndex].channelNumber,channelKeys[channelIndex].key,channelKeys[channelIndex].fieldList);
   
  }
//window.console && console.log('Channel Keys',channelKeys);

// load the most recent 2500 points (fast initial load) from a ThingSpeak channel into a data[] array and return the data[] array
function loadThingSpeakChannel(sentChannelIndex,channelNumber,key,sentFieldList) {
   var fieldList= sentFieldList;
   var channelIndex = sentChannelIndex;
   // get the Channel data with a webservice call
$.getJSON('https://www.thingspeak.com/channels/'+channelNumber+'/feed.json?callback=?&amp;offset=0&amp;results=2500;key='+key, function(data)
   {
   // if no access
   if (data == '-1') {
       $('#chart-container').append('This channel is not public.  To embed charts, the channel must be public or a read key must be specified.');
       window.console && console.log('Thingspeak Data Loading Error');
     }
     for (var fieldIndex=0; fieldIndex<fieldList.length; fieldIndex++)  // iterate through each field
     {
       fieldList[fieldIndex].data =[];
       for (var h=0; h<data.feeds.length; h++)  // iterate through each feed (data point)
       {
         var p = []//new Highcharts.Point();
         var fieldStr = "data.feeds["+h+"].field"+fieldList[fieldIndex].field;
  var v = eval(fieldStr);
  p[0] = getChartDate(data.feeds[h].created_at);
  p[1] = parseFloat(v);
  // if a numerical value exists add it
    if (!isNaN(parseInt(v))) { fieldList[fieldIndex].data.push(p); }
       }
       fieldList[fieldIndex].name = eval("data.channel.field"+fieldList[fieldIndex].field);
   }
     window.console && console.log('getJSON field name:',fieldList[0].name);
     channelKeys[channelIndex].fieldList=fieldList;
     channelKeys[channelIndex].loaded=true;
     channelsLoaded++;
     window.console && console.log('channels Loaded:',channelsLoaded);
     window.console && console.log('channel index:',channelIndex);
     if (channelsLoaded==channelKeys.length){createChart();}
})
   .fail(function() { alert('getJSON request failed! '); });
}
// create the chart when all data is loaded
function createChart() {
// specify the chart options
var chartOptions = {
  chart:
    {
  renderTo: 'chart-container',
      zoomType:'y',
events:
      {
        load: function()
        {
  if ('true' === 'true' && (''.length < 1 && ''.length < 1 && ''.length < 1 && ''.length < 1 && ''.length < 1))
          {
            // If the update checkbox is checked, get latest data every 15 seconds and add it to the chart
setInterval(function()
            {
             if (document.getElementById("Update").checked)
             {
              for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
              { 
               (function(channelIndex)
               {
                // get the data with a webservice call
                $.getJSON('https://www.thingspeak.com/channels/'+channelKeys[channelIndex].channelNumber+'/feed/last.json?callback=?&amp;offset=0&amp;location=false;key='+channelKeys[channelIndex].key, function(data)
                {
                  for (var fieldIndex=0; fieldIndex<channelKeys[channelIndex].fieldList.length; fieldIndex++)
                  {
                    // if data exists
                    var fieldStr = "data.field"+channelKeys[channelIndex].fieldList[fieldIndex].field;
                    var chartSeriesIndex=channelKeys[channelIndex].fieldList[fieldIndex].series;
                    if (data && eval(fieldStr))
                    {
                      var p = []//new Highcharts.Point();
                      var v = eval(fieldStr);
                      p[0] = getChartDate(data.created_at);
                      p[1] = parseFloat(v);
                      // get the last date if possible
                      if (dynamicChart.series[chartSeriesIndex].data.length > 0)
                      {
                        last_date = dynamicChart.series[chartSeriesIndex].data[dynamicChart.series[chartSeriesIndex].data.length-1].x;
                      }
                      var shift = false ; //default for shift
                      // if a numerical value exists and it is a new date, add it
                      if (!isNaN(parseInt(v)) && (p[0] != last_date))
                      {
                        dynamicChart.series[chartSeriesIndex].addPoint(p, true, shift);
                      }   
                    }
                    //window.console && console.log('channelKeys:',channelKeys);
                    //window.console && console.log('chartSeriesIndex:',chartSeriesIndex);
                    //window.console && console.log('channel index:',channelIndex);
                    //window.console && console.log('field index:',fieldIndex);
                    //window.console && console.log('update series name:',dynamicChart.series[chartSeriesName].name);
                    //window.console && console.log('channel keys name:',channelKeys[channelIndex].fieldList[fieldIndex].name);
                  }
                 
                 
                });
               })(channelIndex);
              }
             }
}, 15000);
}
}
}
},
rangeSelector: {
buttons: [{
count: 30,
type: 'minute',
text: '30M'
}, {
count: 12,
type: 'hour',
text: '12H'
      }, {
count: 1,
type: 'day',
text: 'D'
      }, {
count: 1,
type: 'week',
text: 'W'
      }, {
count: 1,
type: 'month',
text: 'M'
      }, {
count: 1,
type: 'year',
text: 'Y'
}, {
type: 'all',
text: 'All'
}],
inputEnabled: true,
selected: 1
},
    title: {
text: ''
},
plotOptions: {
  line: {
        gapSize:5
//color: '#d62020'
//  },
//  bar: {
//color: '#d52020'
//  },
//  column: {
},
series: {
  marker: {
  radius: 2
},
animation: true,
step: false,
        turboThrehold:1000,
borderWidth: 0
}
},
    tooltip: {
      valueDecimals: 1,
      valueSuffix: '',
      xDateFormat:'%Y-%m-%d<br/>%H:%M:%S %p'
// reformat the tooltips so that local times are displayed
//formatter: function() {
      //var d = new Date(this.x + (myOffset*60000));
      //var n = (this.point.name === undefined) ? '' : '<br/>' + this.point.name;
      //return this.series.name + ':<b>' + this.y + '</b>' + n + '<br/>' + d.toDateString() + '<br/>' + d.toTimeString().replace(/\(.*\)/, "");
//}
    },
xAxis: {
  type: 'datetime',
      ordinal:false,
      min: Date.UTC(2013,02,28),
dateTimeLabelFormats : {
        hour: '%H %p',
        minute: '%H:%M %p'
      },
      title: {
        text: 'test'
}
},
yAxis: [{
            title: {
                text: 'Temperature °C'
            },
            id: 'T'
    }, {
            title: {
                text: 'Humidity %'
            },
            opposite: true,
            id: 'H'
    }, {
            title: {
                text: 'Pressure mBar'
            },
            opposite: true,
            id: 'P'
    }, {
            title: {
                text: 'RSSI -dB'
            },
            opposite: true,
            id: 'S'
    }, {
            title: {
                text: 'Relay State'
            },
            opposite: true,
            id: 'R'
    }
],
exporting: {
  enabled: true,
      csv: {
        dateFormat: '%d/%m/%Y %I:%M:%S %p'
        }
},
legend: {
  enabled: true
},
    navigator: {
      baseSeries: 0,  //select which series to show in history navigator, First series is 0
      series: {
            includeInCSVExport: false
        }
},   
    series: []
    //series: [{data:[[getChartDate("2013-06-16T00:32:40Z"),75]]}]     
};

// add all Channel data to the chart
  for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
  {
    for (var fieldIndex=0; fieldIndex<channelKeys[channelIndex].fieldList.length; fieldIndex++)  // add each field
    {
      window.console && console.log('Channel '+channelIndex+' field '+fieldIndex);
      chartOptions.series.push({data:channelKeys[channelIndex].fieldList[fieldIndex].data,
                                index:channelKeys[channelIndex].fieldList[fieldIndex].series,
                                yAxis:channelKeys[channelIndex].fieldList[fieldIndex].axis,
                                //visible:false,
                              name: channelKeys[channelIndex].fieldList[fieldIndex].name});
    }
  }
// set chart labels here so that decoding occurs properly
//chartOptions.title.text = data.channel.name;
chartOptions.xAxis.title.text = 'Date';

// draw the chart
  dynamicChart = new Highcharts.StockChart(chartOptions);

  // update series number to account for the navigator series (The historical series at the bottom) which is the first series.
  for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
  {
    for (var fieldIndex=0; fieldIndex<channelKeys[channelIndex].fieldList.length; fieldIndex++)  // and each field
    {
      for (var seriesIndex=0; seriesIndex<dynamicChart.series.length; seriesIndex++)  // compare each series name
      {
        if (dynamicChart.series[seriesIndex].name == channelKeys[channelIndex].fieldList[fieldIndex].name)
        {
          channelKeys[channelIndex].fieldList[fieldIndex].series = seriesIndex;
        }
      }
    }
  }         
  // add all history
  //dynamicChart.showLoading("Loading History..." );
  window.console && console.log('Channels: ',channelKeys.length);
  for (var channelIndex=0; channelIndex<channelKeys.length; channelIndex++)  // iterate through each channel
  {
    window.console && console.log('channelIndex: ',channelIndex);
    (function(channelIndex)
      {
        //load only 1 set of 8000 points
        loadChannelHistory(channelIndex,channelKeys[channelIndex].channelNumber,channelKeys[channelIndex].key,channelKeys[channelIndex].fieldList,0,1);
      }
    )(channelIndex);
  }
}
});
     
function loadOneChannel(){
  // load a channel selected in the popUp menu.
  var selectedChannel=document.getElementById("Channel Select");
  var maxLoads=document.getElementById("Loads").value ;
  var channelIndex = selectedChannel.selectedIndex;
  loadChannelHistory(channelIndex,channelKeys[channelIndex].channelNumber,channelKeys[channelIndex].key,channelKeys[channelIndex].fieldList,0,maxLoads);
}

// load next 8000 points from a ThingSpeak channel and addPoints to a series
function loadChannelHistory(sentChannelIndex,channelNumber,key,sentFieldList,sentNumLoads,maxLoads) {
   var numLoads=sentNumLoads
   var fieldList= sentFieldList;
   var channelIndex = sentChannelIndex;
   var first_Date = new Date();
   if (typeof fieldList[0].data[0] != "undefined") first_Date.setTime(fieldList[0].data[0][0]+7*60*60*1000);//adjust for 7 hour difference from GMT (Zulu time)
   else if (typeof fieldList[1].data[0] != "undefined") first_Date.setTime(fieldList[1].data[0][0]+7*60*60*1000);
   else if (typeof fieldList[2].data[0] != "undefined") first_Date.setTime(fieldList[2].data[0][0]+7*60*60*1000);
   else if (typeof fieldList[3].data[0] != "undefined") first_Date.setTime(fieldList[3].data[0][0]+7*60*60*1000);
   else if (typeof fieldList[4].data[0] != "undefined") first_Date.setTime(fieldList[4].data[0][0]+7*60*60*1000);
   else if (typeof fieldList[5].data[0] != "undefined") first_Date.setTime(fieldList[5].data[0][0]+7*60*60*1000);
   else if (typeof fieldList[6].data[0] != "undefined") first_Date.setTime(fieldList[6].data[0][0]+7*60*60*1000);
   else if (typeof fieldList[7].data[0] != "undefined") first_Date.setTime(fieldList[7].data[0][0]+7*60*60*1000);
   var end = first_Date.toJSON();
   window.console && console.log('earliest date:',end);
   window.console && console.log('sentChannelIndex:',sentChannelIndex);
   window.console && console.log('numLoads:',numLoads);
   // get the Channel data with a webservice call
$.getJSON('https://www.thingspeak.com/channels/'+channelNumber+'/feed.json?callback=?&amp;offset=0&amp;start=2013-01-20T00:00:00;end='+end+';key='+key, function(data)
   {
   // if no access
   if (data == '-1') {
       $('#chart-container').append('This channel is not public.  To embed charts, the channel must be public or a read key must be specified.');
       window.console && console.log('Thingspeak Data Loading Error');
     }
     for (var fieldIndex=0; fieldIndex<fieldList.length; fieldIndex++)  // iterate through each field
     {
       //fieldList[fieldIndex].data =[];
       for (var h=0; h<data.feeds.length; h++)  // iterate through each feed (data point)
       {
         var p = []//new Highcharts.Point();
         var fieldStr = "data.feeds["+h+"].field"+fieldList[fieldIndex].field;
  var v = eval(fieldStr);
  p[0] = getChartDate(data.feeds[h].created_at);
  p[1] = parseFloat(v);
  // if a numerical value exists add it
    if (!isNaN(parseInt(v))) { fieldList[fieldIndex].data.push(p); }
       }
       fieldList[fieldIndex].data.sort(function(a,b){return a[0]-b[0]});
       dynamicChart.series[fieldList[fieldIndex].series].setData(fieldList[fieldIndex].data,false);
       //dynamicChart.series[fieldList[fieldIndex].series].addPoint(fieldList[fieldIndex].data,false);
       //fieldList[fieldIndex].name = eval("data.channel.field"+fieldList[fieldIndex].field);
       //window.console && console.log('data added to series:',fieldList[fieldIndex].series,fieldList[fieldIndex].data);
   }
     channelKeys[channelIndex].fieldList=fieldList;
     dynamicChart.redraw()
     window.console && console.log('channel index:',channelIndex);
     numLoads++;
     if (numLoads<maxLoads) {loadChannelHistory(channelIndex,channelNumber,key,fieldList,numLoads,maxLoads);}
});
}


And this is the thingspeak.html page:

<!DOCTYPE html>
<html>
  <head>
    <meta content="text/html; charset=UTF-8" http-equiv="content-type">
    <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
    <!--<script src="https://www.thingspeak.com/assets/application-927d1479ab2235e91ae25cb56d33b211.js"
     type="text/javascript"></script>-->
    <script src="http://code.highcharts.com/stock/highstock.js"></script>
    <script src="http://code.highcharts.com/stock/modules/exporting.js"></script>
    <script src="https://rawgithub.com/highslide-software/export-csv/master/export-csv.js"></script>
<script "text/javascript" src="/config/broadcastd.cgi?js=1"></script>
    <script src="thingspeak.js"></script>
    <title>ThingSpeak Multichannel</title>
  </head>
  <body style="background-color: white;">
    <div id="chart-container" style="height: 600px;"><br>
      Please wait..<br>
    </div>
    <div id="below chart"> <button style="width: 89px; margin-top: -18px;" value="Hide All"
        name="Hide All Button"
        onclick="HideAll();">Hide
        All</button><button style="width: 162px; margin-top: -18px;" value="Load More Data"
        name="Load More Data"
        onclick="loadOneChannel();">Load
        More Data </button>
      <select id="Channel Select">
      </select>
      <select id="Loads">
        <option value="1">1 Load</option>
        <option value="2">2 Loads</option>
        <option value="3">3 Loads</option>
        <option value="4">4 Loads</option>
        <option value="5" selected="selected">5 Loads</option>
        <option value="6">6 Loads</option>
        <option value="7">7 Loads</option>
        <option value="8">8 Loads</option>
        <option value="9">9 Loads</option>
        <option value="10">10 Loads</option>
        <option value="15">15 Loads</option>
        <option value="20">20 Loads</option>
        <option value="25">25 Loads</option>
        <option value="30">30 Loads</option>
        <option value="40">40 Loads</option>
        <option value="50">50 Loads</option>
      </select>
      <input id="Update" name="Update" type="checkbox"><span style="font-family: Lucida Grande;">Update
        Chart</span> <span id="Latency" style="font-family: Lucida Grande;">(Latency)</span>
    </div>
  </body>
</html>
Title: Re: Using request.post with different Thermostat [solution]
Post by: Kylix on March 04, 2018, 05:19:42 AM
Practically, all I did was to modify the code in thingspeak.js here:

var channelKeys =[];
channelKeys.push({channelNumber: channel , name:'ESP8266',key: ro_apikey,
                  fieldList:[{field:1,axis:'R'},{field:2,axis:'T'},{field:3,axis:'H'},{field:4,axis:'T'},{field:5,axis:'P'},{field:6,axis:'S'}]});


.......

xAxis: {
  type: 'datetime',
      ordinal:false,
      min: Date.UTC(2013,02,28),
dateTimeLabelFormats : {
        hour: '%H %p',
        minute: '%H:%M %p'
      },
      title: {
        text: 'test'
}
},
yAxis: [{
            title: {
                text: 'Temperature °C'
            },
            id: 'T'
    }, {
            title: {
                text: 'Humidity %'
            },
            opposite: true,
            id: 'H'
    }, {
            title: {
                text: 'Pressure mBar'
            },
            opposite: true,
            id: 'P'
    }, {
            title: {
                text: 'RSSI -dB'
            },
            opposite: true,
            id: 'S'
    }, {
            title: {
                text: 'Relay State'
            },
            opposite: true,
            id: 'R'
    }
],


Those fields must also be added in your channel, on ThingSpeak server.
Title: Re: Using request.post with different Thermostat [solution]
Post by: LukaQ on March 04, 2018, 05:29:05 AM
Thank you, will try it like that
Do you also find, if you want to look back for, say 3-4 days, it takes some time before it loads data. This is why I like what Felix has, load year's data like nothing
Title: Re: Using request.post with different Thermostat [solution]
Post by: Kylix on March 04, 2018, 05:40:38 AM
Fetching data for 1 week doesn't take much and it's done by the ESP8266 module ;) It's not as fast though as Felix's timeseries graph.

This is the main unit:

(https://ibin.co/w800/3tho45zJBXoI.jpg)

And this is the remote node:

(https://ibin.co/w800/3thrQeoJ59ws.jpg)
Title: Re: Using request.post with different Thermostat [solution]
Post by: LukaQ on March 04, 2018, 12:32:34 PM
Quote from: Kylix on March 04, 2018, 05:40:38 AM
it's done by the ESP8266
And I do it with PC, so that not it then. I'll have too look at your code, if there is much difference there, otherwise I don't know why... apart maybe, the amount of data, how much point do you load?
Title: Re: Using request.post with different Thermostat [solution]
Post by: Kylix on March 04, 2018, 12:54:33 PM
I usually load the data for 1 day.