Client-server communication using ESP8266

As I wrote in my previous post, I’m retrieving the information from my electricity meter and I’m sending it to my server to process the data. In this post, I will show how I deal with this simple client-server communication.

If you are insterested in Dallas DS18B20 communication, take a look at my newer post here: ESP8266 + DS18B20 + web server

WiFi connectivity

The ESP8266 is mainly the WiFi module. Let’s make a use of it and connect our thing to the local WiFi on the premise. I want to make it as simple as possible, so I will hardcode the WiFi parameters in the script. This means that each change to the WiFi configuration will need to be adjusted by re-programming the module. I’m good with this. (Update on 2021-02-17: check my new post on WiFi Manager for easy network management)

At the beginning of my script, I added these configuration variables:

[...]
const char* ssid = "your-WiFi-SSID"; // The SSID (name) of the Wi-Fi network you want to connect to
const char* password = "your-WiFi-password"; // The password of the Wi-Fi network
[...]

I also want to have a way to debug my board on the fly, so I added serial port communication configuration along with the WiFi configuration in my setup function:

[...]
void setup() {
Serial.begin(115200); // Start the Serial communication to send messages to the computer
delay(10);
Serial.println('\n');

WiFi.hostname("ESPboard-counter"); // Hostname of the board
WiFi.begin(ssid, password); // Connect to the network
Serial.print("Connecting to ");
Serial.print(ssid); Serial.println(" ...");

int i = 0;
while (WiFi.status() != WL_CONNECTED) { // Wait for the Wi-Fi to connect
delay(1000);
Serial.print(++i); Serial.print(' ');
}

Serial.println('\n');
Serial.println("Connection established!");
Serial.print("IP address:\t");
Serial.println(WiFi.localIP()); // Send the IP address of the ESP8266 to the computer
[...]

The script above is setting the hostname of the board since I want to see it under the proper name on my router. It is also waiting for the WiFi connection. Every second there is a number of seconds spent on connecting sent through the serial port. This way I can see if the connection is established or not and how long it took. Also, if there is no WiFi connection, the script will stay in the setup function.

Sending results

Once we are connected to the WiFi, we can send our counters to the server which will handle them. I want this to be performed once every minute. In the main loop of the board, I implemented a basic if statement to check if the data should be already sent or not. Because the number of milliseconds since board start gets reset from time to time, I also have to check if this has taken place.

[...]
unsigned long current_time = millis();
// send data every 60 seconds
if (current_time - last_sent_time > 60000) {
postData();
last_sent_time = millis();
}
if (current_time < last_sent_time) {
// millis reset
current_time = last_sent_time;
}
[...]

There are few configuration variables placed at the beginning of the script:

[...]
const char* getHost = "host.name.of.receiver.com"; // The data receiver host name (not URL)
const int httpGetPort = 80; // The data receiver host port
String getReceiverURL = "/receiveTheData.php"; // The data receiver script
[...]

Host and receiver script URL are separated because the connection I will perform in the function requires such approach. The function to post the data looks like this:

[...]
void postData() {
WiFiClient clientGet;
String src = "ESP";
int vint = interruptCounter;
String getReceiverURLtemp = getReceiverURL + "?src=" + src + "&int=" + vint;

Serial.println("-------------------------------");
Serial.print(">>> Connecting to host: ");
Serial.println(getHost);

if (!clientGet.connect(getHost, httpGetPort)) {
Serial.print("Connection failed: ");
Serial.print(getHost);
} else {
clientGet.println("GET " + getReceiverURLtemp + " HTTP/1.1");
clientGet.print("Host: ");
clientGet.println(getHost);
clientGet.println("User-Agent: ESP8266/1.0");
clientGet.println("Connection: close\r\n\r\n");

unsigned long timeoutP = millis();
while (clientGet.available() == 0) {
if (millis() - timeoutP > 10000) {
Serial.print(">>> Client Timeout: ");
Serial.println(getHost);
clientGet.stop();
return;
}
}

//just checks the 1st line of the server response. Could be expanded if needed.
while(clientGet.available()){
String retLine = clientGet.readStringUntil('\r');
Serial.print(">>> Host returned: ");
Serial.println(retLine);

// reset counter if successully connected
if (retLine == "HTTP/1.1 200 OK") {
Serial.println(">>> Communication successful");
interruptCounter -= vint;
Serial.print(">>> Changed interrupt counter to: ");
Serial.println(interruptCounter);
} else {
Serial.println(">>> Communication failed!!!");
}
break;
}

} //end client connection if else

Serial.print(">>> Closing host: ");
Serial.println(getHost);

clientGet.stop();
}
[...]

As you can see, I’m checking if the communication was successful and I’m decreasing the counter using the amount sent to the server. I do this because in the meantime the counter could have been updated and I don’t want to change it to zero – I want to remove only values sent. If the communication failed, I will send counter values during the next communication session.

The above approach has one big disadvantage – if there is a longer communication problem between the client and the server, all impulses counted will be sent at once, giving the big spike. I want to change this later, to remember counter values for particular time frames, but not in this first version of the script.

Informational web page

From time to time I want to check if my board is working. I don’t want to connect my computer via serial port each time, so I created a simple HTML page to report the status of the board. The page is served by the board itself. It requires the server to be initiated at the beginning of the script:

[...]
WiFiServer server(80); // set WiFiServer on port 80
[...]

A few additional lines to the setup function:

[...]
// start HTTP server
server.begin();
Serial.printf("Web server started, open %s in a web browser\n", WiFi.localIP().toString().c_str());
[...]

And to the main loop of the script:

[...]
// wait for a client (web browser) to connect
WiFiClient client = server.available();
if (client)
{
handle_request(client);
}
[...]

Finally, the handler functions:

[...]
// prepare a web page to be send to a client (web browser)
String prepareHtmlPage()
{
String htmlPage =
String("HTTP/1.1 200 OK\r\n") +
"Content-Type: text/html\r\n" +
"Connection: close\r\n" + // the connection will be closed after completion of the response
"Refresh: 30\r\n" + // refresh the page automatically every 5 sec
"\r\n" +
"<!DOCTYPE HTML>" +
"<html><body>" +
"<p>Interrupt counter: " + String(interruptCounter) + "</p>" +
"<p>Last sent time: " + String(last_sent_time) + "</p>" +
"<p>Current time: " + String(millis()) + "</p>" +
"</body></html>" +
"\r\n";
return htmlPage;
}

// handle HTTP request to this board
void handle_request(WiFiClient client) {
Serial.println("\n[Client connected]");
while (client.connected())
{
// read line by line what the client (web browser) is requesting
if (client.available())
{
String line = client.readStringUntil('\r');
Serial.print(line);
// wait for end of client's request, that is marked with an empty line
if (line.length() == 1 && line[0] == '\n')
{
client.println(prepareHtmlPage());
break;
}
}
}
delay(100); // give the web browser time to receive the data

// close the connection:
client.stop();
Serial.println("[Client disonnected]");
}
[...]

This is the basic page, with no interaction between me and the board. I can only check values. Of course, you can adjust it to show more data or to provide more possibilities, but I want to keep it as simple as possible for now.

Conclusion

My board is working continuously for some time now and I see no issues. As I wrote above, I plan to add the buffer to store counters separately for past minutes in case of communication issues. I will also adjust my reporting page to show these stored values. But this has to wait for the better moment.

Below you can review the whole code of the script uploaded to the board.

#include <ESP8266WiFi.h>

const char* ssid = "your-WiFi-SSID"; // The SSID (name) of the Wi-Fi network you want to connect to
const char* password = "your-WiFi-password"; // The password of the Wi-Fi network

const char* getHost = "host.name.of.receiver.com"; // The data receiver host name (not URL)
const int httpGetPort = 80; // The data receiver host port
String getReceiverURL = "/receiveTheData.php"; // The data receiver script

const byte interruptPin = 5; // Pin to set interrupt to
int interruptCounter = 0; // counter of interrupt executions

unsigned long last_sent_time = millis(); // time last counter sent to the receiver host

WiFiServer server(80); // set WiFiServer on port 80

void setup() {
Serial.begin(115200); // Start the Serial communication to send messages to the computer
delay(10);
Serial.println('\n');

WiFi.hostname("ESPboard-counter");
WiFi.begin(ssid, password); // Connect to the network
Serial.print("Connecting to ");
Serial.print(ssid); Serial.println(" ...");

int i = 0;
while (WiFi.status() != WL_CONNECTED) { // Wait for the Wi-Fi to connect
delay(1000);
Serial.print(++i); Serial.print(' ');
}

Serial.println('\n');
Serial.println("Connection established!");
Serial.print("IP address:\t");
Serial.println(WiFi.localIP()); // Send the IP address of the ESP8266 to the computer

// set interrupt handler
pinMode(interruptPin, INPUT_PULLUP);
attachInterrupt(digitalPinToInterrupt(interruptPin), handleInterrupt, FALLING);

// start HTTP server
server.begin();
Serial.printf("Web server started, open %s in a web browser\n", WiFi.localIP().toString().c_str());
}

// this one is handling GPIO interrupts
void handleInterrupt() {
static unsigned long last_interrupt_time = 0;
unsigned long interrupt_time = millis();

// If interrupts come faster than 100ms, assume it's a bounce and ignore
if (interrupt_time - last_interrupt_time > 100)
{
interruptCounter++;
Serial.print("<<< Interrupt counter value:");
Serial.println(interruptCounter);
}
last_interrupt_time = interrupt_time;
}

// send data to the receiver host
void postData() {
WiFiClient clientGet;
// We now create and add parameters:
String src = "ESP";
int vint = interruptCounter;
String getReceiverURLtemp = getReceiverURL + "?src=" + src + "&int=" + vint;

Serial.println("-------------------------------");
Serial.print(">>> Connecting to host: ");
Serial.println(getHost);

if (!clientGet.connect(getHost, httpGetPort)) {
Serial.print("Connection failed: ");
Serial.print(getHost);
} else {
clientGet.println("GET " + getReceiverURLtemp + " HTTP/1.1");
clientGet.print("Host: ");
clientGet.println(getHost);
clientGet.println("User-Agent: ESP8266/1.0");
clientGet.println("Connection: close\r\n\r\n");

unsigned long timeoutP = millis();
while (clientGet.available() == 0) {
if (millis() - timeoutP > 10000) {
Serial.print(">>> Client Timeout: ");
Serial.println(getHost);
clientGet.stop();
return;
}
}

//just checks the 1st line of the server response. Could be expanded if needed.
while(clientGet.available()){
String retLine = clientGet.readStringUntil('\r');
Serial.print(">>> Host returned: ");
Serial.println(retLine);

// reset counter if successully connected
if (retLine == "HTTP/1.1 200 OK") {
Serial.println(">>> Communication successful");
interruptCounter -= vint;
Serial.print(">>> Changed interrupt counter to: ");
Serial.println(interruptCounter);
} else {
Serial.println(">>> Communication failed!!!");
}
break;
}

} //end client connection if else

Serial.print(">>> Closing host: ");
Serial.println(getHost);

clientGet.stop();
}

// prepare a web page to be send to a client (web browser)
String prepareHtmlPage()
{
String htmlPage =
String("HTTP/1.1 200 OK\r\n") +
"Content-Type: text/html\r\n" +
"Connection: close\r\n" + // the connection will be closed after completion of the response
"Refresh: 30\r\n" + // refresh the page automatically every 5 sec
"\r\n" +
"<!DOCTYPE HTML>" +
"<html><body>" +
"<p>Interrupt counter: " + String(interruptCounter) + "</p>" +
"<p>Last sent time: " + String(last_sent_time) + "</p>" +
"<p>Current time: " + String(millis()) + "</p>" +
"</body></html>" +
"\r\n";
return htmlPage;
}

// handle HTTP request to this board
void handle_request(WiFiClient client) {
Serial.println("\n[Client connected]");
while (client.connected())
{
// read line by line what the client (web browser) is requesting
if (client.available())
{
String line = client.readStringUntil('\r');
Serial.print(line);
// wait for end of client's request, that is marked with an empty line
if (line.length() == 1 && line[0] == '\n')
{
client.println(prepareHtmlPage());
break;
}
}
}
delay(100); // give the web browser time to receive the data

// close the connection:
client.stop();
Serial.println("[Client disonnected]");
}

// main loop of the board
void loop() {
unsigned long current_time = millis();
// send data every 60 seconds
if (current_time - last_sent_time > 60000) {
postData();
last_sent_time = millis();
}
if (current_time < last_sent_time) {
// millis reset
current_time = last_sent_time;
}

// wait for a client (web browser) to connect
WiFiClient client = server.available();
if (client)
{
handle_request(client);
}

}