Category Archives: ESP8266

ESPCaptiveImagePortal: A Captive Portal That Displays Images

I hate the internet. I hate it not only because it shows us the decay of society in real-time, but also because I do not really understand it. And who can blame me? Technologically it is one giant bowl of spaghetti, and as an idiosyncratic wannabe iconoclast it is even more difficult for me to comprehend on a social level. So naturally when COVID19 hit Europe like a tonne of bricks and the internet immediately imploded I channeled this hatred hard. Now hold your horses, I am not dishing out this pseudo philosophical claptrap in a failed attempt to be topical. This has a point. Just let me get to it…

A friends birthday was coming up, and I came up with a concept for a gift that attempts to satirise not only the way people use the internet to dish out their unqualified opinion, but also the shallow way in which people deliver that opinion.

The Concept

The concept was to have a device create a WiFi access point called “CORONA VIRUS CURE INFORMATION”, and once a user connects to the access point a captive portal would automatically serve a web page that only displays a GIF telling the user to “go home”. Something a little like this, featuring yours truly.

Obviously this kind of concept could be refined to be a little more usable to the general population who are not as bitter as myself. Maybe a “CAT OF THE DAY” web portal that displays pictures of the household cats? Yes. That sounds much more palatable. Or how about including a configuration page where the user could name the access point whatever they like, and an upload page so custom images could be displayed by the captive portal? I can see it now, looking for my local public WiFi Hotspot and being inundated with “5G KILLS BABIES”, “BILL GATES KILLS BABIES”, and “LOCKDOWN KILLS BABIES”. Yes. This is it. This is what I wanted.

And so the concept for the ESPCaptiveImagePortal was born.

The Specifications

After some musing I came up with the following requirements for ESPCaptiveImagePortal:

  • Implement an WiFi access point with a web server.
  • Implement a captive portal that redirects most traffic to a web page that displays a simple image. This page should be able to display different images each time it is loaded if more than one image is available. It should also be able to display a wide range of image formats, including animated images.
  • Implement a configuration page allowing the user to upload images, delete images, and change the access point SSID. This page should have a hard coded username and password for basic authentication.
  • Implement using the ESP8266.

The ESPCaptiveImagePortal Project

The ESPCaptiveImagePortal project can be found on GitHub here. In addition to the ESP8266 arduino core it requires a few Arduino libraries to compile. This includes ESPAsyncTCP, ESPAsyncWebServer, DNSServer, ESPStringTemplate, and ESPFlash. Most of these libraries will be able to be installed using the Arduino IDE through Library Manager, but if you are having trouble, they can also be installed by downloading the code off github directly and installing them manually.

A Working Example

Here is an example of how the ESPCaptiveImagePortal works using the CAT OF THE DAY example I mentioned a little earlier.

CAT OF THE DAY example

Configuration

Obviously we don’t want just anyone who has access to the access point to be able to upload images, so ESPCaptiveImagePortal relies on three forms of security:

  1. The connected user knows the ESP8266 IP address.
  2. The connected user knows the configuration page URI (/supersecretpage by default)
  3. The connected user knows the hardcoded username and password.

It’s important to note the username and password is implemented using basic HTTP authentication which is incredibly weak, and allows anyone sniffing the network to grab the credentials in plain text. Obviously from a security point of view this is not ideal, but I considered it sufficient for such a project.

To access the configuration page, you have to know the IP address of the ESP8266, which is hardcoded to 192.168.4.1 by default. The URI for the configuration page is also hardcoded to “/supersecretpage” by default. So if those defaults are used, it can be accessed by using any browser with “192.168.4.1/supersecretpage” in the address bar.

The authentication that allows access ends up looking a bit like this. Default values are hardcoded in the project that can be changed very easily.

The below image shows the configuration page. It displays the total number of WiFi connections that have been made to the device as well as the following configuration options:

  • Ability to delete uploaded images
  • Ability to upload new images
  • Ability to change WiFi SSID

The below animation shows a sped up example from start to finish of the ESPCaptiveImagePortal configuration, including getting to the configuration page, uploading an image, changing the SSID, and observing the result all through a browser on a smartphone.

A Note On Image Upload Size and Speed

One last final note is that image upload speed is slow due to the use of internal flash memory (SPIFFS) on the ESP8266. The available space on the filesystem in which images are stored is also very constrained, although this will differ depending on the ESP8266 module that is used. It is highly recommended that the image size does not exceed 500Kb in size. If the image is a non-animated image, an image size of ~100Kb is recommended, which results in a negligible upload and download time while still rendering useful results.

Conclusion

It’s hoped that someone will get a kick out of this project. I had a lot of fun developing it, and learnt a lot about the ESP8266 on the way. I will follow up this post with a technical post that will discuss how I achieved all of the specified features, some thoughts on the ESP8266, and what I would do differently in the future, so watch this space!

ESPFlash: An Arduino Library for Storing Data in the ESP Filesystem

27/12/2022 UPDATE: I have finally managed to fix a few annoying bugs with this library and release a new version. Hopefully it continues to get some use out there!

SPIFFS (or SPI Flash File System) is very cool. It is an Open Source Library intended for SPI NOR flash devices on embedded targets. Embedded targets like the ESP8266 and ESP32, which depending on the model can have upward of 3 megabytes of NOR Flash storage available. The cool thing about SPIFFS is that it makes using this flash storage more intuitive with a simple filesystem type interface where you do not have to think about the various intricacies involved.

This being said, it only reduces the complexity to a certain extent. Anyone who has spent some time using the library would have inevitably lost time with simple problems such as keeping filenames less than 32 characters in length, casting data to confine to the 8 bit nature of saving data to NOR Flash memory, and developing different SPIFFS handling functions for different kinds of arrayed data that you want to save.

It seems like a classic use case where generic programming with templates could reduce the complexity of SPIFFS usage by removing the need to constantly cast data types and provide the size of said data to store it in SPIFFS. After making this observation I was surprised to learn that no Arduino library existed that simplified SPIFFS usage, and thus the idea for ESPFlash was born.

The ESPFlash Library

ESPFlash is an abstraction layer that simplifies the storing of vectorised data in the filesystem on the ESP8266 and ESP32. It was created to make SPIFFS usage simple and easier to understand. It can be found on GitHub here. It can also be found using Arduino’s Library Manager, and available when searched for when using the Arduino IDE. Simple examples also exist to help get people started.

ESPFlash Specification

After some consideration I came up with simple specification that ESPFlash should fulfill. The specification requirements include:

  • Simple template based interface to store and retrieve generic vectorised data in flash memory using SPIFFS.
  • Automatically start SPIFFS if it has not already been started.
  • Automatically truncate filenames that are over 32 characters in length.
  • Keep running count of the number of “elements” stored in a file.
  • Ability to overwrite elements.
  • Ability to append elements.
  • Ability to get single elements.
  • Ability to get a multiple number of elements.
  • Ability to clear elements.
  • Ability to add elements that are stored in flash with PROGMEM.

In addition to this, I wanted to develop two extra modules to further simplify to common use cases. The purpose of these modules are the following:

  • Simple SPIFFS based integer counter
  • Simple SPIFFS based string storage

How ESPFlash Simplifies SPIFFS Usage

Here are a couple of examples of how ESPFlash simplifies SPIFFS usage. It does not show the full functionality of ESPFlash. For the full API, you can checkout the GitHub repository.

Creating a file with a filename of “/intExample” and storing a single integer in the file with error checking.

ESPFlash

int testData = 10;
ESPFlash<int> intExample("/intExample")
bool success = intExample.set(10);

SPIFFS

int testData = 10;
bool success = false; 
uint32_t bytesWritten = 0;
File file;
SPIFFS.begin();
file = SPIFFS.open("/intExample", "w");
if(file)
{
  int* testPointer = &testData;
  bytesWritten = file.write((uint8_t*)testPointer, sizeof(testData));
  if(bytesWritten == sizeof(int))
  {
    success = true;
  }
}
file.close();

Creating a file with a filename of “/charArrayExample” and store 10 chars in the file with error checking.

ESPFlash

char testData[10] = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'};
ESPFlash<char> charExample("/charArrayExample");
bool success = charExample.setElements(testData, sizeof(testData));

SPIFFS

char testData[10] = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'};
bool success = false; 
uint32_t bytesWritten = 0;
File file;
uint32_t elementsSizeInBytes = 0;
SPIFFS.begin();  
file = SPIFFS.open("/charArrayExample", "w");
if(file)
{
  elementsSizeInBytes = sizeof(char)*sizeof(testData);
  bytesWritten = file.write((uint8_t*)testData, elementsSizeInBytes);
  if(bytesWritten == elementsSizeInBytes)
  {
    success = true;
  }
}
file.close();

Open a file with a filename of “/lengthExample” and get the number of elements stored in the file with error checking.

ESPFlash

ESPFlash<double> lengthExample("/lengthExample");
uint32_t numberOfElements = lengthExample.length();

SPIFFS

File file;
uint32_t sizeInBytes = 0;
uint32_t sizeInElements = 0;
SPIFFS.begin();  
file = SPIFFS.open("/lengthExample", "r");
if(file)
{      
  sizeInBytes = file.size();
  sizeInElements = sizeInBytes/sizeof(double);
}
file.close();

Open a file with a filename of “/floatArrayExample” and get the last 5 float elements stored in the file with error checking.

ESPFlash

float testGet[5];
ESPFlash<float> floatExample("/floatArrayExample");
bool success = floatExample.getBackElements(testGet, sizeof(testGet));

SPIFFS

float testGet[5];
File file;
uint32_t fileSizeInBytes = 0;
uint32_t fileSizeInElements = 0;
uint32_t firstElementIndex = 0;
uint32_t bytesRead = 0;
uint32_t getArraySizeInElements = sizeof(testGet);
uint32_t getArraySizeInBytes = sizeof(float)*sizeof(testGet);
bool success = false;
SPIFFS.begin();  
file = SPIFFS.open("/floatArrayExample", "r");
if(file)
{
  fileSizeInBytes = file.size();
  fileSizeInElements = fileSizeInBytes/sizeof(float);
  if(getArraySizeInElements <= fileSizeInElements)
  {
    firstElementIndex = fileSizeInBytes  - getArraySizeInBytes;
    file.seek(firstElementIndex, SeekSet);
    bytesRead = file.read((uint8_t*)testGet, getArraySizeInBytes);
    file.close();
    if(bytesRead == getArraySizeInBytes)
    {
      success = true;
    }
  }
}
file.close();

Truncate a filename in excess of 32 characters so it can be used with SPIFFS.

ESPFlash

/* Filenames are automatically truncated if in excess of 32 characters */
/* The file extension is preserved */
ESPFlash<float> floatExample("/thisFilenameIsLargerThan32Characters.txt");

SPIFFS

const char* filename = "/thisFilenameIsLargerThan32Characters.txt";
char filenameBuffer[32];
if(strlen(fileName) < 32)
{
  strcpy(filenameBuffer, fileName);
}
else
{
  strncpy(this->fileName, fileName, 27);
  char* pch = strrchr(fileName, '.');
  strcpy(filenameBuffer+27, pch);
}

Conclusion

ESPFlash is far simpler than using the SPIFFS library directly. It also reduces code footprint, and makes SPIFFS error detection easy. It is hoped that this library will be useful to someone down the track!