ESPFlash: An Arduino Library for Storing Data in the ESP Filesystem

SPIFFS (or SPI Flash File System) is very cool. It is an Open Source Library intended for SPI NOR flash devices on embedded targets. Embedded targets like the ESP8266 and ESP32, which depending on the model can have upward of 3 megabytes of NOR Flash storage available. The cool thing about SPIFFS is that it makes using this flash storage more intuitive with a simple filesystem type interface where you do not have to think about the various intricacies involved.

This being said, it only reduces the complexity to a certain extent. Anyone who has spent some time using the library would have inevitably lost time with simple problems such as keeping filenames less than 32 characters in length, casting data to confine to the 8 bit nature of saving data to NOR Flash memory, and developing different SPIFFS handling functions for different kinds of arrayed data that you want to save.

It seems like a classic use case where generic programming with templates could reduce the complexity of SPIFFS usage by removing the need to constantly cast data types and provide the size of said data to store it in SPIFFS. After making this observation I was surprised to learn that no Arduino library existed that simplified SPIFFS usage, and thus the idea for ESPFlash was born.

The ESPFlash Library

ESPFlash is an abstraction layer that simplifies the storing of vectorised data in the filesystem on the ESP8266 and ESP32. It was created to make SPIFFS usage simple and easier to understand. It can be found on GitHub here. It can also be found using Arduino’s Library Manager, and available when searched for when using the Arduino IDE. Simple examples also exist to help get people started.

ESPFlash Specification

After some consideration I came up with simple specification that ESPFlash should fulfill. The specification requirements include:

  • Simple template based interface to store and retrieve generic vectorised data in flash memory using SPIFFS.
  • Automatically start SPIFFS if it has not already been started.
  • Automatically truncate filenames that are over 32 characters in length.
  • Keep running count of the number of “elements” stored in a file.
  • Ability to overwrite elements.
  • Ability to append elements.
  • Ability to get single elements.
  • Ability to get a multiple number of elements.
  • Ability to clear elements.
  • Ability to add elements that are stored in flash with PROGMEM.

In addition to this, I wanted to develop two extra modules to further simplify to common use cases. The purpose of these modules are the following:

  • Simple SPIFFS based integer counter
  • Simple SPIFFS based string storage

How ESPFlash Simplifies SPIFFS Usage

Here are a couple of examples of how ESPFlash simplifies SPIFFS usage. It does not show the full functionality of ESPFlash. For the full API, you can checkout the GitHub repository.

Creating a file with a filename of “/intExample” and storing a single integer in the file with error checking.

ESPFlash

int testData = 10;
ESPFlash<int> intExample("/intExample")
bool success = intExample.set(10);

SPIFFS

int testData = 10;
bool success = false; 
uint32_t bytesWritten = 0;
File file;
SPIFFS.begin();
file = SPIFFS.open("/intExample", "w");
if(file)
{
  int* testPointer = &testData;
  bytesWritten = file.write((uint8_t*)testPointer, sizeof(testData));
  if(bytesWritten == sizeof(int))
  {
    success = true;
  }
}
file.close();

Creating a file with a filename of “/charArrayExample” and store 10 chars in the file with error checking.

ESPFlash

char testData[10] = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'};
ESPFlash<char> charExample("/charArrayExample");
bool success = charExample.setElements(testData, sizeof(testData));

SPIFFS

char testData[10] = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'};
bool success = false; 
uint32_t bytesWritten = 0;
File file;
uint32_t elementsSizeInBytes = 0;
SPIFFS.begin();  
file = SPIFFS.open("/charArrayExample", "w");
if(file)
{
  elementsSizeInBytes = sizeof(char)*sizeof(testData);
  bytesWritten = file.write((uint8_t*)testData, elementsSizeInBytes);
  if(bytesWritten == elementsSizeInBytes)
  {
    success = true;
  }
}
file.close();

Open a file with a filename of “/lengthExample” and get the number of elements stored in the file with error checking.

ESPFlash

ESPFlash<double> lengthExample("/lengthExample");
uint32_t numberOfElements = lengthExample.length();

SPIFFS

File file;
uint32_t sizeInBytes = 0;
uint32_t sizeInElements = 0;
SPIFFS.begin();  
file = SPIFFS.open("/lengthExample", "r");
if(file)
{      
  sizeInBytes = file.size();
  sizeInElements = sizeInBytes/sizeof(double);
}
file.close();

Open a file with a filename of “/floatArrayExample” and get the last 5 float elements stored in the file with error checking.

ESPFlash

float testGet[5];
ESPFlash<float> floatExample("/floatArrayExample");
bool success = floatExample.getBackElements(testGet, sizeof(testGet));

SPIFFS

float testGet[5];
File file;
uint32_t fileSizeInBytes = 0;
uint32_t fileSizeInElements = 0;
uint32_t firstElementIndex = 0;
uint32_t bytesRead = 0;
uint32_t getArraySizeInElements = sizeof(testGet);
uint32_t getArraySizeInBytes = sizeof(float)*sizeof(testGet);
bool success = false;
SPIFFS.begin();  
file = SPIFFS.open("/floatArrayExample", "r");
if(file)
{
  fileSizeInBytes = file.size();
  fileSizeInElements = fileSizeInBytes/sizeof(float);
  if(getArraySizeInElements <= fileSizeInElements)
  {
    firstElementIndex = fileSizeInBytes  - getArraySizeInBytes;
    file.seek(firstElementIndex, SeekSet);
    bytesRead = file.read((uint8_t*)testGet, getArraySizeInBytes);
    file.close();
    if(bytesRead == getArraySizeInBytes)
    {
      success = true;
    }
  }
}
file.close();

Truncate a filename in excess of 32 characters so it can be used with SPIFFS.

ESPFlash

/* Filenames are automatically truncated if in excess of 32 characters */
/* The file extension is preserved */
ESPFlash<float> floatExample("/thisFilenameIsLargerThan32Characters.txt");

SPIFFS

const char* filename = "/thisFilenameIsLargerThan32Characters.txt";
char filenameBuffer[32];
if(strlen(fileName) < 32)
{
  strcpy(filenameBuffer, fileName);
}
else
{
  strncpy(this->fileName, fileName, 27);
  char* pch = strrchr(fileName, '.');
  strcpy(filenameBuffer+27, pch);
}

Conclusion

ESPFlash is far simpler than using the SPIFFS library directly. It also reduces code footprint, and makes SPIFFS error detection easy. It is hoped that this library will be useful to someone down the track!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.