27/12/2022 UPDATE: I have finally managed to fix a few annoying bugs with this library and release a new version. Hopefully it continues to get some use out there!
SPIFFS (or SPI Flash File System) is very cool. It is an Open Source Library intended for SPI NOR flash devices on embedded targets. Embedded targets like the ESP8266 and ESP32, which depending on the model can have upward of 3 megabytes of NOR Flash storage available. The cool thing about SPIFFS is that it makes using this flash storage more intuitive with a simple filesystem type interface where you do not have to think about the various intricacies involved.
This being said, it only reduces the complexity to a certain extent. Anyone who has spent some time using the library would have inevitably lost time with simple problems such as keeping filenames less than 32 characters in length, casting data to confine to the 8 bit nature of saving data to NOR Flash memory, and developing different SPIFFS handling functions for different kinds of arrayed data that you want to save.
It seems like a classic use case where generic programming with templates could reduce the complexity of SPIFFS usage by removing the need to constantly cast data types and provide the size of said data to store it in SPIFFS. After making this observation I was surprised to learn that no Arduino library existed that simplified SPIFFS usage, and thus the idea for ESPFlash was born.
The ESPFlash Library
ESPFlash is an abstraction layer that simplifies the storing of vectorised data in the filesystem on the ESP8266 and ESP32. It was created to make SPIFFS usage simple and easier to understand. It can be found on GitHub here. It can also be found using Arduino’s Library Manager, and available when searched for when using the Arduino IDE. Simple examples also exist to help get people started.
ESPFlash Specification
After some consideration I came up with simple specification that ESPFlash should fulfill. The specification requirements include:
- Simple template based interface to store and retrieve generic vectorised data in flash memory using SPIFFS.
- Automatically start SPIFFS if it has not already been started.
- Automatically truncate filenames that are over 32 characters in length.
- Keep running count of the number of “elements” stored in a file.
- Ability to overwrite elements.
- Ability to append elements.
- Ability to get single elements.
- Ability to get a multiple number of elements.
- Ability to clear elements.
- Ability to add elements that are stored in flash with PROGMEM.
In addition to this, I wanted to develop two extra modules to further simplify to common use cases. The purpose of these modules are the following:
- Simple SPIFFS based integer counter
- Simple SPIFFS based string storage
How ESPFlash Simplifies SPIFFS Usage
Here are a couple of examples of how ESPFlash simplifies SPIFFS usage. It does not show the full functionality of ESPFlash. For the full API, you can checkout the GitHub repository.
Creating a file with a filename of “/intExample” and storing a single integer in the file with error checking.
ESPFlash
int testData = 10;
ESPFlash<int> intExample("/intExample")
bool success = intExample.set(10);
SPIFFS
int testData = 10;
bool success = false;
uint32_t bytesWritten = 0;
File file;
SPIFFS.begin();
file = SPIFFS.open("/intExample", "w");
if(file)
{
int* testPointer = &testData;
bytesWritten = file.write((uint8_t*)testPointer, sizeof(testData));
if(bytesWritten == sizeof(int))
{
success = true;
}
}
file.close();
Creating a file with a filename of “/charArrayExample” and store 10 chars in the file with error checking.
ESPFlash
char testData[10] = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'};
ESPFlash<char> charExample("/charArrayExample");
bool success = charExample.setElements(testData, sizeof(testData));
SPIFFS
char testData[10] = {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'};
bool success = false;
uint32_t bytesWritten = 0;
File file;
uint32_t elementsSizeInBytes = 0;
SPIFFS.begin();
file = SPIFFS.open("/charArrayExample", "w");
if(file)
{
elementsSizeInBytes = sizeof(char)*sizeof(testData);
bytesWritten = file.write((uint8_t*)testData, elementsSizeInBytes);
if(bytesWritten == elementsSizeInBytes)
{
success = true;
}
}
file.close();
Open a file with a filename of “/lengthExample” and get the number of elements stored in the file with error checking.
ESPFlash
ESPFlash<double> lengthExample("/lengthExample");
uint32_t numberOfElements = lengthExample.length();
SPIFFS
File file;
uint32_t sizeInBytes = 0;
uint32_t sizeInElements = 0;
SPIFFS.begin();
file = SPIFFS.open("/lengthExample", "r");
if(file)
{
sizeInBytes = file.size();
sizeInElements = sizeInBytes/sizeof(double);
}
file.close();
Open a file with a filename of “/floatArrayExample” and get the last 5 float elements stored in the file with error checking.
ESPFlash
float testGet[5];
ESPFlash<float> floatExample("/floatArrayExample");
bool success = floatExample.getBackElements(testGet, sizeof(testGet));
SPIFFS
float testGet[5];
File file;
uint32_t fileSizeInBytes = 0;
uint32_t fileSizeInElements = 0;
uint32_t firstElementIndex = 0;
uint32_t bytesRead = 0;
uint32_t getArraySizeInElements = sizeof(testGet);
uint32_t getArraySizeInBytes = sizeof(float)*sizeof(testGet);
bool success = false;
SPIFFS.begin();
file = SPIFFS.open("/floatArrayExample", "r");
if(file)
{
fileSizeInBytes = file.size();
fileSizeInElements = fileSizeInBytes/sizeof(float);
if(getArraySizeInElements <= fileSizeInElements)
{
firstElementIndex = fileSizeInBytes - getArraySizeInBytes;
file.seek(firstElementIndex, SeekSet);
bytesRead = file.read((uint8_t*)testGet, getArraySizeInBytes);
file.close();
if(bytesRead == getArraySizeInBytes)
{
success = true;
}
}
}
file.close();
Truncate a filename in excess of 32 characters so it can be used with SPIFFS.
ESPFlash
/* Filenames are automatically truncated if in excess of 32 characters */
/* The file extension is preserved */
ESPFlash<float> floatExample("/thisFilenameIsLargerThan32Characters.txt");
SPIFFS
const char* filename = "/thisFilenameIsLargerThan32Characters.txt";
char filenameBuffer[32];
if(strlen(fileName) < 32)
{
strcpy(filenameBuffer, fileName);
}
else
{
strncpy(this->fileName, fileName, 27);
char* pch = strrchr(fileName, '.');
strcpy(filenameBuffer+27, pch);
}
Conclusion
ESPFlash is far simpler than using the SPIFFS library directly. It also reduces code footprint, and makes SPIFFS error detection easy. It is hoped that this library will be useful to someone down the track!