LittleFS and FatFS on ESP32, feedback needed

[lorol]

Updates:

• Combined plugin for SPIFFS, LittleFS and FatFS: LittleFS and FatFS on ESP32, feedback needed #792 (comment)

• For FAT partition, see LittleFS and FatFS on ESP32, feedback needed #792 (comment)

---

Hi I made a LittleFS for ESP32, too. see the example my library and tool plus the core conversation.
Please test and comment. (performance, speed, stability, code size, slashes, directories and root / as well as other compatibility concerns)

Example (modified from the master ESP_AsyncFSBrowser)
Note I did tweaks of Editor as well to handle folders and slashes
https://github.com/lorol/ESPAsyncWebServer/tree/master/examples/ESP_AsyncFSBrowser

Library:
https://github.com/lorol/LITTLEFS

Tool:
https://github.com/lorol/arduino-esp32littlefs-plugin

Core discussion:
espressif/arduino-esp32#3765

Simple usage:
#include <FS.h>
#define SPIFFS LITTLEFS
#include <LITTLEFS.h>

[BlueAndi]

Could you add a library.json too for platformio?

[lorol]

@BlueAndi - Done, just added an improvised one. I have no idea if it makes sense. Please check and let me know.

[BlueAndi]

A first try showed:

The "esp_littlefs.h" includes the "sys/termios.h" which contains baudrate preprocessor defines that clashes with the binary number preprocessor defines of "binary.h" (included in Arduino.h).

I see you need at least IDF v3.3 based stuff, because e.g. the esp_vfs_t doesn't match at my side currently. It looks like I still get IDF v3.2 via platformio. :-(

Edit:
Yes, v3.2 is used by v1.0.4 (https://github.com/espressif/arduino-esp32/releases)

[lorol]

The IDF Arduino is compiled on is this v.3.3: git checkout 46b12a560a29fa6ade07800a4abe12a026183988 The original wrapper https://github.com/joltwallet/esp_littlefs was modified not long ago so it likely is tested on even newer IDF. If PIO uses older IDF, you need to resolve it yourself. My target was to work on Arduino IDE. For me, PIO looks like a F1 car we try to drive with your home car license :)

…

On Wed, Jun 17, 2020 at 3:02 PM Andreas Merkle ***@***.***> wrote: A first try showed: The "esp_littlefs.h" includes the "sys/termios.h" which contains baudrate preprocessor defines that clashes with the binary number preprocessor defines of "binary.h" (included in Arduino.h). I see you need at least IDF v3.3 based stuff, because e.g. the esp_vfs_t doesn't match at my side currently. It looks like I still get IDF v3.2 via platformio. :-( — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#792 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABATPIC566GHUB24N3IQTE3RXEHKRANCNFSM4OAZ37NQ> .

[lorol]

@BlueAndi - I got what you say ...
Well I used the git version of the core, I need to note it somewhere.

Less-likely I will go back to fix. Sorry.

[BlueAndi]

With PIO I get more possibilities, as you mention as F1 car and I use them. ;-)
I will try it again, if I get IDF 3.3. Anyway thanks for your effort.

[lorol]

@BlueAndi
Try now, but first in esp_littlefs.c un-comment the line:
#define CONFIG_LITTLEFS_FOR_IDF_3_2
At least it works now with arduino release core 1.0.4.
Let me know if PIO likes it this way :)

[BlueAndi]

Ok, thanks. I will try and report back. Because I am very busy, I may need some more days.

[lorol]

I updated LITTLEFS in GitHub to be with #define CONFIG_LITTLEFS_FOR_IDF_3_2 by default, which allows build on several IDF versions, including the release 1.0.4 and POI on Arduino platform - it works.

[zekageri]

It will be really good if i would be able to upload little fs just like spiffs on platformio vscode ( upload file system image ).

I'am really looking for that littlefs because the spiffs can't handle the 16mb partition on my esp32. I will try it out definietly!

[lorol]

@zekageri
Just try for fun:

• close vscode
• In your PIO installation, find a file, named "mkspiffs_espressif32_arduino" [.exe] :)
• Go to: https://github.com/earlephilhower/mklittlefs
• Download the mklittlefs[.exe] executable for your OS from releases there
• rename your original mkspiffs_espressif32_arduino[.exe] to sometting else like mkspiffs_espressif32_arduino_bak[.exe]
• put the mklittlefs[.exe] in same folder where mkspiffs_espressif32_arduino_bak[.exe] is
• rename mklittlefs[.exe] to mkspiffs_espressif32_arduino[.exe]
• Re-open your vscode project
• have data folder prepared
• Upload File System Image as usual
• Enjoy
• Don't forget to restore against SPIFFS
• Appreciate if you find the right people to implement it properly in PIO (different command, choice, prompt, py script, config through platformio.ini per project ... etc.)

As last-resort, you can keep both executable files there and write a simple shell or batch script to automate the "swap" hack above.

[zekageri]

i'am trying it right now. I have an error currently wich is :

src/Includes.h:32:16: error: conflicting declaration 'fs::SPIFFSFS LITTLEFS'
 #define SPIFFS LITTLEFS

It looks like LITTLEFS defined somewhere already.

EDIT: if i modify it to #define SPIFFS LITTLEFSFS ( because in LITTLEFS.cpp there is this: LITTLEFSFS LITTLEFS; ) i get an other error wich says :

.pio\build\esp32dev\src\main.cpp.o:(.literal._ZNSt17_Function_handlerIFvP21AsyncWebServerRequestEZL13HTTP_RequestsvEUlS1_E23_E9_M_invokeERKSt9_Any_dataOS1_+0x0): undefined reference to LITTLEFSFS'`

EDIT2 :

Okay nevermind, i renamed all my SPIFFS. to LITTLEFS.
The Code compiles now. ^^

EDIT3 :

I managed to upload the littlefs on PIO with the trick you suggested. No problem, the esp booted up and running with the new file system. After that i forget that i need a couple of more things so i grapped these, copyed to the data folder, and uploaded it again. Now it says that:

[E][vfs_api.cpp:22] open(): File system is not mounted
Guru Meditation Error: Core  1 panic'ed (IntegerDivideByZero). Exception was unhandled.

Even if i put this to the setup:

if(!LITTLEFS.begin(FORMAT_LITTLEFS_IF_FAILED)){
        Serial.println("LITTLEFS Mount Failed");
        return;
    }

And even with fresh upload with the plugin from PIO.

EDIT4:

I belive it is because the JPEGDecoder library. It is using the SPIFFS. I will try it out!

JPEGDecoder by Bodmer

EDIT5: Almost everything is working. The first problem was that this lib used the SPIFFS.h. I changed this too to LITTLEFS and it is okay now. But if some file is not present, ( or not existent ) on the file system, i get an exception and a reboot with backtrace:

E (2424) esp_littlefs: Failed to opendir "/Use
[37mrProg.json". Error  (-2)
fs_api.cpp:64] open(): /littlefs/UserProg.json does not existexist      
E (38744) esp_littlefs: Failed to open
7mir "/NotFound.html". Error  (-2)
E (38751) esp_littlefs: Failed to opendir "/Not
Found.html.gz". Error  (-2)
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhan
dled.
Core 1 register dump:
PC      : 0x400d4cd4  PS      : 0x00060830  A0      : 0x800e3768  A1      : 0x3ffd9cb0
A2      : 0x00000000  A3      : 0x3ffc6a08  A4      : 0x3ffdd194  A5      : 0x3ffc6a08
A6      : 0x0000000d  A7      : 0x00000000  A8      : 0x800d4cd1  A9      : 0x3ffd9c90
A10     : 0x3ffd9cc0  A11     : 0x3f4017b4  A12     : 0xbdc8e301  A13     : 0x3ffc6a08
A14     : 0x00000000  A15     : 0x3ffd9ccc  SAR     : 0x00000004  EXCCAUSE: 0
x0000001c
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x                    4
000150d  LCOUNT  : 0xfffffffb  

Backtrace: 0x400d4cd4:0x3ffd9cb0 0x400e3765:0x3ffd9d00 0x400e136d:0x3ffd9d40 0x400e1431:0x3ffd9d80 0x400e1699:0x3ffd9dc0 0x4015408d:0x3ffd9de0 0x40154109:0x3ffd9e20 0x401547b2:0x3ffd9e40 0x400893c1:0x3ffd9e70
  #0  0x400d4cd4:0x3ffd9cb0 in std::_Function_handler<void (AsyncWebServerRequest*), HTTP_Requests()::{lambda(AsyncWebServerRequest*)#1}>::_M_invoke(std::_Any_data 
const&, AsyncWebServerRequest*&&) at src/Setups.h:181
      (inlined by) _M_invoke at c:\users\dr.random\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf\include\c++\5.2.0/functional:1871
  #1  0x400e3765:0x3ffd9d00 in std::function<void (AsyncWebServerRequest*)>::operator()(AsyncWebServerRequest*) const at lib\ESPAsyncWebServer\src/StringArray.h:73 
      (inlined by) AsyncCallbackWebHandler::handleRequest(AsyncWebServerRequest*) 
at lib\ESPAsyncWebServer\src/WebHandlerImpl.h:123
  #2  0x400e136d:0x3ffd9d40 in AsyncWebServerRequest::_parseLine() at lib\ESPAsyncWebServer\src/WebRequest.cpp:797
  #3  0x400e1431:0x3ffd9d80 in AsyncWebServerRequest::_onData(void*, unsigned int) at lib\ESPAsyncWebServer\src/WebRequest.cpp:797
  #4  0x400e1699:0x3ffd9dc0 in std::_Function_handler<void (void*, AsyncClient*, void*, unsigned int), AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*)::{lambda(void*, AsyncClient*, void*, unsigned int)#8}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&, std::_Any_data const&, unsigned int&&) at lib\ESPAsyncWebServer\src/WebRequest.cpp:797
      (inlined by) _M_invoke at c:\users\dr.random\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf\include\c++\5.2.0/functional:1871
  #5  0x4015408d:0x3ffd9de0 in std::function<void (void*, AsyncClient*, void*, unsigned int)>::operator()(void*, AsyncClient*, void*, unsigned int) const at lib\AsyncTCP\src/AsyncTCP.cpp:1039
      (inlined by) AsyncClient::_recv(tcp_pcb*, pbuf*, signed char) at lib\AsyncTCP\src/AsyncTCP.cpp:934
  #6  0x40154109:0x3ffd9e20 in AsyncClient::_s_recv(void*, tcp_pcb*, pbuf*, signed char) at lib\AsyncTCP\src/AsyncTCP.cpp:1039
  #7  0x401547b2:0x3ffd9e40 in _async_service_task(void*) at lib\AsyncTCP\src/AsyncTCP.cpp:1039
      (inlined by) _async_service_task at lib\AsyncTCP\src/AsyncTCP.cpp:197       
  #8  0x400893c1:0x3ffd9e70 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

I will decode it in a min.

Nah, can't debug now. Looks like PIO monitor filter ~ monitor_filters = esp32_exception_decoder sometimes not working.
But it looks like to me that if multiple files does not exist in the system, it is rebooting. If one or two file, no problem. But if 3 or more then reboot.

This is not necessarily the LITTLEFS problem, i think Async Webserver has this bug when you rapidly requesting something with HTTP_GET then it is going to crash.

[lorol]

use SPIFFS.begin() - with default parameters. not .begin(something)

[zekageri]

use SPIFFS.begin() - with default parameters. not .begin(something)

I get this:

if(!LITTLEFS.begin(FORMAT_LITTLEFS_IF_FAILED)){
        Serial.println("LITTLEFS Mount Failed");
        return;
    }

from LITTLEfs examples.