Random memory corruption faults on ESP32-WROVER rev.1 and rev.2 when running in dual-core mode

Andreas · August 30, 2019, 8:38am

We are still experiencing different issues with the ESP32 on the Pycom devices, see also

Thomas Rogg of Neonius seems to have found a glitch related to PSRAM and dual core mode.

Faster, optimized ESP-IDF fork + PSRAM Issues – neonious Basics
Faster, optimized ESP-IDF fork + PSRAM Issues - ESP32 Forum
ESP32-WROVER is fundamentally broken (but is being fixed by Espressif now) – neonious Basics
https://github.com/espressif/esp-idf/issues/2892
GitHub - neonious/memcrash-esp32: Demonstrates PSRAM on ESP32 still has issues which results into crashes. *** Does not happen with low.js due to workaround

While we are not into the details of this yet, you might enjoy diving into it. It looks like Espressif is working on a workaround solution shipped through ESP-IDF to mitigate the issues for all the silicon which is already out there.

github.com

espressif/esp-idf/blob/a62cbfec/components/esp32/spiram_psram.h#L39-L41


Important is that NORMAL works with the app CPU cache disabled, but gives huge cache coherency
issues when both app and pro CPU are enabled. LOWHIGH and EVENODD do not have these coherency
issues but cannot be used when the app CPU cache is disabled.

Andreas · September 18, 2019, 8:44pm

Besides

there are also some other recent reports which indicate something might be fishy here.

Andreas · October 31, 2019, 12:53pm

Looking for things like https://github.com/espressif/esp-idf/search?q=bad00bad coming from Troubleshooting the recent Pycom Firmware Release 1.20.1.r1, we want to drop some more pointers here.

Andreas · October 31, 2019, 1:13pm

Falling back to unicore mode

lowjs

Thomas Rogg of Neonius says:

so they switched back to single-core mode:

OVMS

Others seem to be doing similar things for now:

Compiling OVMS

opened 11:17AM - 04 Oct 19 UTC

closed 01:26AM - 10 Oct 19 UTC

glynhudson

I'm trying to set up to ESP toolchain to be able to compile OVMS V3. I followed …the [Espressif Setup Guide](https://docs.espressif.com/projects/esp-idf/en/stable/get-started/) to install the latest tool chain and have been able to successfully compile the 'hello-word' example. I'm aware I now need to switch to the [OVMS specific older version of esp-idf](https://github.com/openvehicles/esp-idf) in order to compile OVMS V3. I've tried changing the `$IDF_PATH` to the OVMS esp-idf but I get a `Toolchain version is not supported` warning and some errors (see below). Any trips on how to make this work, I've been trying to follow the guide in the old OVMS developers gdoc. Once I get this working would you like me to write a guide for the new docs page? ``` WARNING: Toolchain version is not supported: esp32-2019r1 Expected to see version: crosstool-ng-1.22.0-80-g6c4433a Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk. WARNING: Compiler version is not supported: 8.2.0 Expected to see version(s): 5.2.0 Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk. WARNING: Toolchain version is not supported: esp32-2019r1 Expected to see version: crosstool-ng-1.22.0-80-g6c4433a Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk. WARNING: Compiler version is not supported: 8.2.0 Expected to see version(s): 5.2.0 Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk. WARNING: Missing submodule components/bootloader/subproject/components/micro-ecc/micro-ecc... Attempting 'git submodule update --init components/bootloader/subproject/components/micro-ecc/micro-ecc' in esp-idf root directory... error: pathspec 'components/bootloader/subproject/components/micro-ecc/micro-ecc' did not match any file(s) known to git. /home/bailey/Downloads/dev/esp-idf/make/project.mk:611: recipe for target '/home/bailey/Downloads/dev/esp-idf/components/bootloader/subproject/components/micro-ecc/micro-ecc/.git' failed make[1]: *** [/home/bailey/Downloads/dev/esp-idf/components/bootloader/subproject/components/micro-ecc/micro-ecc/.git] Error 1 /home/bailey/Downloads/dev/esp-idf/components/bootloader/Makefile.projbuild:41: recipe for target '/home/bailey/Downloads/dev/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/bootloader/bootloader.bin' failed make: *** [/home/bailey/Downloads/dev/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/build/bootloader/bootloader.bin] Error 2 ```

Andreas · November 1, 2019, 11:53pm

Damien George also confirms there are hardware bugs with and compiler workarounds against issues with the SPIRAM on ESP32.

Andreas · November 3, 2019, 4:22pm

Just for the sake of completeness, we are tracking some other presumably multicore pitfalls around the net which might come from the same thing.

Another file corruption issue · Issue #266 · pycom/pycom-micropython-sigfox · GitHub
Guru Meditation Error during socket.connect() · Issue #204 · micropython/micropython-esp32 · GitHub
arduino - ESP32 crashes and doesn't restart with watchdog - Stack Overflow
ESP32 [WiPy 3.0] GPIO onPinFall Causes Crash - Troubleshooting - ZERYNTH Community
Guru Meditation Error: Core 1 panic'ed (Cache disabled but cached memory region accessed) ESP32 ARDUINO IDE · Issue #1299 · espressif/esp-idf · GitHub
timer crash when using it with this library · Issue #854 · tzapu/WiFiManager · GitHub
Cache disabled but cached memory region accessed · Issue #1907 · espressif/arduino-esp32 · GitHub
https://github.com/espressif/arduino-esp32/issues/855
Panic when using Cache and Wifi and reading from flash · Issue #113 · 2dom/PxMatrix · GitHub
Guru Meditation Error: Cache disabled but cached memory region accessed - ESP32 Forum
Guru Meditation Error: Cache disabled but cached memory region accessed - ESP32 Forum
New firmware release 1.10.2.b1 | Pycom user forum

Andreas · February 4, 2020, 10:13pm

There have been some updates on the core issue within the Espressif issue tracker. Enjoy.