with
Board erased successfully in 20.4 seconds
and
Writing lopy4.bin at 0x00110000... (100 %)
Hash of data verified.
Wrote 2027520 bytes (1051577 compressed) at 0x00010000 in 19.7 seconds (822.3 kbit/s)...
and
make install
I confirm the “network card not available error” is gone. Thanks for preparing the pybytesless firmware!
WiFi always needs two attempts to establish a connection
13.8409 [terkin.device ] INFO : Starting networking
13.8697 [terkin.network.wifi ] INFO : Starting stopwatch
13.9666 [terkin.network.wifi ] INFO : Started stopwatch successfully
14.0106 [terkin.network.wifi ] INFO : WiFi STA: Networking address (MAC): {'ap_mac': '80:7d:3a:93:65:01', 'sta_mac': '80:7d:3a:93:65:00'}
14.0228 [terkin.network.wifi ] INFO : WiFi STA: Networking address (IP): ('0.0.0.0', '0.0.0.0', '0.0.0.0', '0.0.0.0')
14.0351 [terkin.network.wifi ] INFO : WiFi: Starting interface
14.0464 [terkin.network.wifi ] INFO : WiFi: Using internal antenna
14.0742 [terkin.device ] INFO : Start curating the garbage collector
14.3673 [terkin.device ] INFO : Curating the garbage collector finished. Free memory: 2394304
14.4658 [terkin.network.core ] INFO : Waiting for the network stack to come up within 5 seconds
14.5229 [terkin.network.core ] INFO : Network stack ready
14.5472 [terkin.network.core ] INFO : Starting mode server on 0.0.0.0:666
19.5743 [terkin.api.http ] INFO : Setting up HTTP API
19.7911 [terkin.api.http ] INFO : Starting HTTP server
20.4807 [terkin.network.wifi ] INFO : WiFi STA: Connecting to configured networks: ['WerNet']. Attempt: #1
20.5030 [terkin.network.wifi ] INFO : WiFi STA: Preparing connection to network "WerNet"
20.5198 [terkin.network.wifi ] INFO : WiFi STA: Getting auth mode
20.5913 [terkin.network.wifi ] INFO : WiFi STA: Auth mode from NVRAM with key=wa.9d66b1d414d4, value=3
20.6116 [terkin.network.wifi ] INFO : WiFi STA: Auth mode is "3"
20.6345 [terkin.network.wifi ] INFO : WiFi STA: Starting connection to "WerNet" with timeout of 30.0 seconds
20.7520 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 29.997 seconds
23.0801 [terkin.network.lora ] INFO : [LoRA] Not joined yet...
24.0355 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 26.713 seconds
25.6984 [terkin.network.lora ] INFO : [LoRA] Not joined yet...
25.7161 [terkin.network.lora ] INFO : [LoRA] joined...
27.0099 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 23.739 seconds
28.2332 [terkin.network.lora ] INFO : [LoRa] socket created
29.9835 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 20.766 seconds
...
47.7789 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 2.969999 seconds
50.7027 [terkin.network.wifi ] ERROR : Invoking "is_connected" failed
Traceback (most recent call last):
File "/flash/lib/terkin/network/wifi.py", line 193, in is_connected
TimeoutError: Connection to AP Timeout!
50.7909 [terkin.network.wifi ] ERROR : WiFi STA: Connecting to "WerNet" failed
Traceback (most recent call last):
File "/flash/lib/terkin/network/wifi.py", line 241, in connect_stations
File "/flash/lib/terkin/network/wifi.py", line 311, in connect_station
WiFiException: WiFi STA: Unable to connect to "WerNet"
50.8319 [terkin.network.wifi ] INFO : WiFi STA: Forgetting NVRAM data for network "WerNet"
50.9227 [terkin.network.wifi ] ERROR : WiFi STA: Connecting to any network candidate failed. Please check your WiFi configuration for one of the 1 station candidates.
50.9454 [terkin.network.wifi ] WARNING: Todo: We might want to buffer telemetry data to flash memory to be scheduled for transmission later.
50.9794 [terkin.network.wifi ] ERROR : WiFi STA: Connecting to configured networks "['WerNet']" failed
Traceback (most recent call last):
File "/flash/lib/terkin/network/wifi.py", line 177, in stay_connected
File "/flash/lib/terkin/network/wifi.py", line 258, in connect_stations
WiFiException: WiFi STA: Connecting to any network candidate failed
51.0679 [terkin.network.wifi ] INFO : WiFi STA: Retrying in 1.643164 seconds
52.7294 [terkin.network.wifi ] INFO : WiFi STA: Connecting to configured networks: ['WerNet']. Attempt: #2
52.7544 [terkin.network.wifi ] INFO : WiFi STA: Preparing connection to network "WerNet"
52.7735 [terkin.network.wifi ] INFO : WiFi STA: Getting auth mode
52.8464 [terkin.network.wifi ] INFO : WiFi STA: Unknown auth mode for network "WerNet", invoking WiFi scan
52.8667 [terkin.network.wifi ] INFO : WiFi STA: Scanning for networks
55.0350 [terkin.network.wifi ] INFO : WiFi STA: Networks available: ['NETGEAR85', 'WLAN-209719', 'Telekom_FON', 'Minionelch', 'FRITZ!Box 7560 DF', 'DIRECT-hqC48x Series', 'ShadowyPlanet', 'Mesh23', 'WerNet', 'freifunk-potsdam.de', 'devolo-f4068d8dc4d9', 'Potsdambabel', 'WLAN-728781', 'FRITZ!Box 7490 HKS', 'ARRIS-41A2', 'o2-WLAN35', 'Kuschelnetz', 'DIRECT-6C-HP OfficeJet Pro 6970', 'WLAN-EVS8XJ', 'TC-9BB6D', 'KDG-029D0', 'KabelBox-A0C8', 'o2-WLAN09']
55.0860 [terkin.network.wifi ] INFO : WiFi STA: Storing auth mode into NVRAM with key=wa.9d66b1d414d4, value=3
55.1090 [terkin.network.wifi ] INFO : WiFi STA: Auth mode is "3"
55.1337 [terkin.network.wifi ] INFO : WiFi STA: Starting connection to "WerNet" with timeout of 30.0 seconds
55.2497 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 29.998 seconds
55.5621 [terkin.network.wifi ] INFO : WiFi STA: Connected to "WerNet" with IP address "192.168.188.40"
55.6405 [terkin.network.wifi ] INFO : WiFi STA: Networking address (MAC): {'ap_mac': '80:7d:3a:93:65:01', 'sta_mac': '80:7d:3a:93:65:00'}
55.6617 [terkin.network.wifi ] INFO : WiFi STA: Networking address (IP): ('192.168.188.40', '255.255.255.0', '192.168.188.1', '192.168.188.189') 47.7789 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 2.969999 seconds
50.7027 [terkin.network.wifi ] ERROR : Invoking "is_connected" failed
Traceback (most recent call last):
File "/flash/lib/terkin/network/wifi.py", line 193, in is_connected
TimeoutError: Connection to AP Timeout!
50.7909 [terkin.network.wifi ] ERROR : WiFi STA: Connecting to "WerNet" failed
Traceback (most recent call last):
File "/flash/lib/terkin/network/wifi.py", line 241, in connect_stations
File "/flash/lib/terkin/network/wifi.py", line 311, in connect_station
WiFiException: WiFi STA: Unable to connect to "WerNet"
50.8319 [terkin.network.wifi ] INFO : WiFi STA: Forgetting NVRAM data for network "WerNet"
50.9227 [terkin.network.wifi ] ERROR : WiFi STA: Connecting to any network candidate failed. Please check your WiFi configuration for one of the 1 station candidates.
50.9454 [terkin.network.wifi ] WARNING: Todo: We might want to buffer telemetry data to flash memory to be scheduled for transmission later.
50.9794 [terkin.network.wifi ] ERROR : WiFi STA: Connecting to configured networks "['WerNet']" failed
Traceback (most recent call last):
File "/flash/lib/terkin/network/wifi.py", line 177, in stay_connected
File "/flash/lib/terkin/network/wifi.py", line 258, in connect_stations
WiFiException: WiFi STA: Connecting to any network candidate failed
51.0679 [terkin.network.wifi ] INFO : WiFi STA: Retrying in 1.643164 seconds
52.7294 [terkin.network.wifi ] INFO : WiFi STA: Connecting to configured networks: ['WerNet']. Attempt: #2
52.7544 [terkin.network.wifi ] INFO : WiFi STA: Preparing connection to network "WerNet"
52.7735 [terkin.network.wifi ] INFO : WiFi STA: Getting auth mode
52.8464 [terkin.network.wifi ] INFO : WiFi STA: Unknown auth mode for network "WerNet", invoking WiFi scan
52.8667 [terkin.network.wifi ] INFO : WiFi STA: Scanning for networks
55.0350 [terkin.network.wifi ] INFO : WiFi STA: Networks available: ['NETGEAR85', 'WLAN-209719', 'Telekom_FON', 'Minionelch', 'FRITZ!Box 7560 DF', 'DIRECT-hqC48x Series', 'ShadowyPlanet', 'Mesh23', 'WerNet', 'freifunk-potsdam.de', 'devolo-f4068d8dc4d9', 'Potsdambabel', 'WLAN-728781', 'FRITZ!Box 7490 HKS', 'ARRIS-41A2', 'o2-WLAN35', 'Kuschelnetz', 'DIRECT-6C-HP OfficeJet Pro 6970', 'WLAN-EVS8XJ', 'TC-9BB6D', 'KDG-029D0', 'KabelBox-A0C8', 'o2-WLAN09']
55.0860 [terkin.network.wifi ] INFO : WiFi STA: Storing auth mode into NVRAM with key=wa.9d66b1d414d4, value=3
55.1090 [terkin.network.wifi ] INFO : WiFi STA: Auth mode is "3"
55.1337 [terkin.network.wifi ] INFO : WiFi STA: Starting connection to "WerNet" with timeout of 30.0 seconds
55.2497 [terkin.network.wifi ] INFO : WiFi STA: Waiting for network to come up within 29.998 seconds
55.5621 [terkin.network.wifi ] INFO : WiFi STA: Connected to "WerNet" with IP address "192.168.188.40"
55.6405 [terkin.network.wifi ] INFO : WiFi STA: Networking address (MAC): {'ap_mac': '80:7d:3a:93:65:01', 'sta_mac': '80:7d:3a:93:65:00'}
55.6617 [terkin.network.wifi ] INFO : WiFi STA: Networking address (IP): ('192.168.188.40', '255.255.255.0', '192.168.188.1', '192.168.188.189')
hereafter no more logs are being produced
Just remembered that we also observed similar things like you outlined above at Stabilität und längere Testzeiträume des Terkin-Datenloggers - #64 by Andreas.
This also sounds familiar, see
Which means success or does the software just freeze afterwards?
I used to see the system variables to be printed and sent to swarm.hiveeyes.org in a loop before. Looks like a freeze
Seeing these weird errors again on your device, would you mind trying again with the webserver disabled?
'http': {
'enabled': False,
},
did this and make sketch-and-run but log still stops after:
WiFi STA: Networking address (IP): ...
and resetting the board leads to Core 1 panic
Guru Meditation Error: Core 1 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x40200754: bad00bad bad00bad bad00bad
PS: I’ve never seen this panic before using the pybytes-less pycom firmware. Can this be related?
I think it has something to do with the new 1.20.1 firmware. I saw this with pybytes activate also, see Troubleshooting the recent Pycom Firmware Release 1.20.1.r1 - #2 by clemens > section Hiveeyes-Software
Sorry to hear that this also hits you on a LoPy4. You might alternatively check out https://packages.hiveeyes.org/hiveeyes/foss/pycom/LoPy4-1.20.1.r1-robert.tar.gz, that’s all we can offer right now. This firmware image was built and provided by robert-hh.
If this error will not go away, you might well consider going back to the official pybytes firmware…
Sorry again, we are also just doing trial-and-error here on a regular basis. However, there are things indicating there might be something fishy under the hood.
Other users are also reporting these core panics on the Pycom user forum.
Please also enjoy
So, things like Soft errors caused by single-event upsets (SEUs) aka. ECC RAM absolutely matters might not be unrelated at all. Maybe some device models or revisions are more fragile than others.
Saying this kiddingly, I strongly believe this issue is more likely related to Random memory corruption faults on ESP32-WROVER rev.1 and rev.2 when running in dual-core mode.
Coming back from there, we built firmware images for the FiPy and the LoPy4 using
#define CONFIG_FREERTOS_UNICORE 1
I can’t tell for sure whether this makes any difference at all as the FiPy on my workbench has been running pretty stable even before without that setting.
Please also pull from our latest master as this will bring you Improve WiFi robustness on first connection attempt · hiveeyes/terkin-datalogger@d3ae518 · GitHub.
The AttributeError comes up, when bus-onewire-0 in settings.py is disabled
"id": "bus-onewire-0",
"family": "onewire",
"number": 0,
"enabled": False,
"pin_data": "P11",
Thanks for letting us know. Are you sure you are running the latest version from master or did you make some modifications locally? At line 329, we are seeing an empty line there.
However, we just added [1] and [2] to mitigate all conditions when accessing a sensor object connected to a bus object which has been disabled through the configuration settings.
Again I saw these core panics also with the unicore firmware. It starts happening after about 10 times flashing the device after PWRON. After dis/reconnecting the device from/to power the error is gone, likely until the next 10 flash procedures.
Thanks for letting us know.
Uploading raw source code through raw REPL is a rather heavy process which contributes to memory fragmentation on the device most probably leading to subsequent crashes, most probably caused by Random memory corruption faults on ESP32-WROVER rev.1 and rev.2 when running in dual-core mode.
If you are getting sick of this, we might want to go for more advanced and efficient methods bringing the source code to the device with less overhead.
I am using the one-stop
make recycle-ng MPY_CROSS=true MPY_TARGET=pycom MPY_VERSION=1.11
these days and would never look back. The background about why this is way more efficient is that it’s using FTP instead of raw REPL for transferring the files and that it will compile the sourcecode to bytecode before, reducing the overall size of stuff to be shipped to the device significantly.
$ du -sch dist-packages terkin
596K total
$ du -sch lib-mpy-1.11-pycom
352K total
This currently requires a network connection over WiFi with the IP address of the device stored in .terkin/floatip like
$ cat .terkin/floatip
192.168.178.40
For automating this, I am using
make terkin-agent action=maintain macs=80:7d:3a:c2:de:44 # libero
to find the device’s IP address and
make connect-wifi ssid=GartenNetzwerk password=<redacted>
to actually get the device into the network if required.
this revealed the following modules missing from the environment:
Do you mind adding these to the requirements files?
These guys are listed in requirements-terkin-agent.txt already. Just run
make setup-terkin-agent
Sorry that we haven’t documented each and every bit of these details yet. You are probably one of the first people running Linux who would like to use that infrastructure.
We followed the instructions on PSRAM Cache Issue stills exist (IDFGH-31) · Issue #2892 · espressif/esp-idf · GitHub thoroughly and just made another wave of releases. If we are lucky, [1] might be even more stable for you.
Dear @Thias,
we just released [1]. It includes some additional fixes on the ESP-IDF level and might also add some improvements on stability. For more details, enjoy [2].
With kind regards,
Andreas.
[1] https://packages.hiveeyes.org/hiveeyes/foss/pycom/vanilla/LoPy4-1.20.1.r1-0.5.0-vanilla-butterfly-csfix.tar.gz
[2] Pycom firmware bakery
We’ve just released another bunch of firmware images with hopefully improved robustness, so [1] might make you happy.