We are conceiving a MicroPython-based datalogger.
@clemens started experimenting with LoRa transmission using a Pycom LoPy.
We stepped in and provided convenient telemetry client wrapper libraries for MicroPython to receive data from LoRa and forward it to MQTT or HTTP.
Data acquisition and telemetry client framework for MicroPython.
Documentation: terkin/src/micropython at master · daq-tools/terkin · GitHub
It supports these transports
as well as these serialization formats:
Besides others, there’s a simple demo program which sends dummy telemetry data in CSV format to the HTTP endpoint of “swarm.hiveeyes.org”:
As you can see here, this already does the job on MicroPython for Unix:
$ micropython -c 'import sys; sys.path.insert(0, ".")' examples/node_http_csv_raw.py
Channel URI: http://swarm.hiveeyes.org/api-notls/hiveeyes/testdrive/area-42/node-1
HTTP Path: /api-notls/hiveeyes/testdrive/area-42/node-1/data
Payload: ## temperature, humidity, weight
Channel URI: http://swarm.hiveeyes.org/api-notls/hiveeyes/testdrive/area-42/node-1
HTTP Path: /api-notls/hiveeyes/testdrive/area-42/node-1/data
Payload: 42.42, 84.84, 35
After receiving the data on the HTTP interface, Kotori republishes it to the MQTT bus, so:
$ mosquitto_sub -h swarm.hiveeyes.org -p 1883 -t 'hiveeyes/testdrive/#' -v
hiveeyes/testdrive/area-42/node-1/data.json {"temperature": 42.42, "humidity": 84.84, "weight": 35.0}
There’s also a gateway/forwarder which receives data from the LoRa subsystem and sends it to HTTP as well. Everything is forwarded verbatim without any decoding involved, the payload just is being announced as “Content-Type: text/csv” when performing the HTTP POST request:
It is based on the contribution “lopy_lora_server.py” by @constantinos on the Pycom Forum (thanks!):
Note: Please be aware that this hasn’t been run on any real iron yet. We appreciate your help in doing this!
Hi there,
we are happy to announce that we have been able to take this to the next level. Based on good experiences by @Ron, @tonke, @RalfL and others with MicroPython (thanks for sharing your efforts!), we are currently working on GitHub - hiveeyes/terkin-datalogger: Datalogger for MicroPython and CPython..
After resolving some dependency hell aspects (e.g. How to use "libpcre" on the Pycom MicroPython runtime? | Pycom user forum) - which is just business as usual - we are now running the telemetry subsystem of the Hiveeyes MicroPython firmware on real hardware (the FiPy by Pycom) and it works pretty well already.
The foundation of the configuration system has been laid out recently (see settings.example.py), so we are tagging this with 0.1.0 now to signal official alpha quality.
@einsiedlerkrebs is already working on LoRaWAN/TTN telemetry while we are also taking considerations into account which have been discussed with @tonke recently. Thank you so much!
As all of this is in a very early state yet, we are looking forward to get more eyeballs to it. Please ask us anything, we will be happy to give any answers and appropriate onboarding assistance.
With kind regards,
Andreas.
Bildernotiz mit Zeitstempel. Alles auch auf dachboden-testdrive-fipy.
Ziel: Beides ineinander konvergieren lassen ;].
Together with @clemens, we have been able to confirm that code from the current development state of the Hiveeyes MicroPython Firmware works successfully with and without sensors attached and MQTT-over-WiFi telemetry enabled.
Upgrade the Pycom device firmware to a recent version, see hiveeyes-micropython-firmware/pycom-firmware-upgrade.rst at master · hiveeyes/hiveeyes-micropython-firmware · GitHub.
# Find serial port.
make list-serials
# Configure serial port.
export MCU_PORT=/dev/ttyS16
# Pre-flight checks.
make list-boards
# Bootstrap dependencies once.
# Usage: After updating the list of dependencies, e.g. through "git pull".
make setup
# Upload firmware framework and sketch and trigger a cold start.
# Usage: Each time you modify code.
make recycle
The Hiveeyes MicroPython Firmware release 0.3.0 is around the corner. Before cutting this release, we will try to improve some more aspects.
settings.py" ermöglichen.While we haven’t been able to resolve every main topic referenced above, we have been able to make reasonable progress on many details. You will be able to get more detailed insights by scanning the raw changelog or the new release page.
We would like to thank all contributors who have been involved making this happen. Thanks also for excellent counseling to @weef, who just joined the game. You know who you are.
When working on this release, we primarily have been focused on optimizing energy consumption. There is two weeks worth of data accompanying this process available at Teststände / [amo] FiPy Testbench Power » 2019-06-12 until 2019-06-24.
We tried hard to annotate each significant interaction we had with the system under test or other relevant events incurred through the environment.
We will be dearly happy to have more testers and developers on the code base. As things are still being minted there and the framework feels reasonably robust right now, this will clearly have a positive impact. In this sense, we would like to encourage you to try this out if this sparks your interest.
Die großen Baustellen – leider noch nicht erledigt, teilweise jedoch bereits in Arbeit und prototypisch funktionabel.
Die ersten Schritte sind gemacht. Wilde Einstiegspunkte gibt es in top/down Reihenfolge bei
Der Code hängt noch ein wenig schief drin und könnte bei der nächsten Iteration besser gebündelt werden, es könnte aber auch schlimmer sein. @tonke und @einsiedlerkrebs haben damit bereits ein wenig gefunkt und jede:r andere kann dies auch tun und wird damit Daten wie gewohnt komfortabel ins Grafana Dashboard bekommen – ansonsten gebt bitte Bescheid wenns hakt!
Eine Vorschau dazu findet sich bereits oben bei Developing Terkin for MicroPython - #4 by Andreas und dachboden-testdrive-fipy.
Für weitere Schritte fehlt es an der Implementierung wesentlicher Telemetriekonzepte für die nahtlose Integration mit der gesamten Infrastruktur. Auch wenn entsprechende Pläne bereits vorliegen, konnten wir hier bisher noch nicht mit der Implementierung weiterkommen.
Dafür müssen ein paar Kabelbäume neu gezogen werden.
When looking beyond our current use on the Pycom FiPy and WiPy modules within the BOB Entwicklung - Hiveeyes, we recently learned MicroPython might be well supported on the ESP8266 as well these times, as 1. Getting started with MicroPython on the ESP8266 — MicroPython latest documentation is also suggesting. We would be happy to see if this little sister of the ESP32 will be able to run this piece as well or if we should prepare for a trimmed-down version when aiming at this target.
Switching between the Pycom Extension board for programming the FiPy via FTDI and the BOB-HAT or BOB-Shield in order to bring it into the field is currently a painful process, see Upload und initiale Konfiguration der MicroPython-Firmware.
Accessing the FiPy over network [1] and turning off deep sleep while in maintenance mode in order to gain steady network connectivity while reprogramming the device [2] makes sense, so we added this to our backlog [3] and started investigating already [4].
The goal for any improvements would be to make managing the FiPy over network as easy as possible [5].
We have been able to conceive the MicroTerkin Agent which helped tremendously for implementing a basic yet effective Wartungsmodus für den Terkin-Datenlogger and will follow up on that on a broader scope with Maintenance mode galore at some point in the future.
We would like to start shedding more light onto the details on which giants’ shoulders we are actually standing each day and night – even the dwarf giants’. Today, we would like to introduce you to the fine Dotty Dict package conceived and maintained by Paweł Zadrożny.
Dotty Dict is a dictionary wrapper for quick access to deeply nested keys. In practice, it looks like that:
>>> from dotty_dict import dotty
>>> dot = dotty()
>>> dot['very.deeply.nested.thing'] = 'spam'
>>> dict(dot)
{'very': {'deeply': {'nested': {'thing': 'spam'}}}}
We chose Dotty Dict in order to access nested data structures like outlined in the configuration file blueprint settings.example.py by conveniently addressing values using flattened keys in dotted accessor notation, like
logging_enabled = self.settings.get('main.logging.enabled', False)
We just found a minor issue with Dotty Dict and further made it ready for MicroPython in order to get rid of the self-knitted dotty_dict-01.patch we have been carrying around.
Both things we addressed there have been handled promptly. Thanks, Paweł!
Ping @Diren.
More updates.
While the feature development diverted into Kontinuierliche Verbesserungen des Terkin-Datenloggers (600er) and [Backlog] Terkin-Datenlogger für BOB, we would like to continue to give you infrastructural updates here.
The sandbox features a new make target "install-ftp" which uploads the whole bunch of sources to the device using FTP. In order to do that, you will have to configure the device’s IP address which you can find out using the new MicroTerkin Agent as outlined at Wartungsmodus für den Terkin-Datenlogger.
The synopsis would be like…
When the device is on the network, everything becomes more efficient.
# Signal IP address communication
export MCU_PORT=192.168.178.33
# Deploy sources using FTP
make install-ftp
# Attach to the console
make console
Currently, a full "make install" takes whopping ~2m10s to complete over UART/FTDI/USB. The new FTP transfer mechanism brings that down to ~30s for a full installation and ~17s for an installation without the dist-packages folder. ↩︎
Installing it should be as easy as "apt install lftp" or similar. ↩︎
A post was merged into an existing topic: Kontinuierliche Verbesserungen des Terkin-Datenloggers (600er)
We finally released Version 0.6.0 of the Terkin-Datalogger. Thanks to all who contributed to this effort!
We took the chance to spread the word about the datalogger software Terkin for MicroPython we are conceiving here on the Pycom user forum.
Enjoy reading.
Just recently, the datalogger program started to spark interest with others and we got support from @poesel to integrate support for BLE [1] [2] as well as LoRa/TTN telemetry from @Thias [3] after following up on what @einsiedlerkrebs and @tonke have been doing the other day. Thank you so much!
Saying this, we will be happy to accept further contributions from the community as we are aiming to make this more generic beyond its original scope of beehive monitoring.
In order to support that, we are exceptionally happy about the recent firmware release from Pycom finally bringing MicroPython 1.11 to the plate [4] and followed up on that by refactoring our codebase in order to bring it into a better shape [5]. Thanks for your input on that, @pinguin.
After shaving some more yaks, we are looking forward to the upcoming release version 0.7.0. As we have been able to unlock MicroPython module freezing for Pycom devices, there will be complete firmware images to flash to your device.
Terkin has finally grown a test harness based on pytest, running on CPython. Enjoy!
$ make test
test/test_basic.py::test_basic_esp32 PASSED [ 3%]
test/test_basic.py::test_basic_wipy PASSED [ 6%]
test/test_basic.py::test_basic_cpython PASSED [ 10%]
test/test_gprs_http.py::test_uplink_gprs_http PASSED [ 13%]
test/test_http_mocking.py::test_mocket_cpython_requests PASSED [ 17%]
test/test_http_mocking.py::test_mocket_socket XFAIL [ 20%]
test/test_http_mocking.py::test_httpretty_cpython_requests PASSED [ 24%]
test/test_http_mocking.py::test_httpretty_socket XFAIL [ 27%]
test/test_http_mocking.py::test_httpserver_cpython_requests PASSED [ 31%]
test/test_http_mocking.py::test_httpserver_socket PASSED [ 34%]
test/test_lorawan.py::test_uplink_system_temperature PASSED [ 37%]
test/test_lorawan.py::test_uplink_environmental_sensors PASSED [ 41%]
test/test_lorawan.py::test_downlink_interval_set PASSED [ 44%]
test/test_lorawan.py::test_downlink_interval_reset PASSED [ 48%]
test/test_lorawan.py::test_downlink_pause PASSED [ 51%]
test/test_lorawan.py::test_downlink_unpause PASSED [ 55%]
test/test_sensors.py::test_sensors PASSED [ 58%]
test/test_sleep.py::test_timesleep PASSED [ 62%]
test/test_sleep.py::test_lightsleep PASSED [ 65%]
test/test_sleep.py::test_deepsleep PASSED [ 68%]
test/test_sleep.py::test_maintenance PASSED [ 72%]
test/test_urequests.py::test_urequests_basic PASSED [ 75%]
test/test_urequests.py::test_redirect_cpython_requests PASSED [ 79%]
test/test_urequests.py::test_redirect_urequests PASSED [ 82%]
test/test_wifi.py::test_wifi_esp32 PASSED [ 86%]
test/test_wifi.py::test_wifi_wipy PASSED [ 89%]
test/test_wifi.py::test_wifi_cpython PASSED [ 93%]
test/test_wifi_http.py::test_uplink_wifi_http PASSED [ 96%]
test/test_wifi_mqtt.py::test_uplink_wifi_mqtt PASSED [100%]
============================ 27 passed, 2 xfailed in 15.84s ==========================
Hi again,
Mike Causer stellt viele wichtige Treiber fuer MicroPython bereit, von denen wir einige bereits unter Treiber- und Modulsammlung für MicroPython ausgestellt haben.
Unabhaengig davon kuratiert er auch unter Awesome MicroPython [1] eine Liste vieler Dinge im MicroPython Umfeld.
Auf Basis unseres Impulses bei Proposing a new section "Frameworks" · Issue #8 · mcauser/awesome-micropython · GitHub hat er nun einen Abschnitt MicroPython Frameworks eingerichtet, wo auch unser Terkin Datelogger seinen Platz gefunden hat.
We feel honored.
Viele Gruesse,
Andreas.
Hi again,
Mike Causer provides many important drivers for MicroPython, some of which we have already exhibited at MicroPython Driver and Module Collection.
Regardless of this, he also curates a list of many things in the MicroPython universe at Awesome MicroPython [1:1].
Based on our impulse at Proposing a new section "Frameworks" · Issue #8 · mcauser/awesome-micropython · GitHub, he set up a section of MicroPython Frameworks, where our Terkin Datalogger has also found its place.
We feel honored.
With kind regards,
Andreas.
Dear Terkin lovers,
it has been a while to see some updates in this space. A bunch of stuff happened since we reflected any updates here, so we would like to give you a short summary about it.
Published in April 2020 already, this release had a long list of updates. Kudos for contributions go to @poesel, @Thias, @wtf, @tonke, @sarusso and many others for testing and reporting bugs.
The patch releases included improvements to packaging, compatibility improvements between Genuine MicroPython and Pycom MicroPython and a fix to I2CBus.power_on (thanks, @tonke!).
Many thanks again to @Thias, @tonke and @clemens for their contributions. This release, published on May 4, 2020, has the following highlights.
Many thanks again to @Thias, @poesel, @tonke and @wtf for their contributions. This release, published on May 22, 2020, has the following highlights.
DS3231micro.py driver, see also Erschließung der DS3231 RTC mit MicroPython und Terkin - #34 by Andreas. Thanks a stack!The 0.11 patch releases mainly improve invocation and installation on CPython through another rigorous iteration phase which was driven by @MKO, who aimed at using Terkin on a Pi Zero W running Raspberry Pi OS – thanks for this!
Released in July 2021, after many improvements to the CPython support in the past, we consider it reasonable ready to work on this environment well, beyond its initial experimental status. On this matter, you might also want to follow Developing Terkin for CPython.
This release added some fixes to running Terkin on CPython once more and integrates a contribution by @Werner, implementing temperature compensation for HX711 sensors – thank you very much!
Thanks to @nznobody, this release, just published today, includes CSV-based telemetry serialization (TelemetryClient.FORMAT_CSV), adds the networking.ntp.enabled setting to enable NTP/RTC synchronization, and adds support for MicroPython for the VEDirect sensor adapter. Kudos!
Additionally, Terkin is now compatible with Mosquitto 2.0, see also Upgrade to Mosquitto 2.0 and is running its test harness on GitHub Actions.
Thank you very much for all your excellent contributions. You know who you are.
With kind regards,
Andreas.