Unlocking and improving the Pythings SIM800 GPRS module for MicroPython

sarusso · February 21, 2020, 4:45pm

@weef sorry I must have mis-explained myself: I meant that the idf4 which is without PPP should work with this AT-based driver for the SIM800, but I was not sure about the benefits of using firmware idf4 over idf3 in general.

I fully agree on the benefits of using PPP (and therefore idf3) instead of relying on SIM800 internal SSL stack via AT commands, also because the SIM800 internal SSL stack does not perform certificate validation, which is something that should be possible to do at least in a not too far future. Agreed also on the protocol version concerns.

The main problem with what loboris does is that as far as I understood it is not a “pluggable driver” but requires an an entire firmware build + patches. Also, its Micropython fork [1] has diverged from the main repo [2], making it not much usable IMHO.

Having an AT-based, almost hardware-agnostic implementation allows for excellent porting across different boards and microcontrollers, this is why I followed this way even given the limitations.

I guess it depends on the use case, for sensitive data the SIM800 internal SSL implementation it is definitely not OK, for sending a bunch of metrics on a cloud service it might be a reasonable compromise.

This said, it would be great to have the Esp32 + SIM800 work out of the box with a vanilla MicroPython and proper SSL stack

[1] GitHub - loboris/MicroPython_ESP32_psRAM_LoBo: MicroPython for ESP32 with psRAM support
[2] Integrating Loboris port back to official uP ESP32 port - MicroPython Forum

Andreas · February 21, 2020, 5:34pm

Hi again,

thanks for having a fruitful discussion around that topic here. My proposal is to improve non-SSL communication for the AT-command-based driver through [SIM800] Problems with AT+HTTPSSL commands · Issue #3 · pythings/Drivers · GitHub and – quoting @sarusso here

– to look into PPP over Serial (PPPoS) support for SIM800 within ESP-IDF for ESP32 for gaining real PPPoS based on the current ESP-IDF.

With kind regards,
Andreas.

P.S.: There’s not much additional information in there, but just to list it as a reference: SIM800 Series_SSL_ApplicationNote_V1.02.

P.P.S.: While this is not specific to the ESP32, I would also like to point out to Driver for Adafruit GSM FONA module - MicroPython Forum (Archive) and https://github.com/adafruit/circuitpython/issues/250 here.

poesel · February 21, 2020, 6:27pm

For this application I take security if its there and works. If it does not I don’t care. If someone would like to know how much my hives weigh he can take a look at the Grafana dashboard. No need to hack protocols.

Revision: 1418B05SIM800L24

Andreas · February 21, 2020, 9:20pm

I’ve made some updates to the driver which should now gracefully fall back to not use SSL at all if the modeminfo string is smaller than SIM800 R14.00.

I’ve outlined the rationale to actually prefer modeminfo (retrieved by ATI) over the firmware revision (retrieved by AT+CGMR) within [SIM800] Problems with AT+HTTPSSL commands · Issue #3 · pythings/Drivers · GitHub.

weef · February 21, 2020, 9:34pm

thanks for explaining your intentions on your library approach. - Citing the loboris fork here by me is a bit unfair as it does not take care on backward compatibility to µpy mainline. I benefit from this fork while using it on an Odroid Go and don’t have to care about compatibility and hence ‘proprietary’ extensions … ;)

Andreas · February 21, 2020, 9:41pm

I liked it actually. Credit whom credit is due. I believe Boris already brought some good things related to PPPoS support into ESP-IDF the other day.

New patch for latest esp-idf added · loboris/ESP32-PPPOS-EXAMPLE@a2d642c · GitHub
PPPoS patch is merged into esp-idf · loboris/ESP32-PPPOS-EXAMPLE@6a36641 · GitHub

clemens · February 21, 2020, 10:02pm

Interesting that my firmware version number ist totally different from the structure. Must be very old! ;-)

Good to know that it works, on this issue someone reported problems ander the TinyGSM lib with https and the same number also: SIM 800L - HTTPS · Issue #235 · vshymanskyy/TinyGSM · GitHub

poesel · February 21, 2020, 10:58pm

It looks like one can update the firmware in the SIM800L. After a bit of googling it looks like I have the latest firmware version. Unfortunately, I haven’t found it anywhere for download. Only hint was here:

It may also be that the firmware update tool can download the firmware itself. Isn’t clear to me.

Simcom also has an official downloads directory with lots of files:
https://simcom.ee/documents/

Andreas · February 22, 2020, 3:33am

Dear Stefano,

This is our first attempt of adding support for PPPoS we have been discussing here and investigated further within PPP over Serial (PPPoS) support for MicroPython on ESP32. Please note this is still untested as I don’t have respective hardware on my workbench.

On Genuine MicroPython, this might work out-of-the-box. However, there definitively will be dragons.

With kind regards,
Andreas.

Andreas · February 22, 2020, 5:22am

If someone wants to use PPP-based networking on Pycom MicroPython, there are now updated firmware images on https://packages.hiveeyes.org/hiveeyes/foss/pycom/vanilla/.

FiPy-1.20.2.rc3-0.8.1-vanilla-squirrel-ppp.tar.gz
GPy-1.20.2.rc3-0.8.1-vanilla-squirrel-ppp.tar.gz
LoPy-1.20.2.rc3-0.8.1-vanilla-squirrel-ppp.tar.gz
LoPy4-1.20.2.rc3-0.8.1-vanilla-squirrel-ppp.tar.gz
SiPy-1.20.2.rc3-0.8.1-vanilla-squirrel-ppp.tar.gz
WiPy-1.20.2.rc3-0.8.1-vanilla-squirrel-ppp.tar.gz

This adds PPP networking from Genuine MicroPython in order to support PPP over Serial (PPPoS), which has not been included on Pycom devices up to this date.

github.com/pycom/pycom-micropython-sigfox

Add PPP networking to support SIM800 PPPoS implementation

pycom:Dev ← daq-tools:ppp-networking

opened 05:14AM - 22 Feb 20 UTC

amotl

+233 -0

Hi there. This adds PPP networking from Genuine MicroPython in order to suppo…rt PPP over Serial (PPPoS). It accompanies the improvement to the pure-Python SIM800 driver by @sarusso. See also https://github.com/pythings/Drivers/pull/5, [1] and [2]. With kind regards, Andreas. [1] https://community.hiveeyes.org/t/ppp-over-serial-pppos-support-for-micropython-on-esp32/2994 [2] https://community.hiveeyes.org/t/unlocking-and-improving-the-pythings-sim800-gprs-module-for-micropython/2978/29 cc @robert-hh

Andreas · February 22, 2020, 5:29pm

Hi guys,

could you also share the output of "AT+CIPSSL=?" with us? It might be better than going for "modem_info".

Thanks,
Andreas.

– SIM 800L - HTTPS · Issue #235 · vshymanskyy/TinyGSM · GitHub

poesel · February 22, 2020, 10:37pm

‘+CIPSSL: (0-1)’

Nicht so der Bringer…

Andreas · February 23, 2020, 4:05am

Man findet die Dokumentation bei 2.4.AT +CIPSSL Set TCP to Use SSL Function auf Seite 9 der SIM800 Series_SSL_ApplicationNote_V1.02. Damit kann man dann wohl abfragen, ob SSL angeschaltet wurde (1) oder nicht (0).

Vergessen wirs einfach und lassen die Implementierung so wie sie ist. Nur bei Lust und Laune… Wenn dann bräuchten wir beide Ausgaben – von Dir von @clemens – um diese vergleichen zu können.

weef · February 23, 2020, 4:05pm

so, zum Vergleich mal eine ‘alte’ firmware rausgekramt; vielleicht könntest Du, @Clemens, auch noch mal schauen - Du müßtest beim letzten Befehl das gleiche bekommen:

ati
SIM800 R13.08

OK
at+cgmr
Revision:1308B08SIM800L16

OK
at+cipssl=?
ERROR

Andreas · February 23, 2020, 4:45pm

Danke! Wenn das bei @poesel “OK” ausgibt, wollen wir das dann als extra Roundtrip zur Capability Detection einbauen, statt – wie momentan implementiert – auf die ati Zeichenkette zu schauen?

poesel · February 23, 2020, 7:19pm

Wie schon geschrieben, die Antwort auf

at+cipssl=?

ist

+CIPSSL: (0-1)

Andreas · February 23, 2020, 8:00pm

Ah ok. Sorry!

Hier noch gschwind die Referenzimplementierung von TinyGSM.

github.com

vshymanskyy/TinyGSM/blob/6770614e7633e32c0847bd0aae3fb3b380de037c/src/TinyGsmClientSIM800.h#L242-L252


      
            bool hasSSL() {
          #if defined(TINY_GSM_MODEM_SIM900)
              return false;
          #else
              sendAT(GF("+CIPSSL=?"));
              if (waitResponse(GF(GSM_NL "+CIPSSL:")) != 1) {
                return false;
              }
              return waitResponse() == 1;
          #endif
            }

Bump.

github.com/pythings/Drivers

Improve SSL capability check

opened 08:03PM - 23 Feb 20 UTC

closed 06:08PM - 29 May 21 UTC

amotl

Instead of relying on the modem revision (`R13.06` vs. `R14.18`), we might want …to use the `AT+CIPSSL=?` command, see also [1]. When the modem is not capable of SSL, it will respond with `ERROR`. Otherwise, it will respond with `+CIPSSL: (0-1)`. The [TinyGSM](https://github.com/vshymanskyy/TinyGSM) reference implementation can be found at [2]. [1] https://community.hiveeyes.org/t/unlocking-and-improving-the-pythings-sim800-gprs-module-for-micropython/2978/34 [2] [TinyGsmClientSIM800.h#L242-L252](https://github.com/vshymanskyy/TinyGSM/blob/6770614e7633e32c0847bd0aae3fb3b380de037c/src/TinyGsmClientSIM800.h#L242-L252)

weef · February 23, 2020, 8:06pm

ja, müssen wir wohl auch so machen, denn poesel will uns leider nicht verraten, ob bei ihm danach noch ne Leerzeile und nen OK kommen… ;)

clemens · February 23, 2020, 8:10pm

bei mir:

ati
  SIM800 R13.08
  OK

at+cgmr
  Revision:1308B07SIM800L16_20141031_1027
  OK

at+cipssl=?
  ERROR

Hatte beim letzten mal testen das AT-Kommando für die Revision in einen bestehenden sketch eingebaut, da ist wohl was bei der Ausgabe verschluckt worden, daher nun am Anfang nicht mehr abgeschnitten.

ansonsten
q.e.d.

poesel · February 23, 2020, 8:51pm

the secret is mine…

Nee, leider gibts da nicht mehr:

>>> modem.execute_at_command('firmware2', None, False)
'+CIPSSL: (0-1)\r\n'