This is just a first attempt and has not been tested yet. @clemens or @poesel might want to give this a try on one of their TTGO T-Call-SIM800x devices. Thanks again, @sarusso!
3 Likes
A post was merged into an existing topic: Portierung des Terkin-Datenlogger auf ESP32 WROVER mit MicroPython 1.11
Success!
I flashed a T-Call with the uPy 1.12 idf3 binary (idf4 has no PPP and I think we need that here).
Then I synced your github unto the device. I commented the debugging messages back in but apart from that no changes. The pins are already correct.
example_usage()
Starting up...
Modem info: "SIM800 R14.18"
Network scan: "[{'name': 'vodafone', 'shortname': 'voda D2', 'id': '26202'}, {'name': 'D1', 'shortname': 'TMO D', 'id': '26201'}, {'name': 'E-Plus', 'shortname': 'E-Plus', 'id': '26203'}]"
Current network: "E-Plus"
Signal strength: "60.0%"
Modem IP address: "10.179.170.111"
Now running demo http GET...
Response status code: 200
Response content: <html><head><title>Current IP Check</title></head><body>Current IP Address: 5.35.166.143</body></html>
Now running demo https POST...
Response status code: 200
Response content: {"args":{},"data":{"myparameter":42},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","host":"postman-echo.com","content-length":"19","accept":"*/*","content-type":"application/json","user-agent":"SIMCOM_MODULE","x-forwarded-port":"443"},"json":{"myparameter":42},"url":"https://postman-echo.com/post"}
First run was a failure because signal strength was 0%. But it worked on the second.
I have a hologram card and this is the first time it worked! 
Grazie @sarusso!
3 Likes
Amazing! Super-happy to read this! 
p.s. I don’t think you need PPP as the communication is entirely serial with the SIM800L, so in theory idf4 should work as well, even if I am not sure what the benefits it would bring.
Many thanks @Andreas! I’ve started to get my hands dirty on that within Entwicklung der Terkin-Sandbox - #20 by clemens ff.
After some fiddling within Portierung des Terkin-Datenlogger auf Genuine MicroPython für ESP32 ff., I am now getting this:
39.8760 [terkin.telemetry ] INFO : Sending HTTP request over GPRS APN internet.telekom to https://swarm.hiveeyes.org/api/hiveeyes/testdrive/area-005/t-call-cg-01/data
40.1040 [terkin.telemetry ] INFO : Payload: {"system.memfree": 3818592, "system.runtime": 20, "system.uptime": 43.213, "system.time": 635552948}
40.8850 [terkin.telemetry ] ERROR : Telemetry to https://swarm.hiveeyes.org/api/hiveeyes/testdrive/area-005/t-call-cg-01 failed
Traceback (most recent call last):
File "/lib/terkin/telemetry.py", line 148, in transmit
File "/lib/terkin/telemetry.py", line 326, in transmit
File "/lib/terkin/telemetry.py", line 417, in send
File "/lib/terkin/network/gprs.py", line 72, in send_request
File "/dist-packages/pythings_sim800.py", line 357, in http_request
File "/dist-packages/pythings_sim800.py", line 173, in execute_at_command
GenericATError: Got generic AT error
41.1580 [terkin.datalogger ] WARNING: Telemetry status: FAILURE. 1 out of 1 targets failed. Status: {'https://swarm.hiveeyes.org/api/hiveeyes/testdrive/area-005/t-call-cg-01': False}
41.3880 [terkin.device ] INFO : Start curating the garbage collector
41.8120 [terkin.device ] INFO : Collecting garbage
42.2730 [terkin.device ] INFO : Curating the garbage collector finished. Free memory: 3900608
42.9730 [terkin.device ] INFO : Waiting for 54.705 seconds
It looks like SSL fails.
Even when switching to non-SSL:
File "/dist-packages/pythings_sim800.py", line 360, in http_request
File "/dist-packages/pythings_sim800.py", line 173, in execute_at_command
GenericATError: Got generic AT error
Mit diesen Auskommentierungen in pythings_sim800.py
# ..Do we have to enable ssl as well?
if url.startswith('https://'):
logger.debug('Http request step #1.3 (enablessl)')
#self.execute_at_command('enablessl')
elif url.startswith('http://'):
logger.debug('Http request step #1.3 (disablessl)')
#self.execute_at_command('disablessl')
else:
raise Exception('I have no idea how to handle communication protocol for URL "{}"'.format(url))
Läuft der sketch durch! Die Übeltäter sind
'enablessl': {'string':'AT+HTTPSSL=1', 'timeout':3, 'end': 'OK'},
'disablessl': {'string':'AT+HTTPSSL=0', 'timeout':3, 'end': 'OK'},
1 Like
$ mosquitto_sub -h swarm.hiveeyes.org -p 1883 -t '#' -v
hiveeyes/testdrive/area-005/t-call-cg-01/data.json {"system.memfree": 3894368.0, "system.time": 178.0, "system.uptime": 178.537, "system.runtime": 155.0}
Yeah!
2 Likes
It started working after commenting out lines 357 and 360 within SIM800L.py.
54.0850 [terkin.telemetry ] INFO : Payload: {"system.memfree": 3807520, "system.runtime": 34, "system.uptime": 57.159, "system.time": 57}
56.9340 [terkin.network.gprs ] INFO : [GPRS] HTTP response status code: 200
57.1580 [terkin.network.gprs ] INFO : [GPRS] HTTP response content: [
{
"message": "Received #1 readings",
"type": "info"
}
]
57.3910 [terkin.datalogger ] INFO : Telemetry status: SUCCESS (1/1)
:-)) Danke @Andreas!!
2 Likes
While it looks like this specific modem firmware does not support SSL at all, I can’t really believe it. However, we might want to investigate by invoking ATI or AT+CGMR.
Datasheet page 35
ATI: Display Product Identification Information
Datasheet page 68
AT+CGMR: Request TA Revision Identification of Software Release
Bin gerade schnell zur Arduino-IDE gewechselt und habe mit der TinyGSM über
modem.sendAT("+CGMR");
das ausgegeben bekommen:
SIM800L16_20141031_1027
Nur der Vollständigkeit halber, muss wohl ein anderes Problem gewesen sein:
@clemens, did you also try the example_SIM800L.py which worked for @poesel (it formerly was a fn call example_usage()) ? What does this script say on your T-call ?
Your connection to https://swarm.hiveyes.org failed, while https://postman-echo.com/post succeeded on poesel’s tests, which should be investigated first: you both have a TTGO Tcall from the same production run - therefore it’s not very likely that a different firmware in the modem is the reason.
Do we know that? We bought the devices seperately.
That came from the line:
#print(‘Modem info: “{}”’.format(modem.get_info()))
in example_SIM800.py. If there is anything else I can take a look at the board this evening.
Unfortunately I got the old and limp salat:
Modem: SIM800 R13.08
Think I bought the T-Call last summer in the time it was announced here at hiveeyes while @poesel ordered it in the last weeks.
Hmm, is this a hardware issue or can it be updated via firmware?
@poesel you may have a look at the “firmware version” or whatever it is via AT+CGMR I did it with the TinyGSM lib but it should be possible via the pythings code, you may supplement it here:
Volodymyr Shymanskyy the author of TinyGSM writes at
https://github.com/vshymanskyy/TinyGSM/issues/29#issuecomment-328802556
On my
SIM800 R14.18it works fine.
The other one,SIM800 R13.08reports that SSL is not supported.
So we should make steps in this direction. Btw. is there a date in SIM800L16_20141031_1027 and has my 2019 device a SIM800 module from 2014-10-31 or does it only look like a date by chance?
1 Like
@clemens, @poesel: OK, I somehow thought you were buying this within a same order, apparently no: even very different fw versions.
But this is one more reason to switch over to PPP when running on an ESP32 and no longer depend on this old IP stack on the SIM800.
…but sadly only without PPP.
well, this is the only way to overcome the limitations of the inbuilt IP stack of the SIM800L in terms of reasonable transport layer security: this stack only “features” SSL2, SSL3 and TLS 1.0, and the firmware is about six years old. The three mentioned cryptographic protocols are considered deprecated; SSL2 and 3 are dead, TLS 1.0 ist dead by the BEAST attack also since at least six years, within this spring 2020 all major browser will switch off their support for TLS 1.0 (and even TLS 1.1).
Even if one gets a SIM800 fw updated, it then will only be able to run outdated ‘secure’ protocols and ciphers. So there no other chance remains to update to decent security except via PPP usage and having the ESP32 IP stack doing the work. This way, it even doesn’t matter what particular fw a respective SIM800 has, as its ‘secure’ features are simple not used (it simply of course needs PPP support in its fw).
Here is how loboris does this (w/ a SIM800L ;) ):
3 Likes
@weef sorry I must have mis-explained myself: I meant that the idf4 which is without PPP should work with this AT-based driver for the SIM800, but I was not sure about the benefits of using firmware idf4 over idf3 in general.
I fully agree on the benefits of using PPP (and therefore idf3) instead of relying on SIM800 internal SSL stack via AT commands, also because the SIM800 internal SSL stack does not perform certificate validation, which is something that should be possible to do at least in a not too far future. Agreed also on the protocol version concerns.
The main problem with what loboris does is that as far as I understood it is not a “pluggable driver” but requires an an entire firmware build + patches. Also, its Micropython fork [1] has diverged from the main repo [2], making it not much usable IMHO.
Having an AT-based, almost hardware-agnostic implementation allows for excellent porting across different boards and microcontrollers, this is why I followed this way even given the limitations.
I guess it depends on the use case, for sensitive data the SIM800 internal SSL implementation it is definitely not OK, for sending a bunch of metrics on a cloud service it might be a reasonable compromise.
This said, it would be great to have the Esp32 + SIM800 work out of the box with a vanilla MicroPython and proper SSL stack
[1] GitHub - loboris/MicroPython_ESP32_psRAM_LoBo: MicroPython for ESP32 with psRAM support
[2] Integrating Loboris port back to official uP ESP32 port - MicroPython Forum
3 Likes
Hi again,
thanks for having a fruitful discussion around that topic here. My proposal is to improve non-SSL communication for the AT-command-based driver through [SIM800] Problems with AT+HTTPSSL commands · Issue #3 · pythings/Drivers · GitHub and – quoting @sarusso here
– to look into PPP over Serial (PPPoS) support for SIM800 within ESP-IDF for ESP32 for gaining real PPPoS based on the current ESP-IDF.
With kind regards,
Andreas.
P.S.: There’s not much additional information in there, but just to list it as a reference: SIM800 Series_SSL_ApplicationNote_V1.02.
P.P.S.: While this is not specific to the ESP32, I would also like to point out to Driver for Adafruit GSM FONA module - MicroPython Forum (Archive) and https://github.com/adafruit/circuitpython/issues/250 here.
1 Like