Using the Adafruit Feather M0 LoRa (RFM95) and TTN

Entwicklung / Development Funkinfrastruktur / Wireless infrastructure
, ,
1

For a first LoRaWAN-Test I used the Adafruit Feather M0 with RFM95 LoRa as Hardware to send test data to The Things Network.

There are some tutorials out for the Feather LoRa. Be aware that you get this device in two flavors: The “32u4” with an ATmega32u4 as processor and the “MO” with ATSAMD21G18 ARM Cortex M0.

For memory reasons and for the option recording sound via the I2S interface I would recommend the M0!

Available Tutorials

The two main discussion threads I got useful information to get the Feather M0 LoRa and TTN running are linked below. In this treads you can find general information and more or less reliable pin definitions. ;-)

A more structured tutorial for the Feather M0 is

Unfortunately the most interesting and inconsistent reported code section about the pin definition is not given in this tutorial. So I will document this in the next section.
Nevertheless I would recommend this tutorial for get it up and running.

Library

Arduino-LMIC library from Matthijs Kooijman

You have to make some changes in the default library for our use case, for details see the linked tutorial.

Pin Definition

To check the “by default” physically connected pins on the Feather see the Adafruit documentation page .

  • [Arduino pin] #8 - used as the radio CS (chip select) pin
  • #3 - used as the radio GPIO0 / IRQ (interrupt request) pin.
  • #4 - used as the radio Reset pin

In the discussion threads are different opinions if GPIO1 and GPIO2 is needed also RST is discussed as optional for the library.

This is my pin definition for the M0, (other pins for the 32u4 version needed!)

const lmic_pinmap lmic_pins = {
    .nss = 8,  
    .rxtx = LMIC_UNUSED_PIN,
    .rst = 4,
    .dio = {3,6,11},
};

Some Hints

Handling the Serial interface is a bit tricky on the Feather M0. Most confusion come from the fact that the Feather operates on two Serial ports. On my (Windows) system the Feather identifies on different ports for uploading and for serial outpu for debugging. After knowing this is is much easier to interface with the board.

For uploading you should doubleclick the reset button on the Feather to "switch to the right Serial. After uploading you have to choose the other Serial.

Range: 1 km

You can check the used gateway - after a successful upload - in the TTN console. My node connected over a range of 1 km to the Deutsche Bahn TTN Gateway on the Hauptbahnhof Berlin.

2017-10-02 15_32_49-Greenshot
2017-10-02 15_32_49-Greenshot.jpg969×524 160 KB

6 Likes
2

Thank you for starting this. I’m about to order an Adafruit Feather LoRa board this month as well. Looking forward to get this running for our use case.

3

Very cool, congratulations @clemens! I’m looking forward to letting the measurement data converge to MQTT somehow in order to propagate it to the collective data pool. I’ve stumbled across some bits of information about how to do that already [1,2], but i don’t have the big picture yet.

Would you mind sharing any information about that? Does your device already publish measurement data to a MQTT topic which is publicly available or what do you think about how would we go for that?

Another option could be to use the “HTTP Integration” feature [3] offered by the TTN service infrastructure. What do you think?

[1] Data API (MQTT) | The Things Network
[2] API Reference | The Things Network
[3] HTTP | The Things Network

1 Like
4

It’s a first attempt to open up TTN for our setup. The RFM95 modules are well known on the Hiveeyes community as successor of the RFM69 modules. So it was my first thought to go this way, also knowing that the protocol / library needs more power and memory. So a normal “Uno” is not the best choice and fast “full”.

I have no real data sent yet. Only test payloads. So not data stream now. And also no concept how to open Hiveeyes for TTN, but I guess we find something, probably a MQTT based solution.

I think you, @Andreas, has seen this TTN tutorial Store and visualize data using Node-RED, InfluxDB and Grafana because in the discussion forum you posted some links to Kotori. In the tutorial the linking software part is Node-RED. I don’t know if Kotori can do the same or more.

1 Like
5

After setting up a LoPy as (single channel) nano gateway and tweaking the LMIC code example I could successfully send data from the Feather M0 with RFM95 via LoRa to a 30 EUR (not fully compatible) TTN Gateway and forward the data via WiFi-Internet to the TTN Server!

The nice thing about that is: You can use the Arduino-Node (with LMiC) as "normal TTN node without any additional gateway in case a public gateway is nearby your apiary. If you have no gateway at your hives: place a cheap LoPy in a radius of around 500 m - with a free line of sight perhaps more - of your bees on a place with Wifi access. So you can make your own internet connectivity on an easy way.

3 Likes
6

3 posts were split to a new topic: Problems getting Adafruit Feather M0 LoRa (RFM95) running

7

now, how to get the data into the system?

Is it possible to add MQTT brigdes for TTN subscriptions on the server?

In my scenario I am sending a “Hello World” string to TTN which can be converted back from hex to a text with the “Payload Format” function that has to be placed into the individual TTN front end. My function just decodes the decrypted hex payload to an ASCII object called text:

function Decoder(bytes, port) {
  // Decode an uplink message from a buffer
  // (array) of bytes to an object of fields.
  var decoded = {};
  decoded.text = String.fromCharCode.apply(null, bytes);
  return decoded;
}

Now via the TTN MQTT API with, e.g.
> mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u 'thiaspdmtest' -P 'access-key-base64' -v

I get the entire package data from the LoRa uplink:

thiaspdmtest/devices/thias_m0_lora/up {"app_id":"thiaspdmtest","dev_id":"thias_m0_lora","hardware_serial":"003EAA5AA396B119","port":1,"counter":174,"payload_raw":"SGVsbG8sIHdvcmxkIVRoaXMgaXMgUG90c2RhbSE=","payload_fields":{"text":"Hello, world!This is Potsdam!"},"metadata":{"time":"2017-10-17T11:41:19.508166057Z","frequency":867.7,"modulation":"LORA","data_rate":"SF7BW125","coding_rate":"4/5","gateways":[{"gtw_id":"eui-b827ebfffe9ee1ab","timestamp":1667050292,"time":"2017-10-17T11:41:19.375299Z","channel":6,"rssi":-78,"snr":9.2,"rf_chain":0,"latitude":52.0,"longitude":13.0,"altitude":100}],"latitude":52.0,"longitude":13.0,"location_source":"registry"}}

The interesting part is the payload_fields, which contain the actual information. Changing the topic to -t '+/devices/+/up/text' only the text field is going to be received:

thiaspdmtest/devices/thias_m0_lora/up/text "Hello, world!This is Potsdam!"

mosquitto_sub also allows to subscribe to multiple topics at the same time. In order to get the several variables covered a bridge has to be configured for every topic and of course we’d need a topic remapping. I found some more useful hints here [1].

Is that something we can start with?

[1] Mosquitto MQTT Bridge-Usage and Configuration

1 Like
8

Very cool! If you can get some JSON into the payload, we should be able to start getting into the proper remapping. You might want to consider using some code doing this with ArduinoJson from node-wifi-mqtt.ino as a blueprint.

Looking forward to it… See also:

We would just have to do it “the other way round”, right? Because we won’t be able to configure anything on the TTN MQTT broker, our MQTT broker on swarm.hiveeyes.org would need to get the “satellite” role.

9

Sending JSON via LoRa would be against the idea of Low Power and Long Range radio transmission. Even the string I transmit is way to large. Data efficiency is top priority here. That’s why I want to have a look at the CayenneLPP standard which further reduces the payload to a minimum. Ideally only a channel number and the value should be transmitted.

If there’s a good chance to get this implemented I’d call that the most elegant and least complex solution.

10

I hear you. I already stumbled across CayenneLPP [1] and TheThingsMessage [2] the other day, but i can’t wrap my head around the specifics regarding hardcoded data types like temperature, humidity, etc. by using the respective encoding methods addTemperature, addRelativeHumidity, etc. This seems somehow non-generic to me as i’m thinking about things like “How to encode multiple temperature values?” or “How to give short, but specific names to my telemetry fields?”. Maybe you can shed some light on this?

[1] CayenneLPP | The Things Network
[2] TheThingsMessage | The Things Network

On the other hand, you might want to have a look at the BERadio C++ library which is conceived for the very same purpose of squeezing as much information as possible into the 61 byte message payload size available over RFM69 while still being flexible about the data to be encoded into it. @einsiedlerkrebs and me worked very hard on this and added some convenient features. Especially, we are somehow proud of the automatic message fragmentation feature:

By featuring automatic message fragmentation, data transmission is safe, even when using radio transceivers with constrained payloads. Yet, it is reasonably compact and still readable by humans (8-bit clean).

While not completely sure if this would fit the use case, we would be happy if this could spark your interest. Feel free to ask any questions about it.

11

…and for the LoPy one needs to port bencode to microPython (when running on mP and not arduino-esp32 ! ,)

I endorse bencode usage also because it can deal with binary data, this makes it also interesting for using it on a downlink channel.

12

I did not dive deep in the TTN “application” area. So just a bit loud thinking than knowing:

Payload size is again the limiting factor. So we have to decide and discus the same things we had on the plate two (?) years ago:

We can transmit values only

With values only we can get a nice compact payload e.g.

2017/10/17 21:40:23,   8.135, 18.4, 80.0, 5.41

but we have to know and to “write down” what the CSV fields are on an other place, here:

Date/Time, Weight, Outside Temp, Outside Humid, Voltage

In case we want to split variable value and variable name / identifier we could have this information on the TTN level. So a TTN application can merge values and names and send a JSON object to hiveeyes. But it must not be on the TTN side. While we have similar use cases by transmitting via GSM the mapping value to identifier can also happen on the hiveeyes side and TTN sends only the untouched CSV block to hiveeyes.

Sending values and identifier in a compressed payload

A more general approach would be to use BERadio, so all information value and identifier is always in the message, advantages

  • no one has to know / to save variable names
  • mis-matching between variables and prior announced variable sets can be avoided
  • you can send different datasets because the gateway / mapping instance needs no static pattern, so you can send once a day the battery level but every 10 minutes weight and temperature

You have to “pay” it with a longer payload.

Reducing payload size

I’m not very happy with the TTN payload, it is not human readable, all libraries made for efficiency, means smaller and more compressed is better. The downside is always you have to know on both sides the “sender” and the “receiver” what values are transmitted exactly. So you can not send weight and temperature and on an other node weight temperature and humidity. It works only when you define the variable set for every single node in advance, changes on runtime are dangerous.

If you can live with this limitations you can squeeze the payload a lot: Unix time stamp instead of human readable date/time combinations 8135 instead of 8.135 kg (so you have to know the decimal places of every variable) or 184 vs. 18.4 °C Cayenne Low Power Payload adds an numer for the “data channel” and the “Data type” with the type the number of decimal places are defined.

There is some advice how to reduce payload size but it is going too far for me. We don’t make rocket science and e.g. dealing with offsets to have smaller payloads is no way for me and relays to much on historic datasets and opens a big gate for errors. So reducing yes, but not to the last drop.

My temporary conclusion

CSV would be a way to go for me. We forward the payload on the TTN side and merge values and variables on the hiveeyes backend side. All is prepared already on the hiveeyes side, GSM data coming in via http already and TTN supports this also see HTTP Integration. Downside: Vairable set must be defined up front and could be hard changed. In cast there is a mismatch we will notice it only when values are not plausible.

BERadio is a bit more “expensive” (in sense of size) but much more generic.

13

Thanks @clemens for the thorough explanation.

Just for the records: There’s still another option by just sending single readings in a plain text format like:

Weight: 42.42

Temperature: 38.4

This also retains a maximum of flexibility on the “naming things” aspect.

14

Also @Thias: I’m as well happy to support the CayenneLPP payload format on the decoder side, no problem. We started doing binary data receiving with our data collector the other day, see LST — Kotori 0.22.7 documentation. So we could revive this subsystem to make it decode the binary data structure defined by Cayenne.

Just let me know if you want to go down this route, then we can dive into this topic together.

15

Assuming a condensed JSON object sent to TTN I tested a scenario with 5 sensors data values and created this Decoder function in TTN:

 function Decoder(bytes, port) {
  var data = JSON.parse( String.fromCharCode.apply(null, bytes) );
  return { 'data.json': {
    temperature_in: data.T,
    temperature_out: data.O,
    humidity_in: data.H,
    battery_level: data.B,
    weight: data.W }
  }
 }

With an artificial JSON object of {"T":33.4,"O":23.4,"H":90.1,"B":70.8,"W":45.25} this results into 47 bytes transmitted (without LoRa headers) and this decoded payload:

{
  "data.json": {
    "battery_level": 70.8,
    "humidity_in": 90.1,
    "temperature_in": 33.4,
    "temperature_out": 23.4,
    "weight": 45.25
  }
}

Subscribing to the TTN MQTT topic -t '+/devices/+/up/data.json' -v returns the reformatted JSON object:

hiveeyes/devices/thias_lora/up/data.json {"battery_level":70.8,"humidity_in":90.1,"temperature_in":33.4,"temperature_out":23.4,"weight":45.25}

47 bytes is less than I expected and could still meet the TTN fair use policies considering a reasonable transmission interval. These are the 5 values I’m sending to our backend and from my perspective the most relevant as well. If we’d agree on a common name space for the variables sent from the node and the MQTT topics that would be one step forward.

Edit:
@Andreas regarding the TTN MQTT structure: The only parts we are free to chose by setting up the TTN environment is the first (TTN Application ID), the third (TTN Device ID) and last (given by the decoder function). All others (devices and up) are fixed.

1 Like
16

Hey @Thias, this sounds pretty cool! Let’s think about how to let this converge to Kotori on swarm.hiveeyes.org when i’m back to Berlin.

1 Like
17

Live data from my Adafruit LoRa node arrives on swarm.hiveeyes.org by configuring a MQTT bridge directly on the server. It subscribes to my TTN applications topic, that provides the data as an JSON object and remaps it to my individual Hiveeyes topic. I know that this is just a temporary solution since we don’t want to tamper with the mosquitto config each and every time a TTN device gets added in the coming years.

/etc/mosquitto/conf.d/bridge-thias-ttn.conf :

# subscribe to TTN topic hiveeyes/devices/thias-hive2/up/data.json                                                                                    
connection ttn-thias
address eu.thethings.network:1883
remote_username hiveeyes
remote_password TTN-APP-KEY
# remap prefixes of remote topic (hiveeyes/devices/) to local topic (hiveeyes/thias/)
topic thias-hive2/up/data.json in 0 hiveeyes/thias/ hiveeyes/devices/
start_type automatic
3 Likes
18

Great! Battery graph looks better now, what did you do / was the problem?

19

I realized the battery was not attached to the board :) and the on-board voltage divider resistors are not accurate at all. Strangely the values measured were still very noisy. I now attached my own 2x220k voltage divider to the board and will fit the A0 pin ADC level to the battery percentage mapping according to the maximum and minimum values. Let’s see if this results in a smoother time series.

1 Like
20

Congratulations @Thias! This is really really cool. I wouldn’t have imagined we could really integrate TTN nodes without touching anything on the infrastructure side (code-wise). Thanks a bunch for your efforts!

From my perspective, it’s fine enough ;]. But i see your point: Mosquitto probably wouldn’t provide a runtime configuration subsystem so we have to look for another thing to manage that kind of bridging dynamically. I can think about three options:

  • Look out for MQTT brokers providing runtime configuration. I’m very happy with Mosquitto, but for sure we can look for other MQTT brokers maybe running besides each other.
  • Wrap Mosquitto into a dynamic configuration adapter which offers an API to register bridges at runtime. Sounds flaky at first, but is certainly doable and we wouldn’t need another piece of infrastructure. As a blueprint, have a look at the NGINX Plus API, which can

update upstream configurations […] on the fly with zero downtime.

  • Write a little bridge-like MQTT forwarder on our own providing the desired functionality. Maybe we could even use mqttwarn for that. Drawback: mqttwarn doesn’t provide a runtime configuration subsystem either ;].

Let us know when you want to dive into that topic. We are looking forward to discuss these options or other ideas with you.

Have fun!

1 Like