Thanks again @wjmb for your inspiring presentation the Beep Meetup January 2019 last weekend! I’m thrilled to “copy” your hardware / setup. I have the Feather M0 with RFM95 and also the I2S mic here on the desk but not yet time to solder it together. In case you want to share your findings and slides her also fell free to open a new topic. Would really appreciate it!!
I will look into this in the weekend (hopefully) … now I am very much looking forward to attending the Things Conference tomorrow and Friday. Best regards, Wouter,
@Clemens, concerning you reproducing my setup … I would very much welcome that if you can find the time.
For me the benefit would be that a second pair of eyes could help taking out bugs (at the moment I am still struggling with the I2S and I am not fully convinced that the FFT output is correct for low frequencies).
Once these bugs are out of the way, I plan to implement the algorithm from Kursk on the node, which might be interesting for you to follow/test/apply. Let me know what you need!
very cool, we will be happy to support you and give your setup more eyeballs. May I humbly ask whether you would like to share your firmware snippets maybe somewhere on GitHub? This would probably make it easier to iterate on that.
Personally, I will be happy to spend some time on the backend side [1] to enable it not only to acquire data from GStreamer streams arriving from Linux SBC machines but also from a raw PCM-over-UDP stream eventually coming from a MEMS mic attached to the ESP32 over I2S.
I am aware that you are already into running the FFT on the MCU, but we generally think it would still be cool to be able to stream full audio when connected to WiFi or so - a setup similar to the one @pbrier was aiming at. Maybe we can build a system which is capable of both things?
Another question comes to mind: How exactly do you receive the outcome from the FFT on the backend side? As @Diren also seems to do that when looking at her nice visualizations in Grafana, we would be eager to implement the missing bits into Kotori in order to make the whole infrastructure available to a larger audience. However, as we haven’t been able to spend more detailed research about the nitty gritty details of telemetry data formats on the wire for this usecase, we would need your assistance on that if you endorse this approach in general.
We will use an ESP32 system with LoRa also for BOB, as planned so far. The LoPy from PyCom. The reason is that people should be able to configure the scale or WiFi credentials or the TTN IDs without getting hands dirty with code. It should be done via a captive portal in the end version. So we need WiFi.
So Wouter, what is your plan? Will you stay on the Cortex M0 or would be a switch to an ESP32 device also possible?
Perhaps this is not a big deal because the same Software will run on both devices but we do not know right now!
I could get involved when in comes to the FFT and the data processing
So far I’ve only used build-in fft functions in R and python, even though I mainly work with R. I haven’t read anything about problems with low frequencies, so if you have some information about that, this would be interesting.
I also thought about coming up with a self-made custom implementation. For example, if we wanted to calculate the fundamental frequency, it might be nice to use the same fft-output for that, but at least in R all build-in functions calculate a fft again.
Greetings,
Diren
I was planning to stay with the Feather M0 LoRa, for the simple reason that I don’t have wifi at my bee stand. Also, this one has I2S.
However, I am open to suggestions. To me LoRa is a plus. Also, having some processing capability is required, because I think that some amount of edge computing will need to be done with working with audio. I2S seems nice, but any other MEMS microphone could do too.
I will look into GitHub … never used it actively before (other then for borrowing code ;-)
Will be happy to get you started. As @clemens also needs to ramp up his skills there, I was originally planning to offer a little Git workshop at the Beep Meetup, but obviously we would have required a two-day conference format in this case ;] - and nevermind: I’m very happy that I was able to attend and that our ways finally crossed.
On the technical side:
I consider this as a very valid scenario. Unlocking things on the ESP32 is happening elsewhere, so please just follow the M0 route. @clemens: Maybe code will converge, maybe not. But our experiences with the topic will grow nevertheless.
As I learned you generally already have good LoRaWAN coverage in the Netherlands, we might well have a look into the LoRaWAN offering by KPN or even TTN with your setup? On the backend side of this, we just started to work on a convenient adapter thing which is called ttnlogger [1] for converging the general data flow into a multi-tenant data acquisition system with the philosophy of Kotori [2].
Feel free to tell me whether this sparks your interest as I currently don’t know how you and @Diren are doing your FFT telemetry yet.
In this image an FFT is shown of a low noise environment. I have been struggling to understand the reason for the “jumps” at 0.5, 0.25, etc. of the spectrum. My present hypotheses are:
Something is not right with the way I obtain the I2S data. Perhaps I don’t take out the right bits from the 32 bit word. (I found that the SPH0645LM4H doesn’t implement the Philips mode right, by starting with data in the first bit, whereas the ICS43432 does).
Something is wrong in the way I cast datatypes (need to go from 18 bit (SPH0645LM4H) or 24 bit (ICS43432) to 16 bit for the FFT algorithm).
Something is not right with the FFT algorithm or the way that I use it.
Either way, it would be good to have a second pair of eyes on this. I will see if I can make all details available tonight. Most likely it will be in the weekend only.
Thanks! Even a pointer (url or so) to a small Github introduction would perhaps already help. I already have Git installed on my Windows PC (unfortunately I don’t have a Linux machine available).
Sure. I will see whether I can collect some appropriate resources which might help you to get started. Obviously there are already many resources on the Internet, but I will try to curate some and bring them into a form which might help members of our communities to get into that topic.
Actually, we will have to learn some details about Git first and then look into the things which are specific to GitHub as a sharing platform.
May I humbly ask you which kind of Git you already installed on your Windows machine? Naturally, there is more than one approach. One of the things colleagues of mine are using on Windows is TortoiseGit, which integrates into the graphical Windows environment excellently, see Screenshots – TortoiseGit – Windows Shell Interface to Git.
Personally, I know Tortoise from the days of CVS (TortoiseCVS) and Subversion (TortoiseSVN) and liked it very much.
Ok, thanks! The GUI and Shell Integration looks reasonable with the vanilla Git for Windows, but the user interface offered by the Tortoise family is yet a bit slicker and usually feels more integrated with the Windows environment re. nativeness.
Feel free to use whatever you like, it will get exciting when actually connecting with GitHub, as this is usually done over SSH, so we will have to create an appropriate SSH keypair for you and use it from Git. As I’m not a Windows user anymore I can’t tell whether GfW or Tortoise already offers user interface support for these configuration steps before actually being able to talk to GitHub. We will probably find out soon ;].
Good to know, so I can assume you are familiar with the concepts as well as the good and ugly things with software revisioning already? Of course, the good things outweigh the ugly things, so everyone dealing with code should actually be doing it ;].
Will be happy to assist you in coming back to this world, now with Git in these days.
Maybe let’s get into the details of this at the weekend if you can afford the time?
the code I just shared with you is pretty much the same as the code that I have running on a node in one of my beehives. It works fine on that hardware (with a SPH0645LM4H-B and DS18B20). The node that I have running on my desk (with an ICS43432 and a BM280) doesn’t like it that much though and I am struggling to find the reason:
I2S.end() hangs after the second time that I do a recording.
Not ending the I2S instances somehow interferes with LoRa (or vice versa), so that doesn’t seem to be an option.
Apart from this, as mentioned before, I don’t trust the FFT spectrum that comes out of this code (cf. the hypothetical causes that I could think of). I have another Arduino sketch to just test that part of the code, which I will upload later.
Obviously, this is work in progress. I hope to figure out how to improve it and will keep you update if I can make steps in the right direction. If you have suggestions, please let me know. Also, let me know if I can help with understanding what I am trying to do.
wow, that’s quite an achievement, thanks for sharing your work! Good to hear you already have a working variant on top of which we might iterate to get the code work on slightly different hardware.
I will be happy to assist you by code-staring and educated guessing (or pretending to do so) in order to find out the reason for the MCU freeze. While I have wrestled with similar things in the past, I ask for your understanding that it might become difficult without having actual hardware on the workbench. However, I don’t hesitate to get into the details modulo some time other obligations might require my attention.
Private vs. public or not, I feel this thread might spiral out of control when we actually start a ping pong game about discussing possible reasons and solutions for this problem, so we really should start a different one like “Feather M0 freezes when recording from a ICS43432 using I2S” for specifically naming and nailing the problem scope.
While I am relatively new to this topic and recognize the code is already working flawlessly with different hardware (kudos!), I recognized some spots in the code which sparked my interest and also found some resources on the Internet which might be related or could lead to more robust code. Maybe we can go through them together?
While I don’t know whether you have something similar on your workbench already, I am humbly asking to check out the overall system behavior when reading I2S using this minimal sketch. Please note I didn’t compile it yet as it’s really just a stripped down version of your sketch. It could either work out of the box or it might need a few adjustments regarding things I have missed.
Thanks for having a look. I will program the hardware later with your test-code. I looked at it and I think this will perform as expected, but it will be good to test and expand upon.
You know what is the strangest thing? So, if I use LoRa and I2S as in the sketch, the code stalls at I2S.end() when it is called after collecting the data for the second LoRa transmission. However, if I insert a bunch of Serial.Println(" test "); statements between each line in the I2S.end() implementation, it works.
