Erschließung von I2S-Support und FFT für MicroPython auf Pycom/ESP32

Entwicklung / Development Audio
, , ,
1

Einleitung

Hier kümmern wir uns darum, wie wir I2S-Support auf Pycom/ESP32 innerhalb der MicroPython-Umgebung realiseren können, um am Ende dann einen geeigneten FFT-Algorithmus darauf aufsetzen zu können.

Auch andere haben Ähnliches vor:

Looking for help to develop an interface for ESP32 I2S
2

Habe den zitierten Text oben auch hier drüben bei ESP32 I2S support · Issue #185 · micropython/micropython-esp32 · GitHub gefunden. Der folgende Post dort im Verlauf ist auch spannend, nun fehlt noch die FFT:

As part of an air and noise pollution sensor project (Street Sense | Hackaday.io) I needed to support an I2S microphone using the ESP32 port. Using the ESP-IDF I implemented support for both I2S Master Rx and Tx . I spent a good deal of effort to bring the C code from a “just works hobbyist” quality to “industrialized” quality.

I put in a PR to mainline uPy:
micropython/micropython#4471

I prepared detailed class usage documentation and working uPy examples for popular breakout microphone and audio DAC breakout boards. Here:
GitHub - miketeachman/micropython-esp32-i2s-examples: Usage and examples for I2S support on the ESP32 microcontroller

Feedback and comments welcome at the PR or at the documentation repo

1 Like
5

Looking at ESP32 I2S support

Fundstück

Exzellent – viel besser. Danke für den Fund! Bei MicroPython » Hardware API » I2S support findet sich die (experimentelle) generische Schnittstellenbeschreibung.

Historie

Hier die Entwicklungshistorie dieser Funktionalität für MicroPython auf ESP32:

Aktueller Stand

Example usage - I2S Master Receive - Adafruit I2S MEMS Microphone (SPH0645LM4H)

from machine import I2S
from machine import Pin

bck_pin = Pin(14)  # Bit clock output
ws_pin = Pin(13)   # Word clock output
sdin_pin = Pin(12) # Serial data input

audio_in = I2S(I2S.NUM0,                                 # create I2S peripheral to read audio
               bck=bck_pin, ws=ws_pin, sdin=sdin_pin,    # sample data from an Adafruit I2S MEMS
               standard=I2S.PHILIPS, mode=I2S.MASTER_RX, # microphone breakout board, 
               dataformat=I2S.B32,                       # based on SPH0645LM4H device
               channelformat=I2S.RIGHT_LEFT,
               samplerate=16000, 
               dmacount=16,dmalen=256)
               
samples = bytearray(2048)                                # bytearray to receive audio samples

num_bytes_read = audio_in.readinto(samples)              # read audio samples from microphone
                                                         # note:  blocks until sample array is full
                                                         # - see optional timeout argument
                                                         # to configure maximum blocking duration

Sieht recht anständig aus. So wie hier ersichtlich in der Python Domäne super kompakt herausgeführt – standing on the shoulders of… image

Gedanken

Wenn wir das – aus der Perspektive von Bee Observer betrachtet – nach Pycom MicroPython portiert bekämen, könnten wir mehrere Fliegen mit einer Klapy… [1] [2] [3].

  1. So langsam sollten wir wirklich irgendwo eine Liste dieser Art von Edelsteinen anlegen, die wir auf diese Weise portieren wollen würden, sobald wir irgendwann soweit sind. ↩︎

  2. Natürlich sollte das Vanilla/Genuine MicroPython genauso auf Pycom Geräten laufen, nur fehlt dann die essentielle Softwareunterstützung für die meisten Funkperipheriegeräte – es bleibt dann vermutlich nur WiFi übrig. ↩︎

  3. Umgekehrt kommen für solche Versuche und der folgenden Anwendung in der Praxis natürlich haufenweise generische ESP32 breakouts in Frage. Wir freuen uns über positive Nachrichten, wenn hier jemand unter uns entsprechende Anstrengungen unternehmen sollte, sich dem Thema zu nähern. Auch Schadensberichte à la “dies und jenes funktioniert nicht” nehmen wir hier gerne im Logbuch auf. ↩︎

6

5 posts were merged into an existing topic: Diskussion zu I2S mit ESP32 und MicroPython

12

Ausblick

P.S.: Regarding hardware, Mike Teachman is also talking about

ESP32 hardware + INMP441 microphone

– esp32: add support for I2S by miketeachman · Pull Request #4471 · micropython/micropython · GitHub

14

FFT auf ESP32

Et voilà: ESP-DSP: The official DSP library for the ESP32 [1].

  1. Müsste noch auf MicroPython adaptiert werden… ↩︎

15

Just wanted to add a relevant discussion on the Pycom Forum here:

@Ron: Either you might know everything already or you might find this an interesting read for some aspects.

1 Like
16

Just for the sake of completeness and because I don’t know where else to put that, I also want to drop Free small FFT in multiple languages here.

However, I actually wanted to outline that after putting Support modonewire from vanilla MicroPython by amotl · Pull Request #356 · pycom/pycom-micropython-sigfox · GitHub forward, we would be reasonably trained of trying this:

1 Like
17

It looks like this is almost finished, especially since it is already used by Street Sense - A portable electronic device to measure air and noise pollution.

There is also another small project building upon it:

The world’s best motion-activated blinkenlights for pinewood derby cars.

– Eric Poulsen / Project Disco · GitLab

1 Like
18

Testing I2S support on Pycom devices

Introduction

Pre-flight tests

>>> from machine import I2S
>>> dir(I2S)
['__class__', '__name__', 'readinto', 'write', 'ALL_LEFT', 'ALL_RIGHT', 'B16', 'B24', 'B32', 'LSB', 'MASTER_RX', 'MASTER_TX', 'NUM0', 'NUM1', 'ONLY_LEFT', 'ONLY_RIGHT', 'PHILIPS', 'RIGHT_LEFT', 'deinit', 'init']

In the loop

We haven’t actually tested this for real. However, the little example program didn’t crash the machine, which is at least something. @clemens and anyone interested: You might want to try this at home – thanks already!

Instructions

The ready-made firmware image [1] might make your life easier. The example programs [2] will have to be adjusted to use the Pycom-specific pin definition like bck_pin = Pin('P9') instead of e.g. bck_pin = Pin(32).

  1. FiPy-1.20.1.r1-0.7.0-vanilla-dragonfly-onewire-i2s.tar.gz ↩︎

  2. GitHub - miketeachman/micropython-esp32-i2s-examples: Usage and examples for I2S support on the ESP32 microcontroller ↩︎

19

Many thanks @Andreas! I have no audio playback hardware for the FiPy here so I will test the

Recording to flash example

instead (this is also the domain we need later ;-).

https://github.com/miketeachman/micropython-esp32-i2s-examples/blob/master/examples/read-mono-mic-write-internal-flash.py

What pins?

In the recording example we have – beside the Vcc and GND pins – this pins:

bck_pin = Pin(14)
ws_pin = Pin(13)
sdin_pin = Pin(27)

The pin definition in Andreas’ sketch (P9 to P11) is a placeholder only. So let’s try to find out what pins we have to use on the FiPy!

It seems that we do not need special pins for I2S, see:
https://github.com/espressif/esp-idf/tree/93a8603c5/examples/peripherals/i2s

The following table describes the pins we use by default (Note that you can also use other pins for the same purpose).

I did not find any proofed pin definitions for the PyCom devices, so in case it is not working we may have a look at the ESP32 examples and used pins, eg. this and that.

So I will stay on the pins Andreas mentioned but in an other order due to better physically wiring:

bck_pin =  Pin('P10')
ws_pin =   Pin('P11')
sdin_pin = Pin('P9')

Minor changes

First I got this error message:

Traceback (most recent call last):
 File "main.py", line 72, in <module>
OSError: [Errno 19] ENODEV

In case I delete in line 72 the slash / it works better and produces a wave file!

s = open('/mic_recording.wav','wb')

Done 256000 bytes written to flash

Finally I changed the channel

channelformat=I2S.ONLY_LEFT,

First recording

I got this recorded file:

So it sound not perfect yet but there is some recording via an I2S mic to FiPy’s flash!

The original code was for a INMP441 mic, I’m using a ICS43432 so this can be a source of error (besides the pins) also.

Btw. I could download the wav file via Atom / pymkr also but it was corrupted, downloading via FTP leads to a playable file.

1 Like
20

Thanks for letting me know. Cheers! We might want to tell some people on the Pycom user forum accordingly [1].

When connecting the pins, there actually are design mistakes you would want to avoid. Regarding this, I collected the resources you referenced already and added some additional ones:

  1. ↩︎
21

I have tested different pin combinations but sound does not get better. So I will try my Adafruit mic I have ordered a long time ago, same was used in the example and test this next (in case I will find it :-).

In the code is a comment

Testing was done with an INMP441 Microphone

but in the readme the Adafruit breakout is mentioned:

I2S Boards Tested

  • Adafruit I2S MEMS Microphone Breakout - SPH0645LM4H

I have ordered some INMP441 mics also, really inexpensive, you can get it for 3-4 EUR at ebay. And in a nice form factor that will fit in a Lockenwickler

22

According to Street Sense Schematic-CPU Module-V01.pdf, Mike Teachman is in fact using the INMP441 for his Street Sense - A portable electronic device to measure air and noise pollution.

Good luck!

23

Getting real

Introduction

@clemens, @einsiedlerkrebs and me asked ourselves how processing the audio data would actually be done after ingesting it through the MicroPython I2S module. As Python will be too slow, data will have to be channeled through C-level implementations from the I2S driver to the analysis algorithm, most probably using DMA.

Measuring noise level (dbA)

moddba.c seems to be the most recent thing Mike Teachman is working on within his Street Sense - A portable electronic device to measure air and noise pollution. Its purpose is to calculate dB(A) from a stream of audio samples.

Within this blog post, he is outlining how it works:

Details

Fresh from the trenches, the C-level module has been integrated into the MicroPython program through continuous calculation of dbA, every second · miketeachman/micropython-street-sense@d625d0a · GitHub. Enjoy these snippets which outline how audio data is fed from the I2S source into the analyser sink. This is just concise and beautiful.

24

This is actually implemented within another module called modi2stools.c.

1 Like
25

A little bit off-topic regarding our requirements, but it outlines activity on the work of I2S support by Mike Teachman.

26

Mike Teachman is still working on I2S and improved the code examples just recently, see Comparing adbc7b7308...2549264e08 · miketeachman/micropython-esp32-i2s-examples · GitHub.

1 Like
27

Is Mike Teachman’s I2S example working with the current pycom firmware? Or do we need a special firmware?

28

The uasyncio library probably needs support from Genuine MicroPython core, so it might not be available on Pycom MicroPython yet.