FFT auf ESP32
Et voilà: ESP-DSP: The official DSP library for the ESP32 [1].
Müsste noch auf MicroPython adaptiert werden… ↩︎
Just wanted to add a relevant discussion on the Pycom Forum here:
@Ron: Either you might know everything already or you might find this an interesting read for some aspects.
1 Like
Just for the sake of completeness and because I don’t know where else to put that, I also want to drop Free small FFT in multiple languages here.
However, I actually wanted to outline that after putting Support modonewire from vanilla MicroPython by amotl · Pull Request #356 · pycom/pycom-micropython-sigfox · GitHub forward, we would be reasonably trained of trying this:
1 Like
It looks like this is almost finished, especially since it is already used by Street Sense - A portable electronic device to measure air and noise pollution.
There is also another small project building upon it:
The world’s best motion-activated blinkenlights for pinewood derby cars.
1 Like
Testing I2S support on Pycom devices
Introduction
Pre-flight tests
>>> from machine import I2S
>>> dir(I2S)
['__class__', '__name__', 'readinto', 'write', 'ALL_LEFT', 'ALL_RIGHT', 'B16', 'B24', 'B32', 'LSB', 'MASTER_RX', 'MASTER_TX', 'NUM0', 'NUM1', 'ONLY_LEFT', 'ONLY_RIGHT', 'PHILIPS', 'RIGHT_LEFT', 'deinit', 'init']
In the loop
We haven’t actually tested this for real. However, the little example program didn’t crash the machine, which is at least something. @clemens and anyone interested: You might want to try this at home – thanks already!
Instructions
The ready-made firmware image [1] might make your life easier. The example programs [2] will have to be adjusted to use the Pycom-specific pin definition like bck_pin = Pin('P9') instead of e.g. bck_pin = Pin(32).
Many thanks @Andreas! I have no audio playback hardware for the FiPy here so I will test the
Recording to flash example
instead (this is also the domain we need later ;-).
What pins?
In the recording example we have – beside the Vcc and GND pins – this pins:
bck_pin = Pin(14) ws_pin = Pin(13) sdin_pin = Pin(27)
The pin definition in Andreas’ sketch (P9 to P11) is a placeholder only. So let’s try to find out what pins we have to use on the FiPy!
It seems that we do not need special pins for I2S, see:
https://github.com/espressif/esp-idf/tree/93a8603c5/examples/peripherals/i2s
The following table describes the pins we use by default (Note that you can also use other pins for the same purpose).
I did not find any proofed pin definitions for the PyCom devices, so in case it is not working we may have a look at the ESP32 examples and used pins, eg. this and that.
So I will stay on the pins Andreas mentioned but in an other order due to better physically wiring:
bck_pin = Pin('P10')
ws_pin = Pin('P11')
sdin_pin = Pin('P9')
Minor changes
First I got this error message:
Traceback (most recent call last):
File "main.py", line 72, in <module>
OSError: [Errno 19] ENODEV
In case I delete in line 72 the slash / it works better and produces a wave file!
s = open('/mic_recording.wav','wb')
Done 256000 bytes written to flash
Finally I changed the channel
channelformat=I2S.ONLY_LEFT,
First recording
I got this recorded file:
So it sound not perfect yet but there is some recording via an I2S mic to FiPy’s flash!
The original code was for a INMP441 mic, I’m using a ICS43432 so this can be a source of error (besides the pins) also.
Btw. I could download the wav file via Atom / pymkr also but it was corrupted, downloading via FTP leads to a playable file.
1 Like
Thanks for letting me know. Cheers! We might want to tell some people on the Pycom user forum accordingly [1].
When connecting the pins, there actually are design mistakes you would want to avoid. Regarding this, I collected the resources you referenced already and added some additional ones:
- http://iot-bits.com/interfacing-an-audio-codec-with-esp32/ (Part 1)
- http://iot-bits.com/interfacing-audio-codec-esp32/
- https://github.com/maspetsberger/esp32-i2s-mems
- Inter-IC Sound (I2S) - ESP32 - — ESP-IDF Programming Guide latest documentation
- esp-idf/examples/peripherals/i2s at 93a8603c545fb8e54741d6685146e2f3b874378d · espressif/esp-idf · GitHub
I have tested different pin combinations but sound does not get better. So I will try my Adafruit mic I have ordered a long time ago, same was used in the example and test this next (in case I will find it :-).
In the code is a comment
Testing was done with an INMP441 Microphone
but in the readme the Adafruit breakout is mentioned:
I2S Boards Tested
- Adafruit I2S MEMS Microphone Breakout - SPH0645LM4H
I have ordered some INMP441 mics also, really inexpensive, you can get it for 3-4 EUR at ebay. And in a nice form factor that will fit in a Lockenwickler
According to Street Sense Schematic-CPU Module-V01.pdf, Mike Teachman is in fact using the INMP441 for his Street Sense - A portable electronic device to measure air and noise pollution.
Good luck!
Getting real
Introduction
@clemens, @einsiedlerkrebs and me asked ourselves how processing the audio data would actually be done after ingesting it through the MicroPython I2S module. As Python will be too slow, data will have to be channeled through C-level implementations from the I2S driver to the analysis algorithm, most probably using DMA.
Measuring noise level (dbA)
moddba.c seems to be the most recent thing Mike Teachman is working on within his Street Sense - A portable electronic device to measure air and noise pollution. Its purpose is to calculate dB(A) from a stream of audio samples.
Within this blog post, he is outlining how it works:
Details
Fresh from the trenches, the C-level module has been integrated into the MicroPython program through continuous calculation of dbA, every second · miketeachman/micropython-street-sense@d625d0a · GitHub. Enjoy these snippets which outline how audio data is fed from the I2S source into the analyser sink. This is just concise and beautiful.
A little bit off-topic regarding our requirements, but it outlines activity on the work of I2S support by Mike Teachman.
Mike Teachman is still working on I2S and improved the code examples just recently, see Comparing adbc7b7308...2549264e08 · miketeachman/micropython-esp32-i2s-examples · GitHub.
1 Like
Is Mike Teachman’s I2S example working with the current pycom firmware? Or do we need a special firmware?
The uasyncio library probably needs support from Genuine MicroPython core, so it might not be available on Pycom MicroPython yet.
This is Arduino C, not MicroPython, but perhaps some parts we can have a look at (in the last third)
Und hier auch noch Beispiel-Code für Arduino-C für einen ESP32, Vorbilder waren die Adafruit-Sketches zu I2S
Seems that Mike Teachman’s read-mono-mic-write-internal-flash.py I’ve used 2019-11 is no longer available, but you can find now an example writing to SD
Make it working on a PyCom device
First I installed Andreas’ FiPy firmware version with I2S support:
FiPy-1.20.1.r1-0.7.0-vanilla-dragonfly-onewire-i2s.tar.gz, use this or a newer version.
The original code uses an other SD card interface, according to SD I changed some lines to fit it for the FiPy.
And I changed the pins for the I2S mic to:
P11 - SCK
P22 - WS
P21 - SD
This is the modified code
# The MIT License (MIT)
# Copyright (c) 2019 Michael Shi
# Copyright (c) 2020 Mike Teachman
# https://opensource.org/licenses/MIT
# Purpose:
# - read 32-bit audio samples from the left channel of an I2S microphone
# - snip upper 16-bits from each 32-bit microphone sample
# - write 16-bit samples to a SD card file using WAV format
#
# Recorded WAV file is named:
# "mic_left_channel_16bits.wav"
#
# Hardware tested:
# - INMP441 microphone module
# - MSM261S4030H0 microphone module
import os
from machine import Pin
from machine import SD
from machine import I2S
#======= USER CONFIGURATION =======
RECORD_TIME_IN_SECONDS = 5
SAMPLE_RATE_IN_HZ = 22050
#======= USER CONFIGURATION =======
WAV_SAMPLE_SIZE_IN_BITS = 16
WAV_SAMPLE_SIZE_IN_BYTES = WAV_SAMPLE_SIZE_IN_BITS // 8
MIC_SAMPLE_BUFFER_SIZE_IN_BYTES = 4096
SDCARD_SAMPLE_BUFFER_SIZE_IN_BYTES = MIC_SAMPLE_BUFFER_SIZE_IN_BYTES // 2 # why divide by 2? only using 16-bits of 32-bit samples
NUM_SAMPLE_BYTES_TO_WRITE = RECORD_TIME_IN_SECONDS * SAMPLE_RATE_IN_HZ * WAV_SAMPLE_SIZE_IN_BYTES
NUM_SAMPLES_IN_DMA_BUFFER = 256
NUM_CHANNELS = 1
# snip_16_mono(): snip 16-bit samples from a 32-bit mono sample stream
# assumption: I2S configuration for mono microphone. e.g. I2S channelformat = ONLY_LEFT or ONLY_RIGHT
# example snip:
# samples_in[] = [0x44, 0x55, 0xAB, 0x77, 0x99, 0xBB, 0x11, 0x22]
# samples_out[] = [0xAB, 0x77, 0x11, 0x22]
# notes:
# samples_in[] arranged in little endian format:
# 0x77 is the most significant byte of the 32-bit sample
# 0x44 is the least significant byte of the 32-bit sample
#
# returns: number of bytes snipped
def snip_16_mono(samples_in, samples_out):
num_samples = len(samples_in) // 4
for i in range(num_samples):
samples_out[2*i] = samples_in[4*i + 2]
samples_out[2*i + 1] = samples_in[4*i + 3]
return num_samples * 2
def create_wav_header(sampleRate, bitsPerSample, num_channels, num_samples):
datasize = num_samples * num_channels * bitsPerSample // 8
o = bytes("RIFF",'ascii') # (4byte) Marks file as RIFF
o += (datasize + 36).to_bytes(4,'little') # (4byte) File size in bytes excluding this and RIFF marker
o += bytes("WAVE",'ascii') # (4byte) File type
o += bytes("fmt ",'ascii') # (4byte) Format Chunk Marker
o += (16).to_bytes(4,'little') # (4byte) Length of above format data
o += (1).to_bytes(2,'little') # (2byte) Format type (1 - PCM)
o += (num_channels).to_bytes(2,'little') # (2byte)
o += (sampleRate).to_bytes(4,'little') # (4byte)
o += (sampleRate * num_channels * bitsPerSample // 8).to_bytes(4,'little') # (4byte)
o += (num_channels * bitsPerSample // 8).to_bytes(2,'little') # (2byte)
o += (bitsPerSample).to_bytes(2,'little') # (2byte)
o += bytes("data",'ascii') # (4byte) Data Chunk Marker
o += (datasize).to_bytes(4,'little') # (4byte) Data size in bytes
return o
bck_pin = Pin('P11')
ws_pin = Pin('P22')
sdin_pin = Pin('P21')
audio_in = I2S(
I2S.NUM0,
bck=bck_pin, ws=ws_pin, sdin=sdin_pin,
standard=I2S.PHILIPS,
mode=I2S.MASTER_RX,
dataformat=I2S.B32,
channelformat=I2S.ONLY_LEFT,
samplerate=SAMPLE_RATE_IN_HZ,
dmacount=50,
dmalen=NUM_SAMPLES_IN_DMA_BUFFER
)
# configure SD card
# slot=2 configures SD card to use the SPI3 controller (VSPI), DMA channel = 2
# slot=3 configures SD card to use the SPI2 controller (HSPI), DMA channel = 1
sd = SD()
os.mount(sd, "/sd")
wav = open('/sd/mic_left_channel_16bits.wav','wb')
# create header for WAV file and write to SD card
wav_header = create_wav_header(
SAMPLE_RATE_IN_HZ,
WAV_SAMPLE_SIZE_IN_BITS,
NUM_CHANNELS,
SAMPLE_RATE_IN_HZ * RECORD_TIME_IN_SECONDS
)
num_bytes_written = wav.write(wav_header)
# allocate sample arrays
# memoryview used to reduce heap allocation in while loop
mic_samples = bytearray(MIC_SAMPLE_BUFFER_SIZE_IN_BYTES)
mic_samples_mv = memoryview(mic_samples)
wav_samples = bytearray(SDCARD_SAMPLE_BUFFER_SIZE_IN_BYTES)
wav_samples_mv = memoryview(wav_samples)
num_sample_bytes_written_to_wav = 0
print('Starting')
# read 32-bit samples from I2S microphone, snip upper 16-bits, write snipped samples to WAV file
while num_sample_bytes_written_to_wav < NUM_SAMPLE_BYTES_TO_WRITE:
try:
# try to read a block of samples from the I2S microphone
# readinto() method returns 0 if no DMA buffer is full
num_bytes_read_from_mic = audio_in.readinto(mic_samples_mv, timeout=0)
if num_bytes_read_from_mic > 0:
# snip upper 16-bits from each 32-bit microphone sample
num_bytes_snipped = snip_16_mono(mic_samples_mv[:num_bytes_read_from_mic], wav_samples_mv)
num_bytes_to_write = min(num_bytes_snipped, NUM_SAMPLE_BYTES_TO_WRITE - num_sample_bytes_written_to_wav)
# write samples to WAV file
num_bytes_written = wav.write(wav_samples_mv[:num_bytes_to_write])
num_sample_bytes_written_to_wav += num_bytes_written
except (KeyboardInterrupt, Exception) as e:
print('caught exception {} {}'.format(type(e).__name__, e))
break
wav.close()
os.umount("/sd")
sd.deinit()
audio_in.deinit()
print('Done')
print('%d sample bytes written to WAV file' % num_sample_bytes_written_to_wav)
and this a quite good recording:
I got not always such a comfortable quality and I have to investigate what the reasons are. One is: you have to keep a distance of 20 cm to get a nice recording, at least for speach.
After some more testing the audio quality is really good and stable. Perhaps a wiring problem before. You can also record sound direct in front of the I2S mic!
[edit] @cedric.cfk reported a bad audio quality, sounds similar to my first recordings. I tried to reproduce it by shaking the wires and connectors a bit and it seems to produce this metallic sound:
[edit2] Just FYI, it was actual a connection problem caused by bad cables (jumper wire) on @cedric.cfk’s hardware setup.
2 Likes
There is now an updated test / example code version for the first step – recording to SD card: bee-observer_bee-monitoring-micropython/mic-to-sd_adapted-fipy_2-advanced at master · ClemensGruber/bee-observer_bee-monitoring-micropython · GitHub
- records x times / files from the I2S mic
- with an adjustable length (seconds)
- increment file naming
- WiFi connection (in
boot.py) to download files via FTP and play it on your computer - Wifi and recording indicator via RGB-LED
Wiring is
For the FiPy you have to use our home brewed firmware with I2S dirver, the official PyCom firmware has this driver not build in! Use at least version FiPy-1.20.1.r1-0.7.0-vanilla-dragonfly-onewire-i2s.tar.gz or better a newer release, see Dragonfly firmware for Pycom/ESP32
1 Like
After getting a good audio stream out of the INMP441 the next “natural” step would be using this stream for a FFT analysis. Due to performance reasons we should count on this:
The official DSP lib from espressif contains a FFT analysis. But this module is not in our firmware yet.
Can someone guide us a bit more detailed through the process of firmware building and including this module?
[edit] Is ulab, see Treiber- und Modulsammlung für MicroPython - #8 by Andreas an alternative? Performance?
1 Like
Yes 
@pythoncoder would surely like to know if an FFT costs a king’s ransom. No, it doesn’t. In fact, a 1024-point float transform can be gotten in less than 2 ms on the pyboard.
1 Like