How to clone, install and run the "audiohealth" program

Entwicklung / Development Fehleranalyse / Troubleshooting
44

Hi Andreas,

I’d like to analyse .wav files from my beehives and have cloned and uploaded audiohealth 0.4.0 from github to my raspberry pi.
Would you be able to tell me if:

  1. there is a new version that I could clone? (current in github is 0.4.0 2017)
  2. Am I doing something wrong? Setup, permissions, error in command lines.
  3. do I need to do some pre editting to the samples ie (40sec samples, only bee sounds, no background noise)

Kind regards Paul

Raspberry Pi 3 Screen:

(.venv27) pi@raspberrypi:~/audiohealth $ audiohealth power --wavfile samples/CF003AD000.wav --pngfile samples/CF003AD000.png
==================
Peaks by frequency
==================
          86.13 Hz            130.14 RMS

==============
Peaks by power
==============
         130.14 RMS             86.13 Hz

========
Analysis
========
Traceback (most recent call last):
  File "/home/pi/audiohealth/.venv27/bin/audiohealth", line 11, in <module>
    load_entry_point('audiohealth', 'console_scripts', 'audiohealth')()
  File "/home/pi/audiohealth/audiohealth.py", line 488, in main
    tmpfile   = power_spectrum(wavfile)
  File "/home/pi/audiohealth/audiohealth.py", line 398, in power_spectrum
    freq500 = max(band500, key=peak_data.get)
ValueError: max() arg is an empty sequence

(.venv27) pi@raspberrypi:~/audiohealth $ audiohealth --version
audiohealth 0.4.0
1 Like
45

@Flat we as hiveeyes built the audiohealth framework based on an audio categorization algorithm from OSBH years ago. In this day OSBH published the relevant code as open source! Unfortunately the business model / mindset changed and OSBH did not release new code and optimized detection algorithms as open source, see:

Afaik OSBH published only the categorization alog and never the learning code. So we had no chance to go on without developing a big part of software new. Unfortunately we had to stopp developing because we have – with the current code – no chance to optimize the detection algorithem. So we gave up eo 2017.

2 Likes
46

@clemens: Thank you for the reply.

I guess the world of AI has changed since 2017 and no longer the cloistered retreat of the few very smart people. Hopefully it has become more mainstream as we move toward simpler AI tools accessible through Google, Azure and AWS.

I did manage to get some results from the legacy OSBH categorization tool. It made a nice learning experience for me to try out. I had some success running two commands with the file that failed in my previous comment.

1. Analyse

(.venv27) pi@raspberrypi:~/audiohealth $ audiohealth analyze --wavfile samples/CF003AD000.wav --analyzer tools/osbh-audioanalyzer/bin/test

Duration: 39s
File:     samples/CF003AD000.wav.dat
Strategy: lr-2.1
==================
Sequence of states
==================
swarm, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active
===================
Compressed timeline
===================
  0s -  10s   swarm           =
 10s - 270s   active          ==========================
==============
Total duration
==============
       260s   active          ==========================
        10s   swarm           =
======
Result
======
The most common events (i.e. the events with the highest total duration) are:
     The colony is mostly in »ACTIVE« state, which is going on for 260 seconds.
     Sometimes, the state oscillates to »SWARM«, for 10 seconds in total.
==========
Disclaimer
==========
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. NO LIABILITY FOR ANY DAMAGES WHATSOEVER.

2. Spectrogram

(.venv27) pi@raspberrypi:~/audiohealth $ audiohealth spectrogram --audiofile samples/CF003AD000.wav --pngfile samples/CF003AD000spec.png

total time: 39.90s, samplerate: 44.10kHz

A nice spectrogram image was created in file location: audiohealth/samples/CF003AD000spec.png

The Spectrogram function works with my .wav files but not .mp3.
A big thank you to all those engineers who built this for the beekeeping community to try out world wide.

2 Likes
47

Hi Paul,

thank you for sharing your observations when using the audiohealth program. As @clemens mentioned, we did not actively follow the development past the initial spike we conceived in 2017.

The code you will find in the repository is the most recent one we have. However, after tag 0.4.0, some more commits have been added. But I assume you cloned the repository and run the most recent version anyway?

I believe you followed all instructions correctly.

I am sorry that I can’t answer that question. Maybe @clemens has some additional thoughts here?

I believe we should get the code up to speed, add support for Python 3 and drop support for Python 2, which is way past end-of-life already. For that purpose, I just created a note at Add support for Python 3 · Issue #3 · hiveeyes/audiohealth · GitHub.

That would be another option (feature) to consider. In order to track it, I just created Add support for analyzing MP3 files · Issue #2 · hiveeyes/audiohealth · GitHub.

Please let us know if we missed to answer any of your questions or how we can support you further. If your journey will yield any better outcome than what we have right now, we will be all ears!

With kind regards,
Andreas.

48

Without looking,i think that the Spectrogram functions comes from Sox.
Have you try to install the .mp3 lib for SOX?

sudo apt-get install libsox-fmt-mp3
or
sudo apt-get install libsox-fmt-all
That sould be install all sound libs for Sox.

I’m thrilled that the code still works so far.
Because I myself will try to immerse myself in the world of audio analysis.

The first steps I have taken so far are. A permanent audio recording to be made with the help of saraswati.To generate data sets for machine learning.

Unfortunately mostly in German. Don’t be afraid to use the translation function and answer in english.

The project is maturing slowly but surely.
I would look forward to test files and cooperation in any case.
But since I don’t want any audio loss for a possible alogerism, i save my data in .flac format.

With kind regards,
Michael.

1 Like
49

I think the algorithm is not very robust. I would expect that a Bach prelude would lead to a “valid” output like swarm or active also and there is no error detection for “wrong” sounds. So it is always a good idea to trim files so that you do not have non-bee sound.

But let me emphasize this again: Detecting backround noise or other not bee based sound would be the job for an good algorithm. And even the output for bee sound I have seen 2017 was very fragile, see Analyze recordings from queen vs. queenless hives using "audiohealth" So I would not count on the ouput at all beside an experimental interest on the program.

MP3 files are optimized for humans, for speach, for music. The secret behind the MP3 technology is that “not important” sound elements are erased, reduced. So the original sound is modified! We (or I :-) do not know how different bee sound is compared to human sound. So it could be that important information in the be sound is deleted by the MP3 algorithm.

In the current state – we do not know what frequencies are important as information source – I would not use any lossy compression. Perhaps this may change in the future or you say it’s not practically doable without heavy compression but it is an imformation loss and we do not know if this information is important. Saying this with less than Wikipedia-knowledge about MP3 compression. So feel free to add more substantial information. ;-)

1 Like
50

@Andreas Your insight is much appreciated. Thankyou for your responses.

@MKO libsox-fmt-all is installed at the setup stage. Running the command again confirms the latest version and declares no updates required. Thanks for the ideas.

@clemens My hive installation generates (40sec) .wav files/min from the colony approx 120Mb and delivers them via wifi or 4G to my Raspberry Pi.
I notice using the results from the .wavfile function are not the same as the .audiofile function using the same sample file (PCM004.WAV)

(.venv27) pi@raspberrypi:~/audiohealth $ audiohealth analyze --audiofile samples/PCM004.WAV --analyzer tools/osbh-audioanalyzer/bin/test
Duration: 30s
File:     /tmp/tmpvRUkoK.wav.dat
Strategy: lr-2.1

==================
Sequence of states
==================
missing_queen, missing_queen, missing_queen

===================
Compressed timeline
===================
  0s -  30s   missing_queen   ===

==============
Total duration
==============
        30s   missing_queen   ===

======
Result
======
The most common events (i.e. the events with the highest total duration) are:

     The colony is mostly in »MISSING_QUEEN« state, which is going on for 30 seconds.

==========
Disclaimer
==========
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. NO LIABILITY FOR ANY DAMAGES WHATSOEVER.

(.venv27) pi@raspberrypi:~/audiohealth $ audiohealth analyze --wavfile samples/PCM004.WAV --analyzer tools/osbh-audioanalyzer/bin/test
Duration: 30s
File:     samples/PCM004.WAV.dat
Strategy: lr-2.1

==================
Sequence of states
==================
active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active

===================
Compressed timeline
===================
  0s - 230s   active          =======================

==============
Total duration
==============
       230s   active          =======================

======
Result
======
The most common events (i.e. the events with the highest total duration) are:

     The colony is mostly in »ACTIVE« state, which is going on for 230 seconds.

==========
Disclaimer
==========
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. NO LIABILITY FOR ANY DAMAGES WHATSOEVER.
51

Hi Paul,

thanks for your report.

Is there any way you could share this file with us?

With kind regards,
Andreas.

52 

wget -O PCM004.WAV \
  https://community.hiveeyes.org/uploads/default/original/2X/7/7595f1ad65c69e20c87dcb324f5d049080315aef.wav
2 Likes
53

Dear Paul,

I just pushed some updates to the repository. Please make sure to follow the slightly updated setup instructions at GitHub - hiveeyes/audiohealth: Bee colony vitality using audio analysis. Applies machine learning algorithms in C++ from the OSBH project..

"audiohealth power" now should work again. I am not sure about "audiohealth spectrogram" though.

I haven’t been able to find time to look into this. Apologies.

Unfortunately, on the audio file you provided, I can’t find the appropriate URL to download the file to my machine. It looks like Discourse does not reveal the original URL any more when uploading artefacts.

With kind regards,
Andreas.

54

Hi Andreas,
I dont have a URL for the file, its a simple .wav file from a Digitech XC-0389. To access the uploaded audio file. Is it possible to hover your the cursor over the uploaded audio file in chat, right click. “Save audio as” then you will have the audio file loaded to your computer.
Kind regards Paul

1 Like
55

or just click the 3 vertical dots next to the file in chat

56

Hi Paul,

Indeed I haven’t been aware of that, thanks a stack!

image

With kind regards,
Andreas.

P.S.: Right when you added your answer, I also have been able to find out about this detail (but also more regarding how Discourse encodes file upload identifiers) by asking at https://meta.discourse.org/t/decoding-upload-identifier-to-url/200117.

57

Dear @Jodaille and @Flat,

I can confirm errors with the files both of you provided, when invoking "audiohealth power ...". Thanks again. For tracking this error, I just created https://github.com/hiveeyes/audiohealth/issues/5.

With kind regards,
Andreas.

58

For me, when using the file PCM004.WAV,

wget -O PCM004.WAV \
  https://community.hiveeyes.org/uploads/default/original/2X/7/7595f1ad65c69e20c87dcb324f5d049080315aef.wav
$ audiohealth analyze --audiofile=PCM004.WAV --analyzer=tools/osbh-audioanalyzer/bin/test

this outputs:

Duration: 30.976031746031747s
File:     /var/folders/06/w9pzygdj7vx53n_0l9q_lhph0000gn/T/tmph5kac3uw.wav.dat
Strategy: lr-2.1

==================
Sequence of states
==================
swarm, swarm, swarm, swarm, swarm, swarm

===================
Compressed timeline
===================
  0s -  60s   swarm           ======

==============
Total duration
==============
        60s   swarm           ======

======
Result
======
The most common events (i.e. the events with the highest total duration) are:

     The colony is mostly in »SWARM« state, which is going on for 60 seconds.
59

I just fixed the errors. The outcome is that both of your files, @Jodaille and @Flat, look like that:

image

image
image1500×1000 143 KB

$ audiohealth power --audiofile=test_audiohealth.wav --pngfile=power.png

image

image
image1500×1000 97.4 KB

$ audiohealth power --audiofile=PCM004.WAV --pngfile=power.png

On the other hand, on a file provided by @clemens, the outcome looks like that:

image

image
image1500×1000 101 KB

$ audiohealth power --audiofile=colony-with-queen-gruber.wav --pngfile=power.png

P.S.: I remember seeing better results when rendering the spectrogram. See, for example, Queen piping: Visualization and analysis - #5 by Andreas.

60

Thank you Andreas. I successfully followed your “clone and install steps” with the following small modification.

  1. Rename existing dir audiohealth to audiohealthold otherwise git clone fails to fire up.
  2. I used sudo apt install python-virtualenv python-scipy python-numpy sox libsox-fmt-all xvfb xauth youtube-dl

I’m now seeing the same results as you. Thankyou

61

Hi clemns,
There are now many more audio files available since 2017.
Nguyen Huu Du et al. (2020) Audio Beehive Monitoring Based on IoT-AI.pdf (782.6 KB)
Kulyukin V (2018) Toward_Audio_Beehive_Monitoring_Deep_Learning.pdf (2.6 MB)
Are there any plans to run the AudioHealth learning script again to develop a new classification strategy lr3.0 or redesign to cnn1.0.

Kind regards Paul
There are so many new and interesting AI applications being developed its never been a better time to be in bee keeping. Very exiting!

62

Hi Andreas,

I have tried the following functions on one .WAV file with mixed results.

1a, 1w  Power with              --audiofile , --wavfile
2a, 2w  Spectrogram with        --audiofile , --wavfile
3a, 3w  Analyzer with           --audiofile , --wavfile

1a. Power with --audiofile

$ audiohealth power --audiofile samples/PCM009.WAV --pngfile samples/PCM009audpwr.png
==================
Peaks by frequency
==================

==============
Peaks by power
==============

========
Analysis
========
Colony has no activity.
Reason: There is no activity around 250Hz.

PCM009audpwr

1w. Power with --wavfile

$ audiohealth power --wavfile samples/PCM009.WAV --pngfile samples/PCM009wavpwr.png
==================
Peaks by frequency
==================
          93.75 Hz            190.13 RMS

==============
Peaks by power
==============
         190.13 RMS             93.75 Hz

========
Analysis
========
Colony has no activity.
Reason: There is no activity around 250Hz.

PCM009wavpwr

2a, Spectrogram with --audiofile

$ audiohealth spectrogram --audiofile samples/PCM009.WAV --pngfile samples/PCM009audspec.png
total time: 47.88s, samplerate: 24.00kHz

PCM009audspec
PCM009audspec1500×1000 90.3 KB

2w, Spectrogram with --wavfile

$ audiohealth spectrogram --wavfile samples/PCM009.WAV --pngfile samples/PCM009wavspec.png
Usage:
      audiohealth analyze --audiofile audiofile --analyzer /path/to/osbh-audioanalyzer [--strategy lr-2.1] [--debug] [--keep]
      audiohealth analyze --wavfile wavfile --analyzer /path/to/osbh-audioanalyzer [--strategy lr-2.1] [--debug]
      audiohealth analyze --datfile datfile --analyzer /path/to/osbh-audioanalyzer [--strategy lr-2.1] [--debug]
      audiohealth convert --audiofile audiofile --wavfile wavfile
      audiohealth spectrogram --audiofile audiofile --pngfile pngfile
      audiohealth power   --audiofile audiofile --pngfile pngfile
      audiohealth power   --wavfile wavfile     --pngfile pngfile
      audiohealth --version
      audiohealth (-h | --help)

3a, Analyzer with --audiofile

$ audiohealth analyze --audiofile samples/PCM009.WAV --analyzer tools/osbh-audioanalyzer/bin/test --strategy lr-2.1
Duration: 47.87206349206349s
File:     /tmp/tmpqsyb7a51.wav.dat
Strategy: lr-2.1

==================
Sequence of states
==================
missing_queen, swarm, swarm, swarm, collapsed, collapsed, collapsed, swarm, collapsed

===================
Compressed timeline
===================
  0s -  10s   missing_queen   =
 10s -  40s   swarm           ===
 40s -  70s   collapsed       ===
 70s -  80s   swarm           =
 80s -  90s   collapsed       =

==============
Total duration
==============
        40s   swarm           ====
        40s   collapsed       ====
        10s   missing_queen   =

======
Result
======
The most common events (i.e. the events with the highest total duration) are:

     The colony is mostly in »SWARM« state, which is going on for 40 seconds.
     Sometimes, the state oscillates to »COLLAPSED«, for 40 seconds in total.

==========
Disclaimer
==========
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. NO LIABILITY FOR ANY DAMAGES WHATSOEVER.

3w. Analyzer with --wavfile

$ audiohealth analyze --wavfile samples/PCM009.WAV --analyzer tools/osbh-audioanalyzer/bin/test --strategy lr-2.1
Duration: 47.872s
File:     samples/PCM009.WAV.dat
Strategy: lr-2.1

==================
Sequence of states
==================
active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active, active

===================
Compressed timeline
===================
  0s - 360s   active          ====================================

==============
Total duration
==============
       360s   active          ====================================

======
Result
======
The most common events (i.e. the events with the highest total duration) are:

     The colony is mostly in »ACTIVE« state, which is going on for 360 seconds.

==========
Disclaimer
==========
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. NO LIABILITY FOR ANY DAMAGES WHATSOEVER.

I’m not sure if the problem lies in the signal strength of the audio sample or the ability of lr-2.1 to discern.

Kind regards Paul

1 Like
63

As mentioned above. We do not have the OSBH learning script and it is closed source never published (afaik) from OSBH. So we would have to develop it new from scratch and also the matching analysing part. But I’m not so deep in the code perhaps @Andreas can add some insights what we have and what OSBH has closed and not public developed since 2017.

1 Like