Web-Service "Audioanalyse"

Entwicklung / Development Audio
3

Spezifikation der HTTP Schnittstelle

Dies ist ein Vorschlag, wie eine flexible HTTP Schnittstelle für solche Zwecke gestaltet sein könnte.

Upload

Als mögliche Antwort des Aufrufs wäre u.U. folgendes denkbar:

HTTP/1.1 201 Created
Content-Type: text/uri-list

https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9.json
https://swarm.hiveeyes.org/api/hiveeyes/testdrive/area-42/node-1/item/3b57fcef938b27846d262c83c0f574f8015b5bd9.json?time=2017-07-25T00:40:41Z&has_queen=true

Unter der Haube würde die Datei dann archiviert und (asynchron) analysiert werden.

Retrieval

Eine GET Anfrage an eine dieser URIs liefert dann folgende Antwort:

HTTP/1.1 200 OK
Content-Type: application/json

{
    "item": {
        "resource": "https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9/audio.wav",
        "type": "audio",
        "name": "beehive_audio_2017-07-25T00:40:41+00:00.wav",
        "sha1": "3b57fcef938b27846d262c83c0f574f8015b5bd9"
    },
    "meta": {
        "time": "2017-07-25T00:40:41+00:00",
        "has_queen": true
    },
    "address": {
        "network": "testdrive",
        "location": "area-42",
        "node": "node-1"
    },
    "analysis": {
        "image": {
            "sonagram": "https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9/sonagram.png",
            "power-spectrum": "https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9/power-spectrum.png"
        },
        "report": {
            "power-spectrum": "https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9/power-spectrum.txt",
            "osbh-ml": "https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9/osbh-ml.txt"
        },
        "state": {
            "osbh-ml": "active"
        }
    }
}

User interface

Eine Benutzerschnittstelle in HTML ist - wie immer - ein etwas höherer Aufwand. Sie könnte aber irgendwann über eine URI à la

https://swarm.hiveeyes.org/item/hiveeyes/3b57fcef938b27846d262c83c0f574f8015b5bd9.html

zur Verfügung gestellt werden und damit die Möglichkeit bieten, Audiosamples komfortabel über eine Webseite hochladen und die Analyseergebnisse schöner darstellen zu können.

4

Coole Idee Clemens, dann machen viele mit, es gibt viele Daten zu korrelieren und das verbessert die Analyse. Allerdings werden auch neue Themen auftauchen, nicht-technische Benutzer können uns ganz schön fordern, verschiedene Endgeräte, verschiedene Apps, also, es ist auch eine Herausforderung die Platform zu öffnen, Grüsse, Markus

5

2 posts were split to a new topic: What actually is the “http” command used all over the place?

6

Dear Andreas,

I sometime use inotify-wait (inotify-tools), watching change in a directory.

It allow to launch a script, using filename as argument when a file is created.

We use it to allow some privilege users to upload (sftp) xml files in order to import products data from PIM to eCommerce platform and distinguish web server process to import one.

The sound file upload process can be handle by webserver, then inotify could launch the analyze, storing result somewhere and make a report.

7

Dear Jody,

Thanks for your suggestion!

We are aware of inotify, but we will integrate the code of “audiohealth” directly into Kotori, as both are written in Python and Kotori already brings in all relevant dependencies (you know about the hassle regarding installation of SciPy and NumPy, right?).

By doing so, there’s no need to decouple the processing steps using an external tool. The heavy processing step of audio analysis will be decoupled from the web server request/response cycle by running it asynchronously in a separate thread. The thread pool management will be controlled by Twisted and thimble, which we already use for performing MQTT message processing using a dedicated thread pool, a change we introduced to improve concurrency behavior the other day.

On the other hand, using inotify could be an interesting option for improving the standalone operation of “audiohealth” itself, which will be here to stay besides being introduced as an additional dependency to Kotori. We are happy to receive pull requests on GitHub (even when it’s only about documentation) for a use case where incoming audio samples will be handed over to the program invocation of audiohealth by a cool one-liner using inotify. Go ahead!

With kind regards,
Andreas.

8

Dear Andreas,

very nice, easier than having a crontab to check/launch the screen running inotify-wait script ! :-)

Thank you for sharing, I am learning a lot <3

9

I won’t say it’s easier, but we have platform-gradeness as an overall goal. To get there, we aim at an integrated, robust solution like the other infrastructure parts we already run in production. Amongst other things, this is possible using the powerful Twisted framework offering all tooling required for writing such a piece of software, which contains multiple services and provides primitives for both multithreaded and asynchronous programming.

You are welcome, just keep on asking. We are always happy to share more insights about the overall design process of our infrastructure.

10

Introduction

@einsiedlerkrebs and @andreas discussed some possibilities about which infrastructure components to add to our software stack to fulfill the requirements regarding media file and metadata storage, archival and retrieval. On top of that, we would be able to build the “Audioanalyse” web service as well as a general “Stockkarte” subsystem and more without having to reinvent the wheel in too many details.

Proposal

We want to use Kinto, a generic JSON document store with sharing and synchronisation capabilities. For getting more into the details, please have a look at its documentation, especially the overview page. Kinto is backed by Mozilla, where it is used in production already:

Kinto is used at Mozilla and released under the Apache v2 licence.

At Mozilla, Kinto is used in Firefox for global synchronization of frequently changed settings like blocklists, and the Web Extensions storage.sync API. It is also used in Firefox for Android for A/B testing and delivering extra assets like fonts or hyphenation dictionaries.

We have been following the development of Kinto from its roots in Cornice and Cliquet since 2015 and are reasonably convinced it suits our needs very well. We already used Cornice and Pyramid in some web application development projects in the past, so we are familiar with these infrastructure components and their subsystems under the hood.

Example media file upload API

As you can see from some bits of documentation, the media file upload API is not far away from our proposed API specification above:

Original specification

http \
    --form POST \
    https://swarm.hiveeyes.org/api/hiveeyes/testdrive/area-42/node-1/item \
    audio@${wavfile} time=${timestamp} has_queen=yes

Kinto attachment API

http \
    --auth alice:passwd \
    --form POST \
    http://localhost:8888/v1/buckets/website/collections/assets/records/c2ce1975-0e52-4b2f-a5db-80166aeca689/attachment \
    data='{"type": "wallpaper", "theme": "orange"}' \
    "attachment@~/Pictures/background.jpg"

Example applications

To see what others did using Kinto, please enjoy discovering:

References

We collected some software components (also beyond Kinto) which could be used to implement the desired functionality…

Backend

Archival / Upload

Command line

Frontend

Retrieval

11

Seems you both dropped the Telemeta idea? ;)

12

Dear @weef,

thanks for asking!

Introduction

You are referring to this post …

… right?

Outcome

We actually evaluated Telemeta, installed an instance, uploaded and annotated some audio data (see https://telemeta.hiveeyes.org/archives/corpus/). The “Marker” feature to annotate audio samples interactively is pretty cool:

image
image364×618 103 KB

To try it on your own, see https://telemeta.hiveeyes.org/archives/items/tim_williams_swarm_6_001/.

Revision

But we both like the idea of API-first driven development, built on top of modern software components. While Telemeta was my favorite candidate before, i like the discrete approach better now, which also better fits our overall system design philosophy. And @einsiedlerkrebs also convinced me somehow… :-).

The whole Telemeta system is way more heavy than the lightweight API-only approach using Kinto. Maybe we will embed the TimeSide component into the user interface after nailing down the API. In fact, this was my favorite component of Telemeta and already bundles all the audio-processing and -analysis stuff.

We are happy to hear your opinion and whether you have any objections regarding the current proposal. As i really like some specific features from Telemeta/TimeSide, it’s not completely from the table yet.

13

OSBH is suggesting the same to its peers to gather more audio samples:
https://community.akerkits.com/t/tutorial-recording-hive-audio-with-a-smart-phone/506

14

I have replied on the OSBH posting and made suggestions how to improve the upload and bundle it with a first analysis, so beekeeper get some “incentives” for providing data to the project:

Thanks for uploading your sound file to improve OSBH

OSBH Analysis Output

File:     samples/colony-with-queen-gruber.wav.dat
Strategy: dt-1.0

==================
Sequence of states
==================
active, active, active

===================
Compressed timeline
===================
  0s -  30s   active          ===

==============
Total duration
==============
        30s   active          ===

Your Sound as Visualization:

It looks really great!

The Power Spectrum:

You have peaks around 100 Hz and 250 Hz!

Some Statistics:

Samples read:           3432960
Length (seconds):     38.922449
Scaled by rms:         0.002571
Maximum amplitude:     7.352207
Minimum amplitude:    -5.996453
Midline amplitude:     0.677877
Mean    norm:          0.770147
Mean    amplitude:     0.020792
RMS     amplitude:     1.000000
Maximum delta:         4.270675
Minimum delta:         0.000000
Mean    delta:         0.172402
RMS     delta:         0.230643
Rough   frequency:         1618
Volume adjustment:       52.910
15

Just for the records: We just stumbled across “Django Filer”, which could also be a viable alternative for the file management part of the audio archival and -analysis web service. See its documentation, source code and an associated tutorial.

However, adding metadata fields to media items obviously requires changes to the source code including a database migration step. This is why we still would prefer a solution based on Kinto, which promises hassle-free, self-service operations offering the possibility to let the database schema being collaboratively evolved by the community in the spirit of a Folksonomy.

16

Have you seen the buzzbox announcement, they have a stand alone audio analyzing app now:

from the newsletter:

Download Our Free App
Tap the buttons below to download the OSBeehives app and analyse hive health from your phone, even without a BuzzBox sensor!

That’s really fun because we made some proposal – see this thread here ;-) – to realize this a good half year ago! Great news @Aaron and @Tristan_OSBH!!

17

… I was about to complain, that there is no (new) code on github, again.
But no need for that, since the name of this project company is silently changing:

Internet Archive's Wayback Machine

Loading...

https://www.opensourcebeehives.com |
	    15:56:39 March 19, 2018

Got an HTTP 301 response at crawl time

Redirecting to...

https://www.osbeehives.com/
18

Yes, a bit odd this naming inflation
Open Source Beehives > akerkits > now osbeehives

It’s a pity that not only the names are changing but really interesting and good content in the discussion boards is gone!

19

Some things fortunately have been saved from the Internet Archive, enjoy https://community.osbh.mirror.hiveeyes.org/.

Yeah, looks like OSBH is going commercial. That’s not a bad thing on its own, but it’s really sad that there’s no open source software released by them any more. Maybe they also switched the GitHub organization and just didn’t announce it yet?

Do you have any news, @Aaron or @Tristan_OSBH? We would really feel sad if we would lose you to corporate completely!

1 Like
20

I checked:
https://github.com/osbeehives
is still available.

21

Hey guys,

Let me start by saying this could be a really long conversation, and so I will skim.

Having worked in open source hardware communities of varying flavours for years now, I supposed I have become somewhat jaded by what I see as users generally taking a lot without giving much (if anything) back. By this I mean people downloading files you work hard to produce, often critisising them (and you) feircly when they discover some kind of error or fault, and never (not once in my experience) improving these creations and sharing them back. Beyond this, there are so many “commitments” from people to work on things, and almost zero follow through.

Even this community for example was interested in recieving an audio recorder at some point to help gather data. There wasn’t follow through, and thus energy wasted once again. After a while this becomes exhausting.

But this is all ok if you’re working on a project for fun, as a hobby (as I suspect you guys are? Correct me if I am wrong). But when this is your full time job, and you are giving away so much, and reciving so little in return, let alone financial stability, the open source philosophy becomes an empty concept, one that causes it’s creators to suffer, rather than thrive. One cannot survive on a ideology.

So yes, we have not open sourced the classification algorythm. Why? Because there are a growing number of companies doing this, who will be eager to take our work, close it, build on it, sell it, and give us nothing in return. We cannot continue to work in this way, how could we? We have families and bills.

Still, all the beehives remain open source, and we will soon be releasing the BuzzBoard, which is an os version of BuzzBox onto which you can add your own sensors. The data will be open as a user opt-in. We have not gone “completely corporate”.

In conclusion, I hope you guys can appreciate that this wasn’t a desicion we took lightly. It was one built on years of experience, and coming to terms with the reality of living in a capitalistic world. Perhaps an OS economy is still possible, and projects like ours were originally, will be able to thrive. But in order for that to work, people need to change their attitudes and commitment levels towards the cause.

1 Like
22

A post was split to a new topic: OSBH Pesticide Research