Hi again,
Saraswati is growing up. Version 0.3.0 brings in the following improvements, partly based on different kinds of suggestions from the community. Thanks a stack and kudos to @weef, @roh, @clemens and @Diren.
Features
This is a more detailed overview of the recently added features summarized within the change log at saraswati/CHANGES.rst at 0.3.0 · hiveeyes/saraswati · GitHub.
Run recorder subsystem in separate thread
This was needed to be able to integrate the uploader subsystem (see below) using another thread.
Use 5 minutes recording chunk size as default
It can be adjusted be using the --chunk-duration= option, which takes an integer value in seconds.
Add uploader based on rsync
When the --upload= option is given, Saraswati will attempt to upload its spool directory to an rsync target. By default, it will do this each 5 minutes, which can be adjusted by using the --upload-interval= option.
# Record a single channel and upload via rsync.
saraswati record \
--channel="testdrive source=autoaudiosrc" \
--upload="rsync://foobar@daq.example.org:/var/spool/saraswati/testdrive/wp0Kel53aw/area-42/audionode-01"
Please note: By using rsync’s --remove-source-files option under the hood, the spooled files on the local machine will get purged after a successful upload.
Further parameters taken into account here, are, in SaraswatiUploader:
-
PICKUP_AGE_THRESHOLD = 25
This makes sure no files will be touched which are still actively written to by the recorder subsystem. Only after the last modification time is min. 25 seconds ago, the file will be considered for uploading. This is implemented by using an appropriate find command with an option -not -newermt '-{PICKUP_AGE_THRESHOLD} seconds' operating on Saraswati’s spool directory.
-
BANDWIDTH_MAX = 80
This value will get passed to rsync's --bwlimit= option.
-
IO_TIMEOUT = 15
This value will get passed to rsync's --timeout= option.
Protect against running out of disk space
Disk space availability will be checked regularly. When a minimum threshold value is undershot, the recording will be suspended. After enough disk space is free again, the recording will be resumed. This is implemented by toggling GStreamer’s pipeline state flags between Gst.State.PLAYING and Gst.State.NULL.
By default, disk space checks will run each 60 seconds and will check for a minimum free disk space of 10%. Both values are defined per:
-
SaraswatiRecorder.SERVICE_TASK_INTERVAL = 60 and
-
SaraswatiRecorder.DISK_SPACE_MINIMUM_THRESHOLD = 0.1.
Conclusion
We believe all those features will contribute to make Saraswati ready for use in production-like environments. By trying to implement them in a resilient manner, we are aiming for low maintenance overhead in different situations when operating this software.
We will be happy to hear about any outcome, feel free to create respective issues about bug reports or feature requests at GitHub - hiveeyes/saraswati: Saraswati is a robust, multi-channel audio recording, transmission and storage system.
With kind regards,
Andreas.
/cc @Diren, @tox, @weef, @roh, @MKO