If you read some of my previous articles you know that I’m quite fond of audiobooks. In past I was looking for good media server, which supports audiobooks and ended with airsonic (a subsonic clone). The problem with majority of media servers is that they rely totally on audio tags, which are often messed in audiobooks. Also many of “advanced” functionalities are not applicable to audiobooks (random play, shuffle tracks, moods, etc.) I’m bit of old school so I rely more on reasonable directory structure and do not want to mess with tags for every audiobook I download. I also think that for personal collection I do not need likes, favorites, sharing and similar stuff, as I can easily remember which books are good and which I have listened or want to listen, but I do need few things, which are usually missing – like bookmarks. Interesting function is to continue listen on a device, when I left on previous device, but since I basically listen only on my mobile phone, it does not seems to be critical. So ideal audiobooks server actually requires much less functionality than today’s media servers provide. As I’m progressing with Rust language I decided two weeks ago to create simple audio streaming server adhering to KISS principle – Keep It Simple, Stupid, – Result of this exercise is an application that provides minimum viable functionalities for streaming and listening of audiobooks – it’s called Audioserve. In this article I’ll show it’s basic design and demo current application in a video.
Required functionality
- Support for common audio formats, especially mp3, opus (I think opus is a great format for audio books), m4b, m4a …
- HTTP audio streaming
- SSL/TLS support
- Some simple authentication mechanism, no need for users etc., as this is a personal server, but something to prevent unauthorized access
- Provide transcoding to opus to save bandwith and to support more formats
- Browse audiobooks as they are stored – e.g. in their directory structure, order alphabetically according to file/dictionary names
- Support some additional metadata on folder level – cover image and audiobook/author description
- Some simple search
- Prefer audiobooks split to chapters, but should be able also to play long files (several hours)
- Play whole book – automatically start next chapter after current chapter is finished
- Quick and convenient seek – quickly find required part in audio file – even for large files
- Remember last position, to continue with it listening later, optionally bookmarks to different positions in different audiobooks
- Simple web based client, possibly native mobile (Android) client for better cashing, off-line playback
- Possibility to cache ahead chapters or even whole books – so one can survive connectivity loss or even listen off-line
- Small, lean and fast
Implementation
Audioserve is implemeted in Rust language, it’s using the hyper library for HTTP support, but no complex framework to make it lean. Audioserve provides very simple JSON API to browse the collection and functions to serve audio files directly or transcoded (external ffmpeg utility is required). Authentication is achieved through shared secret phrase. Audioserve also contains bundled web client for modern browsers (latest Firefox or Chrome).
From above requirements reliable caching now implemented only in Android client. In web clients it depends now solely on HTML5 media player, Firefox seems to provide better caching then Chrome. Reliable caching will require deep dive into advanced browser APIs ( Service Workers and/or Media Source Extensions), for which I did not have time yet. Also generic bookmarks are available only in Android client (thought it should not be difficult to implement in Web), only one last played position is remembered.
Demo
UPDATE: And also now Android client is now available you can check in this article.
Installation
Easiest way how to try Audioserve is to use provided Dockerfile and run Docker container. See README in Audioserve repository.
For time being for can also check online live demo.
Currently Audioserve is tested only on Linux.
This is great! I had to mess around with rust versions on Ubuntu 16.04 (the default 1.21 didn’t seem to work, and had to move to the nightly version using the rustup.sh shell script), and then had a few issues with npm and the version of node.js there too.
I did try and use the docker image but that had problems too.
Anyway, finally worked and works very well; matches my requirements for audiobooks perfectly, being folder-based sorting rather than metadata based. It works less well on Android which seems to have very aggressive power-saving features or similar which often result in server connection errors half way through, and a loss of the current time position when refreshing the page (could the current value be written back every few seconds?).
Longer term I guess the Android client will fix the caching issues (can’t wait!)
Just from a feedback point of view, I altered the transcoding values to add a few more and lower the quality down to suit a cell-connected mobile phone. I found that even at 8kb/s bit rate and compression level of 4, opus audio is still pretty acceptable.
A great feature would be to add a transcoding selector wthin the gui, maybe saving selection to a cookie or the like; that way you could tailor the settings to mobile, pc or other device individually.
Great work – thanks again!
Andy
Thanks for feedback.
For instalation problems – if you have some details please post issue on github.
(could the current value be written back every few seconds?) – that’s is what should be happening in web client now – as player updates play time it’s stored to localStorage of the browser and then used to seek position in file after reload.
A great feature would be to add a transcoding selector wthin the gui – I was thinking in similar directions – rather then selecting/enforcing transcoding in server, it should provide all possible transcodings (on different URLs) and client can choose. I hope I can implement something like this in future.
Longer term I guess the Android client will fix the caching issues I started to work on native Android client ( written in Kotlin – very early version is on github) – but Android development is more complex and boring then I expected – so it might take time.
For browser client I was looking into service-workers, which might provide some caching functionality.
Thanks for the feedback. Sounds like some good features on the way!
Largely now, due to the reduction of the transcoded filesize the caching in the firefox browser is good enough for a 20 minute mp3 for me.
Is there any way of allowing the use of a virtual directory? I’m successfully using a different port fronted with HAProxy to deliver from my web server, but I’d ideally like to keep the 443 port and direct to a virtual directory instead. All I’d need is for the web server to work with http://webserver/audioserve instead of http://webserver/
I tried rewriting the HTTP uri with haproxy and whilst it mostly works the authentication and transcode seeking doesn’t for some reason which spoils the show a bit!
Thanks again, a great program!
Sorry to spam your website, but I finally got the service running as a virtual directory with haproxy. For future reference these are the lines of config which allow the service (which likes by default to be referenced as the “root” site at port 3000, or whatever port you set it up with):
# in the frontend config #
..
acl audioserve path_beg -i /audioserve
acl audioserve_bare path -i /audioserve
# to redirect – accessing via your virtual directory (in this case http://webhost/audioserve) without a trailing slash, fails, so redirect.
http-request redirect code 302 location https://%[req.hdr(host)]:/audioserve/ if audioserve_bare
use_backend bk_audioserve_3000 if audioserve
..
# in the backend config
..
# again, assuming your virtual directory access is via /audioserve/
http-request set-var(req.filtereduri) capture.req.uri,regsub(/audioserve[/]?,/,g)
http-request set-path https://%[req.hdr(host)]%[var(req.filtereduri),regsub(//,/,g)]%[query]
..
Thanks, a small upgrade, but i can now use it where providers have filtered the non standard port I was using on the “outside” of my server.
Kind Regards
Andy
NP,
I was already using nginx as reverse proxy, where it worked without issues so I assumed haproxy should be able to do the same, this is part of my nginx config:
location /audioserve/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header Host $http_host;
proxy_max_temp_file_size 0;
proxy_pass http://127.0.0.1:3000/;
proxy_redirect http:// https://;
}
Also the android client is at pre-alpha stage – I’m already using it to listen to audio books now to test usability, however many things are now still missing and – namely media caching (only some basic caching in currently played item through built in exoplayer cache). This means for instance that transcoded stream cannot be sought. Also search, recently played list and bookmarks are missing – only one book is played and if you leave directory, position is lost. Android development is bit more complex so I spent a lot time on things like fragment lifetime just be be able to display current directory right after phone rotation, activity close and reopen and after back action. And it’s all quite fragile, so it can crash frequently. Code is here https://github.com/izderadicka/audioserve-android
This is such a cool project, I got it up and running in docker last night. I think the simplicity of it is an excellent decision. With audiobooks I don’t want tons of features, I mostly want to just open the app and click play.
Android app also seems simple but, efficient and responsive. My only suggestions on the Android app would be to have it pause on headphones/Bluetooth disconnect, and to pause the audio on notifications/Google maps instructions instead of just lowering volume. Overall, I think this is a great option for personal audiobooks, thanks for sharing it.
Thanks for nice words.
For android – normally app should pause and then resume when other app uses audio output, for sure it works for calls and other audio players. I think there are different levels of audio focus, so this might be the case. For pause on disconnect it’s bit complicated, especially when BT gets into picture, right now there is no such functionality in the app. I’d recommended to add issues to github for each.
These a probably just for the android side of things, but
I have some input to design and so called features.
Sloppy fingers like mine has an easy way of pressing skip when I intend to play/pause. I would rather enter the app to lock/unlock these buttons as when listening to audiobooks I rather just play/pause and change volume from time to time.
I seriously need something to remember what I have listened to. Sometimes a new book in a series comes out and I have no idea after a while, what I have listened to and what not. Make a file that follows the directory structure. Set an option for marked as read?
I maaaay have some books that is (much)larger than 250mb and I would rather listen to them with one app instead of using several, to the security aspect of this? 250vs1000 is like…
Yes incidental touches are also problem for me. Not much on buttons but on list items – probably if I’ll start from scratch I’ll think of another design for now.
Agree mark as read would be useful, but it’ll require more considerations how to implement (when and how to mark as read, how to store, directory vs single file ebook, write access to collection, …)
Last paragraph I do not comprehend – it’s no problem to have 1 or 2 GB sized audiobooks – I do not remember about some limitations – only if folder has too many audio files (>200) it’s slow.
For size limitation I just read on the android client that it was limited to 250mb and that would collaborate with the fact that without verbosing or debugging I was not able to open a file of 600mb
Marked as read; is a bit of a thoughtexperiment. I had a lot of text written here, but nothing you probably hadn’t thought about and to much text to finish. Either way, loads of decisions with pros and cons for any option. Short: I rather like the old school way of a hidden file everywhere. For safety, a single file is probably the only sane way.
Oh yes, you right it’s android client limitation only (I was just thinking about server itself). You can use chapters on server side to play large audiofiles (see https://github.com/izderadicka/audioserve#single-file-audiobooks-and-chapters).
I do have strong preference for smaller files -ideally file per chapter.
Marked as read – the is couple of options – hidden file per directory, separate file or client only. All has some pros and cons. But for each to work it’ll require some work on client, which I’m not sure I’ll be willing to do, android programing is not much enjoyable (at east for me).