Encore's Provider Library, internally referred to as "providerlib", is the library allowing third-party developers to create music sources ("providers") and sound processing ("DSP") plug-ins for Encore.
Plug-ins can be made in either full Java, or with 99% of native (C++) code. Plug-in discovery is made thanks to AndroidManifest attributes, and plug-in initialization is done via AIDL (service interface) calls in Java. Music sources plug-ins will need to implement a full set of AIDL methods and callbacks, while DSP plugins only need a few methods.
Music providers will need to implement the IMusicProvider
AIDL, whereas the DSP will need to
implement IDSPProvider
AIDL.
App can register one or more callbacks to the provider, so it must register them all locally, and when calling a callback method, it must be done on all registered callbacks.
The major part of data transferred through AIDL requires two important information: the plug-in's
ProviderIdentifier
and an entity unique reference (a String
). The former is assigned by
the main application and must be retained by the provider and passed along callbacks, while the
second is assigned by the plug-in to uniquely identify an entity (a song, an artist, an album,
etc). It is strongly recommended to make sure entities references are unique across all providers,
hence why you should have your service's name at the beginning of the reference. A good example
of reference is spotify:album:abcdefghiklmnopq
.
The main app builds itself an entity cache that is cleared when the app is killed. Calls won't
be made to getSong
, getAlbum
, getArtist
, etc if the app has the entity in its cache.
If the entity might have changed, it is up to the provider to call relevant callbacks to notify
the app that the entity has changed (using onSongUpdated
, onArtistUpdated
, etc).
It is strongly recommended to make your service return as soon as possible, updating the entity
later via callbacks if needed. If your service needs to make potentially lengthy operations
(network, calculations, heavy loading), it is advised to return an "empty" entity, by only
setting the entity's reference and ProviderIdentifier
, and setting the isLoaded flag to
false using entity.setIsLoaded(false)
. Later, when the data has been fetched in a separate
thread, you can call the relevant method (onSongUpdated
, etc) on the registered callbacks to
notify of the full entity data.
Important note: When fetching a set of tracks, for example a playlist, the app will request the track information using getSong() if the track isn't already known. If the track is known, but not loaded (isLoaded() returns false and setIsLoaded(true) hasn't been called), the app won't request getSong again and relies on onSongUpdate() callback.
The actual audio data is transferred through a local domain UNIX socket, which is assigned by the
main application when it initializes plug-ins. It is then up to the plug-in to transfer data using
our Java classes or our C++ classes (see src/main/jni/nativesocket
or
src/main/java/com/fastbootmobile/encore/providers
for C++ and Java code).
In the case of a music provider, the audio must be decoded (PCM) by the provider itself and transmitted to the main app using the socket. Here's the typical flow for music playback:
- The main app initializes the plug-in on boot, calling
setAudioSocketName
with the audio socket name to use. The provider will initialize anAudioClientSocket
(Java) orSocketClient
(C++) with the provided name. These classes will keep the connection open and idle. - The main app calls
playSong(String ref)
on the provider, which will validate the URI and return true. Considering the positive return code, the app will set itself in "buffering" state. - The provider will initialize decoding and will start fetching the requested track, then start decoding samples.
- When the first samples are decoded and ready to be sent to the main app, the provider must call
the
onSongPlaying
callback to notify buffering is done and playback will start. The app will then hide the buffering animation and consider music is playing. - On the first played samples, you must send the format information using
AudioSocket.writeFormatInfo
/SocketClient::writeFormatInfo
. By default, the app will setup a 44.1kHz, 16 bits, stereo playback engine, so if your sample rate changes, you must notify the app using this call. - The decoded samples must then be sent to the main app using the audio socket. If you're using
Java, using
AudioSocket.writeAudioData(short[], int)
, or in C++SocketClient::writeAudioData(uint8_t*, int)
. Once the samples have been processed, the app will reply with the number of bytes written. Zero bytes written indicates the audio buffers are full and the plug-in must retry to send the same samples later. Otherwise, the number of bytes written are returned. In Java, the response is asynchronous and returned inAudioSocket.ISocketCallback.onAudioResponse
, so it is up to you to implement proper locking. In C++, you can either get the response asynchronously in yourSocketCallbacks
implementation, or synchronously by settingwriteAudioData
's last parameter totrue
which will make the method return the number of bytes replied by the main app. - When the track is done (e.g. when EOF has been seen), the provider will call
onTrackEnded
to notify the app that the next track can be played.
Note that in the current implementation, the audio engine will only queue either the full sent audio data array or nothing. This means that the number of bytes returned by a
writeAudioData
call will either return the same number of bytes that were sent, or zero. It is however possible that in the future, audio data may be split and only part of the transmitted audio data will be stored in the audio playback engine buffers. It is recommended to implement support for this behavior.
DSPs work in a similar fashion to music providers, to the difference they're not decoding or
generating music. When the user puts your DSP plugin in its processing chain, the app will
automatically feed you with the audio data coming either from the provider, or the previous plugin
in the chain. No action is required in either Java or C++, as long as the DSP opened the connection
to the socket. This data will arrive in the onAudioData
callback. The provider must then process
the sample, and return them to the app by calling writeAudioData
, just like a music provider.
DSPs however don't need to care about the number of bytes written, as space is guaranteed by the
app if you got audio data.
Encore supports fetching album art from various web sources (MusicBrainz, Google Images, etc),
but these services might sometimes be slower than a local album art, or an album art provided
directly by the service. On calls to getSongArt
, getAlbumArt
, getArtistArt
or
getPlaylistArt
, providers may return true or false depending on their capability to provide an
image for the specified entity. When returning true, the app won't fetch the art itself, and will
wait on the provider to provide an image.
All illustrations are cached by the main app.
The app will wait up to 6 seconds for the art to be downloaded until cancelling the load to free up processing time for other illustrations. If the provider returns an art after the timeout is expired, it will still be cached by the app and will show up on the next display request (e.g. after scrolling down and back up, or when closing and reopening a view).
Encore's main app gather some information from the EchoNest API, which understands services' URIs directly in their database.
If you want to enable support for Rosetta-stone in your provider and offer the user additional services (similar artists, and Automix support), you must:
- Make sure your references are the actual service's URIs (e.g. a Spotify provider would use Spotify's URIs as references)
- Make sure you return the Rosetta-stone ID space in
getSupportedRosettaStonePrefixes()
API method (see the list of ID spaces)
The providerlib comes with two parts: a native (C++) part, and a Java part. Plug-ins can be made in both full Java, or fully native code.
- The Java part contains multiple types of files:
- The AIDL files allows for main service calls (get songs information, get playlists, etc) and app callbacks methods (playback started, track ended, etc).
- The Java files contains two packages, a
model
package and aproviders
package- The
model
package contains the shared structures between the main app and the providers (Song
,Album
,Artist
,Playlist
andSearchResult
). All of these classes inherit from the common classBoundEntity
which commonizes the reference parameter as well as the provider identifier. When sending data to the main app, you MUST use these classes. - The
providers
package, which contains the constants and the audio socket classes if you wish to handle the audio data purely in Java. - There is a separate
Encore.Plugin
class which contains generated code for the socket's protobuf protocol. You should not need to deal with this class directly, besides the fields used inISocketCallbacks
.
- The
- The JNI folder contains the native implementation of the audio socket communication protocol.
In a similar way to Java, there is a
SocketClient
and aSocketCallbacks
to communicate with the main app and get responses.
Feel free to check out the open-sourced providers to get started, as well as the boilerplate AndroidManifest and Plugin Service java files. Here are a few steps to guide you through:
- Create a project and import the
providerlib
module in one of these ways:- copy the folder and add the module to your
settings.gradle
file - or in Android Studio: download the library (using
Download ZIP
), useFile -> New -> Import Module
to import the downloaded folder, go tobuild.gradle
file and add under thedependecies{}
section:compile project( ':encore-providerlib' )
- copy the folder and add the module to your
- Add the required meta-data and actions in your AndroidManifest.xml
(see
com.fastbootmobile.encore.providers.Constants
) - Create a Service and implement either
IMusicProvider
orIDSPProvider
depending on whether you're working on a music provider, or a DSP processor - Create the Audio socket in either Java or Native code in
setAudioSocketName
- Do whatever you need