Importing In Android Studio

[oluwabajio]

Hello,

i would love to use spleeter in one of my android apps (java).

i have successfully implemented Deezer spleeter on my server (aivocalremover.com), but I would like to find out if its possible to bind your C++ spleter on my android java app.

[jinay1991]

Hello @oluwabajio

Thanks for your message and feature request. I have this on my road-map but unfortunately this can not be possible now due to lack of compatibility of Model (ckpt/*.pb) to Android Tensorflow Lite environment. This has been requested to original repository deezer/spleeter#354.

If you can port TensorFlow model somehow to Android App, this repository/library can be ported to Android App via Android NDK

Please keep the above mentioned issue in check to get update on when the Deezer Model will be upgraded/re-trained to support TF v2.x and TFLite. Once that is possible, I will extend this repository to support for TFLite as well which can later be portable to Android/iOS easily.

Thanks
Jinay

[oluwabajio]

Hello, Thanks for this, i would keep tabs till when the models are compatible with tflite. I really do not want to go the NDK route.

[oluwabajio]

Hello,
I am still keeping tabs on this issue.

Spleeter is now compatible with TF2.

Announcement: https://gitter.im/spleeter/community

PR: deezer/spleeter#498

Do you have any idea how this new version can be ported to tflite.

Thanks.

[jinay1991]

Thanks for the update, I have started working on this. Soon I will release with the update and functional TFLite Spleeter Codebase.

I already generated TFLite models with the help and update from gvne/spleeterpp#24 - I need to make some adjustment to my repository in order provie TFLite based inference as it is currently broken.

Already released model versions: https://github.com/jinay1991/spleeter/releases/tag/v2.3.

[oluwabajio]

Thanks for the update, I really appreciate.
Now implementing it with tflite on android, what are the pre-processing I need to do to the input mp3 audio data before feeding it to the tflite interpreter.
Also what post-processing needs to be done

[VVasanth]

Hi @jinay1991 - Thanks for the update...But could you please confirm whether have you generated the TFLite model from the latest release of spleeter, which is compatible with TF2.0 or from the older version of spleeter models?

I would like to understand whether have you generated the TFLite from 'saved model' file or from 'check points' - as the tflite model converted from checkpoints are said to have performance issues (as per 'spleeter-tflite-convert' library). Could you let me know pls?

[jinay1991]

Hi @jinay1991 - Thanks for the update...But could you please confirm whether have you generated the TFLite model from the latest release of spleeter, which is compatible with TF2.0 or from the older version of spleeter models?

I would like to understand whether have you generated the TFLite from 'saved model' file or from 'check points' - as the tflite model converted from checkpoints are said to have performance issues (as per 'spleeter-tflite-convert' library). Could you let me know pls?

Hi @VVasanth I have simply taken the original checkpoints from Spleeter Public Release - which is TF1.x, though I am still waiting for deezer to provide re-trained model with TF2.0 model definitions - (Issue: deezer/spleeter#354)

I haven't tried myself yet for the performance benchmark with native TFLite C++ APIs. (Though you are right, as per the author of "spleeter-tflite-converter" library, it has some performance issues.)

[jinay1991]

Thanks for the update, I really appreciate.
Now implementing it with tflite on android, what are the pre-processing I need to do to the input mp3 audio data before feeding it to the tflite interpreter.

Would you be kind to share your work (Android App) (GitHub Link)?
Pre-processing: (Decode/Demux your audio file into raw data (i.e. plain array of floats) and pass it to waveform:0 tensor of the model)

Also what post-processing needs to be done

Post-processing: None (read output tensors depending on your chosen model, and you will receive raw data (i.e. plain array of floats) which can again Remux/Encode to your choice of container (MP3/MP4))

[oluwabajio]

Ok, I would implement it tomorrow and share the link with you.
Thanks

[jinay1991]

@oluwabajio I have just published a release https://github.com/jinay1991/spleeter/releases/tag/2.3.0 which works fine with TFLite and provided Spleeter Model (derived from TF v1.15 checkpoints)

[jinay1991]

Importing to Android for Bazel Project using NDK is explained here -> https://docs.bazel.build/versions/master/android-ndk.html

[oluwabajio]

Thanks for the update. i would work on it and share my feedbacks. Thanks.

[oluwabajio]

Please @jinay1991 can i communicate with you via mail, i have an issue with conversion of the input audio.

THanks

[plazar99]

I have a multimedia app in development, and have been looking into integrating Spleeter into my android app. I would be happy to help you move this forward, if you need additional help.

[oluwabajio]

Hello @plazar99 can we communicate via mail please. i really need to get spleeter working for my mobile app.

[Semester7]

Hello @plazar99 can we communicate via mail please. i really need to get spleeter working for my mobile app. my email is [email protected]

Hello @oluwabajio Is it possible to give updates on how you are getting spleeter to work on your app? I am also looking for the same. Thank you.

[oluwabajio]

Hello @Semester7 i integrated the tflite model in my android app, pass in the audio data, but i was not successful with it.

You can review my code here and see if i made errors.

[farihachaiti]

Hello @oluwabajio

Thanks for your message and feature request. I have this on my road-map but unfortunately this can not be possible now due to lack of compatibility of Model (ckpt/*.pb) to Android Tensorflow Lite environment. This has been requested to original repository deezer/spleeter#354.

If you can port TensorFlow model somehow to Android App, this repository/library can be ported to Android App via Android NDK

Please keep the above mentioned issue in check to get update on when the Deezer Model will be upgraded/re-trained to support TF v2.x and TFLite. Once that is possible, I will extend this repository to support for TFLite as well which can later be portable to Android/iOS easily.

Thanks Jinay

Hi. Is this possible now? Can spleeter be implemented on Android apps now? What is the update to this issue?

[KangChou]

Hello, can you share the Android source code of the C++ spleeter integration? Because I am currently doing research in this area, thank you for your sharing and guidance.
@oluwabajio
@farihachaiti
@VVasanth
@jinay1991
@plazar99

[jinay1991]

Hello, can you share the Android source code of the C++ spleeter integration? Because I am currently doing research in this area, thank you for your sharing and guidance.
@oluwabajio
@farihachaiti
@VVasanth
@jinay1991
@plazar99

@oluwabajio already provided his code:

"You can review my code here and see if i made errors. Also you can communicate with me at [email protected].

[KangChou]

Hello,

i would love to use spleeter in one of my android apps (java).

i have successfully implemented Deezer spleeter on my server (aivocalremover.com), but I would like to find out if its possible to bind your C++ spleter on my android java app.

My current questions are summarized as follows:

I want to use the spleeter to deploy on the Android side, and I hope that the interface of the spleeter c++ dynamic library can be called. At present, the dynamic library you gave still cannot be called.

What I want to know is the dynamic library you just gave, how can I use the C++ program to call it and realize the separation of the vocal accompaniment of music?

Because I want to deploy it on mobile android, although I have tried the method on android, but it doesn't seem to work.

[oluwabajio]

Hello @KangChou i tried to get the models working on android, but i was not successful with it.

Here is my code.

https://github.com/oluwabajio/Spleeter-Test-Copy/blob/new_update/app/src/main/java/Spleeter/test/MainActivity.java

        float floats[] = music_data_as_float;
        for (int i = 0; i < 1280000-4; i+=2) {
            floats[i/2] = sampleFloatArray[i];
           floats[640000+(i/2)] = sampleFloatArray[i+1];
        }

        int[] probabilityShape =  tflite.getInputTensor(0).shape();
        DataType probabilityDataType =  tflite.getInputTensor(0).dataType();


        TensorBuffer tensorBuffer = TensorBuffer.createDynamic(imageDataType);
       tensorBuffer.loadArray(floats, probabilityShape);
        ByteBuffer inByteBuffer = tensorBuffer.getBuffer();

        Object[] input = new Object[1];
        input[0] = inByteBuffer;

         TensorBuffer outputTensorBufferI =
                TensorBuffer.createFixedSize(probabilityShape, probabilityDataType);

        TensorBuffer outputTensorBufferV =
                TensorBuffer.createFixedSize(probabilityShape, probabilityDataType);

        Map outputs = new TreeMap<>();
        outputs.put(0, outputTensorBufferI.getBuffer());
        outputs.put(1, outputTensorBufferV.getBuffer());

[KangChou]

Hi, has this issue been resolved so far?

[farihachaiti]

Here is mine.
https://github.com/farihachaiti/DoKaraokeAI

[Fr4nKB]

Hi, I'm no expert with tensorflow but I managed to do a relatively simple and working port in kotlin
https://github.com/Fr4nKB/SpleeterAndroidPort/
I post this here in case someone (like me) needs such tool