How to get acceleration on mali-G610 (orange pi 5)

[plord12]

Hi All,

Now that #11019 is resolved, anyone got acceleration working with maili GPU ?

Firstly I found it hard to find a definitive source for libmali ... eventually I used the deb packages at https://github.com/tsukumijima/libmali-rockchip/releases ( I chose libmali-valhall-g610-g6p0-gbm_1.9-1_arm64.deb ).

I updated hwaccel.ml.yml :

services:
  armnn:
    devices:
      - /dev/mali0:/dev/mali0
    volumes:
      - /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
      - /usr/lib/aarch64-linux-gnu/libmali.so.1.9.0:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)

and docker-compose.yml :

  immich-machine-learning:
   container_name: immich_machine_learning
   # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
   # Example tag: ${IMMICH_VERSION:-release}-cuda
   image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-armnn
   extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
     file: hwaccel.ml.yml
     service: armnn # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable

Now when I click on the smart search job I see :

$ docker logs -f immich_machine_learning
...
[07/12/24 14:32:52] DEBUG    Setting model format to armnn                      
[07/12/24 14:32:52] INFO     Loading visual model 'ViT-B-32__openai' to memory  
[07/12/24 14:32:52] DEBUG    Loading visual preprocessing config for CLIP model 
                            'ViT-B-32__openai'                                 
[07/12/24 14:32:52] DEBUG    Loaded visual preprocessing config for CLIP model  
                            'ViT-B-32__openai'                                 
arm_release_ver of this libmali is 'g6p0-01eac0', rk_so_ver is '5'.
[07/12/24 14:32:53] INFO     Loading ANN model                                  
                            /cache/clip/ViT-B-32__openai/visual/model.armnn ...
[07/12/24 14:32:54] INFO     Loaded ANN model with ID 0                         
[07/12/24 14:33:00] DEBUG    Checking for inactivity...                         
[07/12/24 14:33:10] DEBUG    Checking for inactivity...                         
[07/12/24 14:33:20] DEBUG    Checking for inactivity...                         
[07/12/24 14:33:30] DEBUG    Checking for inactivity...                         
[07/12/24 14:33:40] DEBUG    Checking for inactivity...                         
[07/12/24 14:33:50] DEBUG    Checking for inactivity...                         
[07/12/24 14:34:00] DEBUG    Checking for inactivity...                         
[07/12/24 14:34:10] DEBUG    Checking for inactivity...                         
[07/12/24 14:34:20] DEBUG    Checking for inactivity...                         
[07/12/24 14:34:30] DEBUG    Checking for inactivity...                         
[07/12/24 14:34:39] ERROR    Worker (pid:10) was sent code 139!        
...

So it seems libmali is being picked up. But not much info on the failure.

On the host I see :

# dmesg -w
....
[ 1120.001326] mali fb000000.gpu: Power transition timed out unexpectedly
[ 1120.001360] mali fb000000.gpu: 	MCU desired = 1
[ 1120.001378] mali fb000000.gpu: 	MCU sw state = 2
[ 1120.001394] mali fb000000.gpu: Current state :
[ 1120.001415] mali fb000000.gpu: 	Shader=0000000000000000
[ 1120.001433] mali fb000000.gpu: 	Tiler =0000000000000000
[ 1120.001450] mali fb000000.gpu: 	L2    =0000000000000001
[ 1120.001467] mali fb000000.gpu: 	MCU status = 2
[ 1120.001484] mali fb000000.gpu: Cores transitioning :
[ 1120.001502] mali fb000000.gpu: 	Shader=0000000000000000
[ 1120.001520] mali fb000000.gpu: 	Tiler =0000000000000000
[ 1120.001538] mali fb000000.gpu: 	L2    =0000000000000000
[ 1120.001555] mali fb000000.gpu: Sending reset to GPU - all running jobs will be lost
[ 1120.001581] mali fb000000.gpu: Preparing to soft-reset GPU
[ 1120.001610] mali fb000000.gpu: Wait for MCU power on failed on scheduling tick/tock
[ 1120.002221] mali fb000000.gpu: Resetting GPU (allowing up to 500 ms)
[ 1120.002243] mali fb000000.gpu: Register state:
[ 1120.002269] mali fb000000.gpu:   GPU_IRQ_RAWSTAT=0x00000000   GPU_STATUS=0x00000000  MCU_STATUS=0x00000002
[ 1120.002294] mali fb000000.gpu:   JOB_IRQ_RAWSTAT=0x00000000   MMU_IRQ_RAWSTAT=0x00000000   GPU_FAULTSTATUS=0x00000000
[ 1120.002318] mali fb000000.gpu:   GPU_IRQ_MASK=0x00000000   JOB_IRQ_MASK=0x00000000   MMU_IRQ_MASK=0x00000000
[ 1120.002339] mali fb000000.gpu:   PWR_OVERRIDE0=0x00000000   PWR_OVERRIDE1=0x00000000
[ 1120.002360] mali fb000000.gpu:   SHADER_CONFIG=0x00000000   L2_MMU_CONFIG=0x00000000   TILER_CONFIG=0x00000000
[ 1120.002390] mali fb000000.gpu: Firmware trace buffer dump:
[ 1120.002962] mali fb000000.gpu: reloading firmware
[ 1120.003605] mali fb000000.gpu: Reset complete
[ 1122.564515] mali fb000000.gpu: Power transition timed out unexpectedly
[ 1122.564540] mali fb000000.gpu: 	MCU desired = 1
[ 1122.564551] mali fb000000.gpu: 	MCU sw state = 2
[ 1122.564563] mali fb000000.gpu: Current state :
[ 1122.564578] mali fb000000.gpu: 	Shader=0000000000000000
[ 1122.564591] mali fb000000.gpu: 	Tiler =0000000000000000
[ 1122.564602] mali fb000000.gpu: 	L2    =0000000000000001
[ 1122.564614] mali fb000000.gpu: 	MCU status = 2
[ 1122.564624] mali fb000000.gpu: Cores transitioning :
[ 1122.564636] mali fb000000.gpu: 	Shader=0000000000000000
[ 1122.564647] mali fb000000.gpu: 	Tiler =0000000000000000
[ 1122.564661] mali fb000000.gpu: 	L2    =0000000000000000
[ 1122.564671] mali fb000000.gpu: Sending reset to GPU - all running jobs will be lost
[ 1122.564686] mali fb000000.gpu: Preparing to soft-reset GPU
[ 1122.564722] mali fb000000.gpu: Wait for MCU power on failed on scheduling tick/tock
[ 1122.564995] mali fb000000.gpu: Resetting GPU (allowing up to 500 ms)
[ 1122.565008] mali fb000000.gpu: Register state:
[ 1122.565024] mali fb000000.gpu:   GPU_IRQ_RAWSTAT=0x00000000   GPU_STATUS=0x00000000  MCU_STATUS=0x00000002
[ 1122.565039] mali fb000000.gpu:   JOB_IRQ_RAWSTAT=0x00000000   MMU_IRQ_RAWSTAT=0x00000000   GPU_FAULTSTATUS=0x00000000
[ 1122.565054] mali fb000000.gpu:   GPU_IRQ_MASK=0x00000000   JOB_IRQ_MASK=0x00000000   MMU_IRQ_MASK=0x00000000
[ 1122.565068] mali fb000000.gpu:   PWR_OVERRIDE0=0x00000000   PWR_OVERRIDE1=0x00000000
[ 1122.565082] mali fb000000.gpu:   SHADER_CONFIG=0x00000000   L2_MMU_CONFIG=0x00000000   TILER_CONFIG=0x00000000
[ 1122.565100] mali fb000000.gpu: Firmware trace buffer dump:
[ 1122.565320] mali fb000000.gpu: reloading firmware
[ 1122.565932] mali fb000000.gpu: Reset complete

I've no clue what this means.

Thanks!

[smiggiddy]

did you figure this out?

[plord12]

When searching photos I see -

                             FileNotFoundError: Model file not found:           
                             /cache/clip/ViT-B-32__openai/textual/model.armnn   
[08/26/24 07:14:55] WARNING  ARMNN is available, but model 'ViT-B-32__openai'   
                             does not support it.       

I guess this is still pending.

[jacopomaroli]

it seems to work for me:

[09/11/24 17:19:17] INFO     Starting gunicorn 23.0.0                           
[09/11/24 17:19:17] INFO     Listening at: http://[::]:3003 (9)                 
[09/11/24 17:19:17] INFO     Using worker: app.config.CustomUvicornWorker       
[09/11/24 17:19:17] INFO     Booting worker with pid: 10                        
[09/11/24 17:19:23] INFO     Started server process [10]                        
[09/11/24 17:19:24] INFO     Waiting for application startup.                   
[09/11/24 17:19:24] INFO     Created in-memory cache with unloading after 300s  
                             of inactivity.                                     
[09/11/24 17:19:24] INFO     Initialized request thread pool with 8 threads.    
[09/11/24 17:19:24] DEBUG    Checking for inactivity...                         
[09/11/24 17:19:24] INFO     Application startup complete.                      
[09/11/24 17:19:31] DEBUG    Setting model format to armnn                      
[09/11/24 17:19:31] INFO     Loading visual model 'ViT-B-32__openai' to memory  
[09/11/24 17:19:31] DEBUG    Loading visual preprocessing config for CLIP model 
                             'ViT-B-32__openai'                                 
[09/11/24 17:19:31] DEBUG    Loaded visual preprocessing config for CLIP model  
                             'ViT-B-32__openai'                                 
arm_release_ver: g13p0-01eac0, rk_so_ver: 10
[09/11/24 17:19:31] INFO     Loading ANN model                                  
                             /cache/clip/ViT-B-32__openai/visual/model.armnn ...
[09/11/24 17:19:32] INFO     Loaded ANN model with ID 0    

problem is I get load on my CPU and running

sudo watch -n 1 cat /sys/kernel/debug/rknpu/load

or even

sudo watch -n 1 cat /sys/kernel/debug/rkrga/load

shows no activty. how do I confirm it's actually using HW accel?

my setup: orangepi 5 Joshua-Riek ubuntu

orangepi@opi5:~$ uname -r
5.10.0-1012-rockchip
orangepi@opi5:~$ uname -a
Linux opi5 5.10.0-1012-rockchip #12-Ubuntu SMP Wed Aug 14 22:22:22 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
orangepi@opi5:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.4 LTS
Release:        22.04
Codename:       jammy

I downloaded and installed libmali-valhall-g610-g13p0-gbm_1.9-1_arm64.deb from the repo in the first comment.
the only change I had to do was mounting both libmali.so and libmali.so.1 into my container.
this distro also includes a libmali-valhall-g610-g13p0-x11-wayland-gbm.so but mounting that as libmali.so will trigger an infinite amount of other requirements for other *.so files mostly related to X, so I pulled the lean non-wayland library I mentioner earlier

[Mahoo12138]

Look forward to any progress from you.

[LeoRX]

it took some time for me to get things going, but I'm also seeing the "Power transition timed out unexpectedly" issue. :(

forgot to mention I'm running rock pi 5a

[mertalev]

Rockchip boards tend to be sensitive to the power block and cable you use. I had to try like 4 different things before I found something that worked reliably. Check what others have used with success for your model. A "dumb", non-PD charger with the right voltage and amperage is more likely to work well.

[LeoRX]

@mertalev Thank you for your suggestion. Unfortunately I can't find the power supply and cable that suits. tried a few high power rated cable and various power supply, including the Radxa 30w power adapter.

[LeoRX]

Finally! after another few hours of fiddle around and got this working.
I can't comment why "Power transition timed out unexpectedly" happens, but it is most likely due to missing... something. I had cli version of the Radxa OS and ended up install the KDE version, everything started to work afterwards.