Issues Running Video2X with RealESRGAN Models on Ubuntu 22.04 (CUDA 12.1.1)

[maauso]

I'm attempting to run Video2X on an Ubuntu 22.04 container with CUDA 12.1.1 in a to upscale batches of old videos.
However, I'm encountering errors when trying to use specific RealESRGAN models. Below are the setup details, steps taken, and errors encountered.

System Details:

Operating System: Ubuntu 22.04
CUDA Version: 12.1.1
Container Image: runpod/pytorch:2.1.1-py3.10-cuda12.1.1-devel-ubuntu22.04
Video2X Models: realesrgan-plus and realesr-animevideov3

Setup Steps:

Start a Docker container with GPU support:

docker run --runtime=nvidia --gpus all --entrypoint /bin/bash -it runpod/pytorch:2.1.1-py3.10-cuda12.1.1-devel-ubuntu22.04

Clone the Video2X repository with submodules:

git clone --recurse-submodules https://github.com/k4yt3x/video2x.git

Run the installation for Ubuntu:

make ubuntu

Errors Encountered:

Using realesrgan-plus model:

build/video2x -i /tmp/TestVideo.mkv -o test.mkv -f realesrgan -r 2 -m realesrgan-plus

[2024-11-04 15:35:01.018] [info] Press SPACE to pause/resume, 'q' to abort.
[2024-11-04 15:35:01.022] [info] Output video dimensions: 1440x1152
[libx264 @ 0x7f06ec036dc0] using SAR=12/11
[libx264 @ 0x7f06ec036dc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f06ec036dc0] profile High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x7f06ec036dc0] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=5 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=36 lookahead_threads=6 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[2024-11-04 15:35:01.046] [error] RealESRGAN model param file not found: models/realesrgan/realesrgan-plus-x2.param
[2024-11-04 15:35:01.046] [error] Failed to initialize filter
[2024-11-04 15:35:01.120] [error] Video processing failed with error code -

build/video2x -i /tmp/TestVideo.mkv -o test.mkv -f realesrgan -r 4 -m realesr-animevideov3

[2024-11-04 15:37:51.945] [info] Press SPACE to pause/resume, 'q' to abort.
[2024-11-04 15:37:51.969] [info] Output video dimensions: 2880x2304
[libx264 @ 0x7f72f0036dc0] using SAR=12/11
[libx264 @ 0x7f72f0036dc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f72f0036dc0] profile High, level 5.1, 4:2:0, 8-bit
[libx264 @ 0x7f72f0036dc0] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=5 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=36 lookahead_threads=6 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
vkCreateInstance failed -9
vkCreateInstance failed -9
vkCreateInstance failed -9
Segmentation fault (core dumped)

Details:

• The first error suggests that the realesrgan-plus model file (realesrgan-plus-x2.param) is missing from the models/realesrgan directory.
• The second error with realesr-animevideov3 seems related to Vulkan (vkCreateInstance failed -9) and ultimately causes a segmentation fault.

Thank you for your assistance!

[k4yt3x]

build/video2x -i /tmp/TestVideo.mkv -o test.mkv -f realesrgan -r 2 -m realesrgan-plus

For this, realesrgan-plus only has a x4 model.

build/video2x -i /tmp/TestVideo.mkv -o test.mkv -f realesrgan -r 4 -m realesr-animevideov3
vkCreateInstance failed -9

You might need to install nvidia-docker2.

[k4yt3x]

Btw have you tried the official container?

[maauso]

For this, realesrgan-plus only has a x4 model.

Does this mean I need to use -r 4? It was straightforward to understand which flags to use to improve the quality of real-life videos.

You might need to install nvidia-docker2.

It's installed and nvidia-smi is available into the container.

Btw have you tried the official container?

Yes, it works like a charm! The issue is that I want to run Video2X on cloud providers like Runpod, where Arch Linux isn't highly compatible.

So, the path I'm aiming for is to try to have it up and running on Ubuntu.

[k4yt3x]

Does this mean I need to use -r 4?

Yes. I may need to add more checks for the CLI. This is made more clear on the GUI for now.

The issue is that I want to run Video2X on cloud providers like Runpod, where Arch Linux isn't highly compatible.

I can't quite support custom containers on arbitrary systems. Since I'm taking care of the whole project all by myself, I just don't have time to ensure it's available for every distro. However, I'll be releasing deb packages soon for Ubuntu 24.04, and I can improve the compatibility of the container by switching to NVIDIA's vulkan image or something.

[k4yt3x]

Try 6.1.0's Ubuntu deb maybe. Also take a look at the official docker to see if there's anything you're missing.

The new version allows you to list and select the GPU for both RealESRGAN and libplacebo. It might come in handy.

[maauso]

Version 6.10 on an Ubuntu Docker worked great! Thanks.

However, I think the process isn't using the GPU. There's a similar case in issue #1205.
I'll update some information there to avoid duplicating the topic across multiple threads.

And this one can be closed thanks @k4yt3x !