[Build] Error when trying to build docker image of ONNXRuntime with CUDA support

[banu-narayana-dnjp]

Describe the issue

While running the script, it expects to find "tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt". But, there is no such file in that path.

Urgency

No response

Target platform

CUDA

Build script

git submodule update --init
docker build -t onnxruntime-cuda --build-arg CUDA_VERSION=12.6.1
--build-arg CUDNN_VERSION=9.5.0.50
--build-arg GIT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
--build-arg GIT_COMMIT=$(git rev-parse HEAD)
--build-arg ONNXRUNTIME_VERSION=$(cat ../VERSION_NUMBER)
-f Dockerfile.cuda ..

Error / output

=> ERROR [stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip ins 3.9s

[stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=61;70;75;80;86;90" onnxruntime_BUILD_UNIT_TESTS=OFF:
2.365 Collecting psutil
2.531 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (22 kB)
2.689 Collecting setuptools
2.706 Downloading setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB)
2.763 Collecting wheel
2.781 Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB)
2.825 Collecting packaging
2.843 Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
2.897 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (287 kB)
2.934 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 287.3/287.3 kB 8.1 MB/s eta 0:00:00
2.953 Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB)
3.040 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 14.7 MB/s eta 0:00:00
3.058 Downloading wheel-0.45.1-py3-none-any.whl (72 kB)
3.068 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 10.4 MB/s eta 0:00:00
3.087 Downloading packaging-24.2-py3-none-any.whl (65 kB)
3.094 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 13.3 MB/s eta 0:00:00
3.146 Installing collected packages: wheel, setuptools, psutil, packaging
3.673 Successfully installed packaging-24.2 psutil-6.1.0 setuptools-75.6.0 wheel-0.45.1
3.866 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt'

---

Dockerfile.cuda:47

46 | # Create a virtual environment and install dependencies, then build ONNX Runtime with CUDA support.
47 | >>> RUN cd /code
48 | >>> && python3 -m venv /code/env
49 | >>> && . /code/env/bin/activate
50 | >>> && pip install --upgrade psutil setuptools wheel packaging
51 | >>> && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt
52 | >>> # && pip install -r /home/banu_narayana/Documents/onnxruntime/tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt
53 | >>> && python /code/tools/ci_build/build.py --build_dir /code/build/Linux
54 | >>> --allow_running_as_root --skip_submodule_sync
55 | >>> --use_cuda --cuda_home /usr/local/cuda
56 | >>> --cudnn_home /code/build/cudnn
57 | >>> --build_shared_lib --skip_tests
58 | >>> --config Release --build_wheel --update --build --parallel
59 | >>> --cmake_generator Ninja
60 | >>> --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}" onnxruntime_BUILD_UNIT_TESTS=OFF
61 |

ERROR: failed to solve: process "/bin/sh -c cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}" onnxruntime_BUILD_UNIT_TESTS=OFF" did not complete successfully: exit code: 1

Visual Studio Version

No response

GCC / Compiler Version

No response

[fs-eire]

@banu-narayana-dnjp could you help to verify if #22946 fixes this issue?

[banu-narayana-dnjp]

@banu-narayana-dnjp could you help to verify if #22946 fixes this issue?

No, the same error persists even if there is a requirements.txt file.

=> ERROR [stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrad 3.9s

[stage-0 5/5] RUN cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/python/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines onnxruntime_ENABLE_NVTX_PROFILE=ON --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=61;70;75;80;86;90" onnxruntime_BUILD_UNIT_TESTS=OFF:
2.238 Collecting psutil
2.404 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (22 kB)
2.560 Collecting setuptools
2.577 Downloading setuptools-75.6.0-py3-none-any.whl.metadata (6.7 kB)
2.624 Collecting wheel
2.641 Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB)
2.686 Collecting packaging
2.707 Downloading packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
2.763 Downloading psutil-6.1.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (287 kB)
2.800 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 287.3/287.3 kB 8.1 MB/s eta 0:00:00
2.820 Downloading setuptools-75.6.0-py3-none-any.whl (1.2 MB)
2.990 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 7.3 MB/s eta 0:00:00
3.009 Downloading wheel-0.45.1-py3-none-any.whl (72 kB)
3.017 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 15.3 MB/s eta 0:00:00
3.036 Downloading packaging-24.2-py3-none-any.whl (65 kB)
3.043 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.5/65.5 kB 14.4 MB/s eta 0:00:00
3.095 Installing collected packages: wheel, setuptools, psutil, packaging
3.630 Successfully installed packaging-24.2 psutil-6.1.0 setuptools-75.6.0 wheel-0.45.1
3.825 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'tools/ci_build/github/linux/python/requirements.txt'

---

Dockerfile.cuda:47

46 | # Create a virtual environment and install dependencies, then build ONNX Runtime with CUDA support.
47 | >>> RUN cd /code
48 | >>> && python3 -m venv /code/env
49 | >>> && . /code/env/bin/activate
50 | >>> && pip install --upgrade psutil setuptools wheel packaging
51 | >>> # && pip install -r tools/ci_build/github/linux/docker/inference/x86_64/python/cpu/scripts/requirements.txt
52 | >>> && pip install -r tools/ci_build/github/linux/python/requirements.txt
53 | >>> && python /code/tools/ci_build/build.py --build_dir /code/build/Linux
54 | >>> --allow_running_as_root --skip_submodule_sync
55 | >>> --use_cuda --cuda_home /usr/local/cuda
56 | >>> --cudnn_home /code/build/cudnn
57 | >>> --build_shared_lib --skip_tests
58 | >>> --config Release --build_wheel --update --build --parallel
59 | >>> --cmake_generator Ninja
60 | >>> --cmake_extra_defines onnxruntime_ENABLE_NVTX_PROFILE=ON
61 | >>> --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}" onnxruntime_BUILD_UNIT_TESTS=OFF
62 |

ERROR: failed to solve: process "/bin/sh -c cd /code && python3 -m venv /code/env && . /code/env/bin/activate && pip install --upgrade psutil setuptools wheel packaging && pip install -r tools/ci_build/github/linux/python/requirements.txt && python /code/tools/ci_build/build.py --build_dir /code/build/Linux --allow_running_as_root --skip_submodule_sync --use_cuda --cuda_home /usr/local/cuda --cudnn_home /code/build/cudnn --build_shared_lib --skip_tests --config Release --build_wheel --update --build --parallel --cmake_generator Ninja --cmake_extra_defines onnxruntime_ENABLE_NVTX_PROFILE=ON --cmake_extra_defines ONNXRUNTIME_VERSION=$(cat ./VERSION_NUMBER) "CMAKE_CUDA_ARCHITECTURES=${CMAKE_CUDA_ARCHITECTURES}" onnxruntime_BUILD_UNIT_TESTS=OFF" did not complete successfully: exit code: 1

[fs-eire]

3.825 ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'tools/ci_build/github/linux/python/requirements.txt'

This file should exist.

could you try update

pip install -r tools/ci_build/github/linux/python/requirements.txt

to

pip install -r /code/tools/ci_build/github/linux/python/requirements.txt