ASR Quartznet cannot be compiled to OpenVINO IR format in NPU

[cheahber]

OpenVINO Version
2024.4.0

Operating System
Ubuntu 22.04

Processor
Intel® Core™ Ultra 7 155H × 22

Graphic
Mesa Intel® Arc(tm) Graphics (MTL)

Memory
32 GB

Device used for inference
NPU

NPU driver version
Linux NPU Driver v1.8.0 (Tried v1.6.0 as well)

Issue description

I am running the demo sample: https://github.com/openvinotoolkit/open_model_zoo/blob/master/demos/speech_recognition_quartznet_demo/python/README.md

CPU and GPU mode are working as expected. However, it's not working in NPU.

Step-by-step reproduction

1. Follow the steps in README.md to perform (a) Download and convert the Quartznet model. (b) Download the sample audio file.
2. Modify line 109 in speech_recognition_quartznet_demo.py, change it to NPU
3. Run the command:

python3 speech_recognition_quartznet_demo.py -m public/quartznet-15x5-en/FP32/quartznet-15x5-en.xml -i how_are_you_doing_today.wav

Relevant log output
Snippet: -

[ INFO ] OpenVINO Runtime
[ INFO ] 	build: 2024.4.0-16579-c3152d32c9c-releases/2024/4
[ INFO ] Reading model public/quartznet-15x5-en/FP32/quartznet-15x5-en.xml
Traceback (most recent call last):
  File "/home/intel/Desktop/open_model_zoo/demos/speech_recognition_quartznet_demo/python/speech_recognition_quartznet_demo.py", line 143, in <module>
    sys.exit(main() or 0)
  File "/home/intel/Desktop/open_model_zoo/demos/speech_recognition_quartznet_demo/python/speech_recognition_quartznet_demo.py", line 134, in main
    quartz_net = QuartzNet(core, args.model, log_melspectrum.shape, args.device)
  File "/home/intel/Desktop/open_model_zoo/demos/speech_recognition_quartznet_demo/python/speech_recognition_quartznet_demo.py", line 66, in __init__
    compiled_model = core.compile_model(model, device)
  File "/home/intel/.local/lib/python3.10/site-packages/openvino/runtime/ie_api.py", line 543, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:104:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:697:
Exception from src/plugins/intel_npu/src/plugin/src/compiled_model.cpp:62:
Exception from src/plugins/intel_npu/src/compiler/src/zero_compiler_in_driver.cpp:848:
L0 pfnCreate2 result: ZE_RESULT_ERROR_UNKNOWN, code 0x7ffffffe

[avitial]

@cheahber sorry for the delay, we are able to reproduce the issue. Have captured this as a bug, will share more details soon.

Ref. 154772