Local Demo Issue

[Haozong-Zeng]

Hello, thanks for the great work. I am having issue with running the local demo. I set up the environment with the requirements in this repo and the comfyui one. The script is trying to load json files that do not exist on any version of SD 1.5 from huggingface.

python styleshot_image_driven_demo.py --style "testB/rgb_000000009939.jpg" --content "testA/rgb_000000000291.jpg" --preprocessor "Lineart" --prompt "a photo of two white mice" --output "output/test.jpg"

Traceback (most recent call last):
File "styleshot_image_driven_demo.py", line 70, in
main(args)
File "styleshot_image_driven_demo.py", line 48, in main
pipe = StyleContentStableDiffusionControlNetPipeline.from_pretrained(base_model_path, controlnet=content_fusion_encoder, safety_checker=None)
File "/home/haozong/anaconda3/envs/styleshot/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/home/haozong/anaconda3/envs/styleshot/lib/python3.8/site-packages/diffusers/pipelines/pipeline_utils.py", line 723, in from_pretrained
config_dict = cls.load_config(cached_folder)
File "/home/haozong/anaconda3/envs/styleshot/lib/python3.8/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
File "/home/haozong/anaconda3/envs/styleshot/lib/python3.8/site-packages/diffusers/configuration_utils.py", line 377, in load_config
raise EnvironmentError(
OSError: Error no file named model_index.json found in directory runwayml/stable-diffusion-v1-5/unet.

[Jeoyal]

Hi @Haozong-Zeng , thank you for your interest in our work. You can search for the keyword stable-diffusion-v1-5 to find other accessible repositories on Huggingface (e.g., benjamin-paine/stable-diffusion-v1-5).

[Haozong-Zeng]

Hi @Haozong-Zeng , thank you for your interest in our work. You can search for the keyword stable-diffusion-v1-5 to find other accessible repositories on Huggingface (e.g., benjamin-paine/stable-diffusion-v1-5).

Thanks for the suggestion. I tried to use that repo and fixed some issues (e.g. no metadata in /safety_checker/***.safetensors, should be metadata={'format': 'pt'}) but got an error message:

AttributeError: 'ControlNetModel' object has no attribute 'nets'

debug info:

The attributes of controlnet are: ['T_destination', '_automatically_saved_args', '_backward_hooks', '_backward_pre_hooks', '_buffers', '_compiled_call_impl', '_deprecated_kwargs', '_forward_hooks', '_forward_hooks_always_called', '_forward_hooks_with_kwargs', '_forward_pre_hooks', '_forward_pre_hooks_with_kwargs', '_internal_dict', '_is_full_backward_hook', '_keep_in_fp32_modules', '_keys_to_ignore_on_load_unexpected', '_load_state_dict_post_hooks', '_load_state_dict_pre_hooks', '_modules', '_no_split_modules', '_non_persistent_buffers_set', '_parameters', '_state_dict_hooks', '_state_dict_pre_hooks', '_supports_gradient_checkpointing', '_version', 'attn_processors', 'call_super_init', 'class_embedding', 'config', 'config_name', 'device', 'dtype', 'dump_patches', 'encoder_hid_proj', 'has_compatibles', 'ignore_for_config', 'is_gradient_checkpointing', 'training']

full message here:

Traceback (most recent call last):
File "/home/haozong/projects/ComfyUI/execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/haozong/projects/ComfyUI/execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/haozong/projects/ComfyUI/execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "/home/haozong/projects/ComfyUI/execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/haozong/projects/ComfyUI/custom_nodes/ComfyUI-StyleShot/node.py", line 217, in generate
generation = pipeline.generate(
^^^^^^^^^^^^^^^^^^
File "/home/haozong/projects/ComfyUI/custom_nodes/ComfyUI-StyleShot/ip_adapter/ip_adapter.py", line 803, in generate
images = self.samples(
^^^^^^^^^^^^^
File "/home/haozong/projects/ComfyUI/custom_nodes/ComfyUI-StyleShot/ip_adapter/ip_adapter.py", line 755, in samples
images = self.pipe(
^^^^^^^^^^
File "/home/haozong/anaconda3/envs/comfyui/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/haozong/projects/ComfyUI/custom_nodes/ComfyUI-StyleShot/ip_adapter/ip_adapter.py", line 1049, in call
else controlnet.nets[0].config.global_pool_conditions
^^^^^^^^^^^^^^^
File "/home/haozong/anaconda3/envs/comfyui/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 155, in getattr
return super().getattr(name)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/haozong/anaconda3/envs/comfyui/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1709, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'ControlNetModel' object has no attribute 'nets'

[Jeoyal]

Hi, it seems like ControlNetModel loading error in ComfyUI, have you check the checkpoints?