Support Flux.1 Tools

[anthonywu]

https://blackforestlabs.ai/flux-1-tools/

FLUX.1 Tools consists of four distinct features that will be available as open-access models within the FLUX.1 [dev] model series, and in the BFL API supplementing FLUX.1 [pro]:

• FLUX.1 Fill: State-of-the-art inpainting and outpainting models, enabling editing and expansion of real and generated images given a text description and a binary mask.
  • ref impl: https://github.com/black-forest-labs/flux/blob/main/src/flux/cli_fill.py
  • can borrow the Gradio UI for Fill workflow: https://github.com/black-forest-labs/flux/blob/main/demo_st_fill.py
• FLUX.1 Depth: Models trained to enable structural guidance based on a depth map extracted from an input image and a text prompt.
• FLUX.1 Canny: Models trained to enable structural guidance based on canny edges extracted from an input image and a text prompt.
• FLUX.1 Redux: An adapter that allows mixing and recreating input images and text prompts.
  • ref impl: https://github.com/black-forest-labs/flux/blob/main/src/flux/cli_redux.py

[raysers]

This is truly exciting news! I noticed that ComfyUI officially supported it a few hours ago, but as usual, it’s likely still not very friendly for Mac users. If mflux could follow up with support, that would be fantastic. As a user, I’m really looking forward to it! Thanks to the official team for including this in the plans. In fact, perhaps the team could consider giving this proposal a higher priority...

[filipstrand]

Would be great to support these features! Personally, I'll focus on finishing the Dreambooth feature near term, and then look into these new tools (and of course prioritise reviewing and helping out with contributions :) ). I have not tried these yet, but it feels like some of them (e.g the canny one) are very similar to features we currently support with controlnet. Would be interesting to compare and discuss to what extent we can/should replace these with the "official" ones from BFL.

[raysers]

Would be great to support these features! Personally, I'll focus on finishing the Dreambooth feature near term, and then look into these new tools (and of course prioritise reviewing and helping out with contributions :) ). I have not tried these yet, but it feels like some of them (e.g the canny one) are very similar to features we currently support with controlnet. Would be interesting to compare and discuss to what extent we can/should replace these with the "official" ones from BFL.

Yes，大佬，you should continue focusing on your current work as planned. Flux.1 Tools seems to be a sudden release, more like an unexpected event. However, not every unexpected event requires an urgent response, and at the very least, it shouldn’t disrupt your existing plans.

I’m not a code contributor, but I’d like to share my thoughts on Flux.1 Tools purely from the perspective of a user and observer:

As a user, I naturally hope that mflux will support more and more features. At the same time, as an observer, I see that Flux.1 Tools could be a potential shortcut to reduce the workload for the official team:

1. FLUX.1 Fill

It’s said to deliver better inpainting results than any current method on Flux1, which could directly address issue #88.

2. FLUX.1 Depth & FLUX.1 Canny

   I noticed that "depth" is already on the official research list. If Flux.1 Tools is introduced, it might save the team from duplicating efforts on depth research.
   As for the "canny" feature, it overlaps with existing functionalities and might not be necessary unless it’s proven to deliver significantly better results.

   Additionally, FLUX.1 Depth and FLUX.1 Canny come with LoRA versions. Based on evaluations, their performance is comparable to the full 23GB ControlNet version. I tested both LoRAs yesterday, but unfortunately, they’re not compatible with mflux. If they were, they could save the official team a lot of work. Furthermore, each LoRA is just over 1GB+, making them much more storage-friendly.

3. FLUX.1 Redux

   I’m not entirely sure if it’s similar to ipadapter since I’m unfamiliar with it. It simply struck me as an intriguing feature.

When I see such a comprehensive toolbox, as a user, I naturally hope that the official team can follow up and support it so we can experience these features sooner. That’s why I think this proposal might be worth considering as a higher priority.

Additionally, I noticed that Mr. Wu seems to have addressed issue #103 during his work process, which was an unexpected bonus. I’m aware of this model because it has recently gained a lot of attention in the Chinese community.

The above are just some genuine thoughts of mine. I’ve boldly expressed them here, hoping they align with the spirit of sincere communication in this community.