feat(backend): add stablediffusion-ggml #4289
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| func (sd *SDGGML) GenerateImage(opts *pb.GenerateImageRequest) error { | ||
| t := C.CString(opts.PositivePrompt) | ||
| defer C.free(unsafe.Pointer(t)) |
Check warning
Code scanning / Golang security checks by gosec
Use of unsafe calls should be audited Warning
| defer C.free(unsafe.Pointer(t)) | ||
|
|
||
| dst := C.CString(opts.Dst) | ||
| defer C.free(unsafe.Pointer(dst)) |
Check warning
Code scanning / Golang security checks by gosec
Use of unsafe calls should be audited Warning
✅ Deploy Preview for localai ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Signed-off-by: Ettore Di Giacinto <[email protected]>
mudler
force-pushed
the
feat/stablediffusion-ggml
branch
from
597a649
to
b5bcda2
Compare
Signed-off-by: Ettore Di Giacinto <[email protected]>
Signed-off-by: Ettore Di Giacinto <[email protected]>
|
Testing with: name: sd-ggml
backend: stablediffusion-ggml
parameters:
model: huggingface://city96/FLUX.1-dev-gguf/flux1-dev-Q2_K.ggufWIP as doesn't work still |
Signed-off-by: Ettore Di Giacinto <[email protected]>
Signed-off-by: Ettore Di Giacinto <[email protected]>
|
Testing it with: name: sd-ggml
backend: stablediffusion-ggml
parameters:
model: huggingface://city96/FLUX.1-dev-gguf/flux1-dev-Q2_K.gguf
options:
- "diffusion_model"
- "clip_l_path:clip_l.safetensors"
- "t5xxl_path:t5xxl_fp16.safetensors"
- "vae_path:ae.safetensors"
- "sampler:euler"
download_files:
- filename: ae.safetensors
sha256: afc8e28272cd15db3919bacdb6918ce9c1ed22e96cb12c4d5ed0fba823529e38
uri: https://huggingface.co/ChuckMcSneed/FLUX.1-dev/resolve/main/ae.safetensors
- filename: clip_l.safetensors
sha256: 660c6f5b1abae9dc498ac2d21e1347d2abdb0cf6c0c0c8576cd796491d9a6cdd
uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors
- filename: t5xxl_fp16.safetensors
sha256: 6e480b09fae049a72d2a8c5fbccb8d3e92febeb233bbe9dfe7256958a9167635
uri: https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp16.safetensors |
| sd.threads = int(opts.Threads) | ||
|
|
||
| modelFile := C.CString(opts.ModelFile) | ||
| defer C.free(unsafe.Pointer(modelFile)) |
Check warning
Code scanning / Golang security checks by gosec
Use of unsafe calls should be audited Warning
| size := C.size_t(unsafe.Sizeof((*C.char)(nil))) | ||
| length := C.size_t(len(opts.Options)) | ||
| options = (**C.char)(C.malloc(length * size)) | ||
| view := (*[1 << 30]*C.char)(unsafe.Pointer(options))[0:len(opts.Options):len(opts.Options)] |
Check warning
Code scanning / Golang security checks by gosec
Use of unsafe calls should be audited Warning
| defer C.free(unsafe.Pointer(dst)) | ||
|
|
||
| negative := C.CString(opts.NegativePrompt) | ||
| defer C.free(unsafe.Pointer(negative)) |
Check warning
Code scanning / Golang security checks by gosec
Use of unsafe calls should be audited Warning
Description
This PR is a first attempt into adding a very straightforward backend for stablediffusion.cpp (which is using ggml-based engine).
Notes for Reviewers
It does not wire up all the options available to stablediffusion.cpp, the purpose of this PR is to introduce the backend first, optimizations are left as an exercise for later.
SchedulerType,EnableParameters(used as sampler method)Signed commits