Skip to content

Conversation

linoytsaban
Copy link
Collaborator

@linoytsaban linoytsaban commented Sep 16, 2024

In the recent PuLID Flux adaptation, authors (ToTheBeginning Yanze Wu, guozinan126, guozinan126) also introduce true CFG - a simple & straight forward way to support negative prompts and "real" CFG for the distilled flux models.

Based on their idea and code, this PR adds a flux pipeline supporting negative prompts.
Inference can be done as follows -

import torch
import diffusers
from pipeline_flux_with_cfg import FluxCFGPipeline

pipe = FluxCFGPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", 
                                    torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload()

prompt = "a watercolor painting of a unicorn"
negative_prompt = "pink"


#### without true cfg
torch.manual_seed(0)
img_1 = pipe(prompt).images[0].resize((512,512))


#### with true cfg
torch.manual_seed(0)
img_2 = pipe(prompt=prompt, negative_prompt=negative_prompt, true_cfg=1.5, guidance_scale=3.5).images[0].resize((512,512))

output examples courtesy of @apolinario 🤗:

Group 1-37

Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu yiyixuxu merged commit 8fcfb2a into huggingface:main Sep 16, 2024
8 checks passed
@linoytsaban linoytsaban deleted the cfg-flux-exploring branch September 17, 2024 10:38
@arnold408
Copy link

arnold408 commented Sep 21, 2024

Hi,
Will this work with Flux.1-schnell ? What should be the "guidance_scale" ?
Thanks

leisuzz pushed a commit to leisuzz/diffusers that referenced this pull request Oct 11, 2024
* true_cfg

* add check negative prompt/embeds inputs

* move to community pipelines

* move to community pipelines

* revert true cfg changes to the orig pipline

* style

---------

Co-authored-by: YiYi Xu <[email protected]>
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
* true_cfg

* add check negative prompt/embeds inputs

* move to community pipelines

* move to community pipelines

* revert true cfg changes to the orig pipline

* style

---------

Co-authored-by: YiYi Xu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants