Getting horrible results with thiebaud_xl_openpose

[dayunbao]

I just upgraded my AWS EC2 instance type to a g5.xlarge so it can better handle SD XL. The speed up I got was impressive. Anyway, I did two generations to compare the quality of the images when using thiebaud_xl_openpose and when not using it.

When not using it the results are beautiful:

But when I use it, the results are horrible:

Is anyone else getting similar results? I make use of the OpenPose model extensively, but these results make it unusable.

Here are the prompts and settings I used.

I started the server with the following command line arguments: webui.sh --xformers --no-half-vae

Without thiebaud_xl_openpose:

A (humanoid: 1.5) (samurai:1.2) cat (standing:1.3) in a cyberpunk alleyway lit by neon signs
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1844297559, Size: 1024x1024, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 63aeecb90f, VAE: sdxl_vae.safetensors, Version: v1.6.0-1-g0562456f

With thiebaud_xl_openpose:

A (humanoid: 1.5) (samurai:1.2) cat (swinging:1.3) a sword in a cyberpunk alleyway lit by neon signs
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1844297559, Size: 1024x1024, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 63aeecb90f, VAE: sdxl_vae.safetensors, ControlNet 0: "Module: dw_openpose_full, Model: thibaud_xl_openpose [c7b9cadd], Weight: 1, Resize Mode: Crop and Resize, Low Vram: True, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced", Version: v1.6.0-1-g0562456f

[dayunbao]

I just realized that I forgot to include --no-half-controlnet. So I just did that and tried again. The results aren't much better:

The prompt is slightly different, and I used a different sampler:

A (humanoid: 1.5) (samurai:1.2) (black: 1.2) cat (standing:1.3) (holding a katana in one hand: 1.3) in a cyberpunk alleyway lit by neon signs
Negative prompt: Extra limbs, extra fingers, deformed hands, low quality,
Steps: 50, Sampler: UniPC, CFG scale: 7, Seed: 3057226458, Size: 1024x1024, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 63aeecb90f, VAE: sdxl_vae.safetensors, ControlNet 0: "Module: dw_openpose_full, Model: thibaud_xl_openpose [c7b9cadd], Weight: 1, Resize Mode: Crop and Resize, Low Vram: True, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced", Version: v1.6.0-1-g0562456f

[dayunbao]

The results with other OpenPose models aren't much better, in fact, they don't seem to even be using the pose:

t2i-adapter_diffusers_xl_openpose:

kohya_controllllite_xl_openpose_anime_v2:

I used the dw_openpose_full preprocessor with each. Should I have used something different?

[huchenlei]

I think none of the current SDXL controlnet openpose model is trained with hand/face keypoints data. So when you use these models, you should probably either use openpose preprocessor that does not include hand/face or manually remove hand/face using editor.

[dayunbao]

OK, but there is still something wrong. The only SD XL OpenPose model that consistently recognizes the OpenPose body keypoints is thiebaud_xl_openpose. But that model destroys all the images. It's unusable. So I've been using kohya_controllllite_xl_openpose_anime_v2, but that only seems to recognize the OpenPose body keypoints about 35-40% of the time, assuming it's recognizing them at all, and it's not just luck that the poses of some of the characters match the keypoints.

An example. This is the input:

These are the results:

The output of the logs during image generation:

Oct 05 06:40:41: 2023-10-05 06:40:41,443 - ControlNet - INFO - Loading model from cache: kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]
Oct 05 06:40:41: 2023-10-05 06:40:41,447 - ControlNet - INFO - Loading preprocessor: none
Oct 05 06:40:41: 2023-10-05 06:40:41,447 - ControlNet - INFO - preprocessor resolution = 512
Oct 05 06:40:41: 2023-10-05 06:40:41,483 - ControlNet - INFO - ControlNet Hooked - Time = 0.05967235565185547

Here's the results using thiebaud_xl_openpose:

and the API logs:

Oct 05 06:44:40: 2023-10-05 06:44:40,247 - ControlNet - INFO - Loading model: thibaud_xl_openpose [c7b9cadd]
Oct 05 06:44:40: 2023-10-05 06:44:40,336 - ControlNet - INFO - Loaded state_dict from [/home//stable-diffusion-webui/extensions/sd-webui-controlnet/models/thibaud_xl_openpose.safetensors]
Oct 05 06:44:40: 2023-10-05 06:44:40,348 - ControlNet - INFO - controlnet_sdxl_config
Oct 05 06:44:47: 2023-10-05 06:44:47,437 - ControlNet - INFO - ControlNet model thibaud_xl_openpose [c7b9cadd] loaded.
Oct 05 06:44:47: 2023-10-05 06:44:47,509 - ControlNet - INFO - Loading preprocessor: none
Oct 05 06:44:47: 2023-10-05 06:44:47,509 - ControlNet - INFO - preprocessor resolution = 512
Oct 05 06:44:47: 2023-10-05 06:44:47,541 - ControlNet - INFO - ControlNet Hooked - Time = 7.52367377281189

Do I need to change some settings, or is kohya_controllllite_xl_openpose_anime_v2 just not working?

[dayunbao]

I made a fresh call after restarting the server to make sure everything was being loaded the same way. Looks like it is:

Oct 05 06:57:46: 2023-10-05 06:57:46,315 - ControlNet - INFO - Loading model: kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]
Oct 05 06:57:46: 2023-10-05 06:57:46,915 - ControlNet - INFO - Loaded state_dict from [/home/stable-diffusion-webui/extensions/sd-webui-controlnet/models/kohya_controllllite_xl_openpose_anime_v2.safetensors]
Oct 05 06:57:47: 2023-10-05 06:57:47,440 - ControlNet - INFO - ControlNet model kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb] loaded.
Oct 05 06:57:47: 2023-10-05 06:57:47,447 - ControlNet - INFO - Loading preprocessor: none
Oct 05 06:57:47: 2023-10-05 06:57:47,447 - ControlNet - INFO - preprocessor resolution = 512
Oct 05 06:57:47: 2023-10-05 06:57:47,491 - ControlNet - INFO - ControlNet Hooked - Time = 1.3990898132324219

[dayunbao]

After running a very similar generation in both the web UI and in my app via the API, I think there's a bug somewhere in the API.

Web UI settings:

A cute Asian woman standing in a kitchen
Steps: 50, 
Sampler: DPM++ 3M SDE Karras, 
CFG scale: 7, 
Seed: 3178681427, 
Size: 1024x1024, 
Model hash: 31e35c80fc, 
Model: sd_xl_base_1.0, 
VAE hash: 63aeecb90f, 
VAE: sdxl_vae.safetensors, 

ControlNet 0: 
    "Module: openpose, 
    Model: kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb], 
    Weight: 1, 
    Resize Mode: Crop and Resize, 
    Low Vram: False, 
    Processor Res: 512, 
    Guidance Start: 0, 
    Guidance End: 1, 
    Pixel Perfect: False, 
    Control Mode: Balanced"

Version: v1.6.0-1-g0562456f

Request body:

{
    "prompt": "A cute Asian woman standing in a kitchen",
    "negative_prompt": "",
    "cfg_scale": 7,
    "sampler_name": "DPM++ 3M SDE Karras",
    "batch_size": 8,
    "seed": 0,
    "steps": 50,
    "alwayson_scripts": {
        "controlnet": {
            "args": [
                {
                    "input_image": "a base64 string",
                    "enabled": true,
                    "lowvram": false,
                    "pixel_perfect": false,
                    "module": "openpose",
                    "model": "kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]",
                    "processor_res": 512,
                    "threshold_a": 64,
                    "threshold_b": 64,
                    "weight": 1,
                    "guidance_start": 0,
                    "guidance_end": 1,
                    "control_mode": 0,
                    "resize_mode": 1
                }
            ]
        }
    }
}

Results:

[dayunbao]

Another test, since I realized I wasn't passing the image width and height when calling txt2img:

Request body:

{
    "prompt": "A cute Asian woman standing in a kitchen",
    "negative_prompt": "",
    "cfg_scale": 7,
    "sampler_name": "DPM++ 3M SDE Karras",
    "batch_size": 8,
    "seed": 0,
    "height": 1024,
    "width": 1024,
    "steps": 50,
    "alwayson_scripts": {
        "controlnet": {
            "args": [
                {
                    "input_image": "",
                    "enabled": true,
                    "lowvram": false,
                    "pixel_perfect": false,
                    "module": "openpose",
                    "model": "kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]",
                    "processor_res": 512,
                    "threshold_a": 64,
                    "threshold_b": 64,
                    "weight": 1,
                    "guidance_start": 0,
                    "guidance_end": 1,
                    "control_mode": 0,
                    "resize_mode": 1
                }
            ]
        }
    }
}

I am passing the input_image, by the way. I just didn't want to paste a huge base64 encoded string.

I used the same OpenPose keypoints:

But it still seems that kohya_controllllite_xl_openpose_anime_v2 is ignoring the keypoints:

[wandrzej]

overall most if not all openpose controlnets for SDXL are bad. They either ignore the pose, or hit the quality, or both.
afaik there was a half decent one from T2I, but still not even close to what one could get with SD1.5 models.

[dayunbao]

I'll try the T2I model some more, thanks.

My co-founders and I were already thinking about on training our own OpenPose model for ControlNet. Now it looks like we'll definitely have to,

[dayunbao]

Is this just a problem when using Automatic1111's web UI? Does it work any better in ComfyUI?

[dayunbao]

There is a new OpnePose XL model available from the maker of thiebaud_xl_openpose:

https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0/tree/main

Maybe this one would work better? @lllyasviel do you think we could get support added for this model?

[dayunbao]

@lllyasviel I dropped that model into the appropriate folder, and tried using it since the yaml files aren't used any more. The API log said it was loaded, but it didn't seem to work. The images didn't respect the OpenPose keypoints at all. I don't know if there is any special configuration that needs to be done before that model can be used, though.

[dedkamaroz]

I'm just wondering if you've had any updates about this issue?

[dayunbao]

No, unfortunately. Unless someone has released new ControlNet OpenPose models for SD XL, we're all borked. The problem seems to lie with the poorly trained models, not ControlNet or this extension. So I'll close this. If my startup is able to get funding, I'm planning on setting aside money specifically to train ControlNet OpenPose models.

[sdbds]

https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite/
you can test my openpose model.