Hand FIXING Controlnet - MeshGraphormer

Hey, Olivio, been watching your videos for a while now. I just wanted to say what an absolute help your guides are and I am thankful that your channel exists. I tried comfyui a few months ago but I gave up because of large learning curve, but with your help, I'm not only cruising through it but learning faster than I thought possible.

Hey Olivio, I intergrated the mask captioning etc. to my workflow, but now the same error is occuring everytime. I tried a bit a round, but I am still clueless.
Error occurred when executing KSampler:
mat1 and mat2 shapes cannot be multiplied (308x2048 and 768x320)
... ( 100 lines complex code )

Need feet fingers

It's ridiculous that we have to build a workflow with THAT many nodes just to make hands look somewhat natural. There has to be a better way.

I was following your instruction using MeshGraphormer, I got error about "yacs" problem, and I have no idea about what that is, do you have any idea to fix that? I've google for that, and try to install it, but still get that error...

does the controlnet is really necessary? I´ve achieved the same result by passing the meshgraphormer mask through a VAE encode for impainting and it worked, I think it´s simplier, but I wonder if it compromisses the quality... thx.

I made a workflow for improving the face using a depth map, I would like to link to this process the improvement of hands using a depth map, as well as the process of enlarging with detail without losing quality. For me it turns out either soapy or pixelated around the edges.

ERROR : (IMPORT FAILED) comfyui-art-venture
How to fix ?

ERROR : When loading the graph, the following node types were not found:
AV_ControlNetPreprocessor
Nodes that have failed to load will show as red on the graph.
I tried many things, always pop up

Can you make the same tuto but with Stable Diffusion A1111 please

Great tute Olivio!

If when loading this one you receive the error:

When loading the graph, the following node types were not found:
AV_ControlNetPreprocessor
Nodes that have failed to load will show as red on the graph

You will be wanting the Art Venture node pack which isn't auto-found in the manager.

If installing that gives you media pipe errors you need to re-install the auxiliary processor nodes (reboot in between uninstall & reinstall and update Comfy - you know the drill).

Thanks to @sirdrak and @josesimoes1516 for providing those puzzle pieces.

Also protip: Jovimetrix nodes allows you to predefine the colours for all your nodes. It makes life 1000% better when everything is visually consistent and you don't have to do it manually every time!

I think I'm giving up on comfyui, so many noodles are confusing me, makes me crazy

Super valuable video, thanks a lot!

new new new,,,,but.....

segs preprocessor?

Hi Olivio, thats simply great, thank you very much for the workflow!

I can't get this to work ,every time I get an error

Error occurred when executing MeshGraphormer-DepthMapPreprocessor:

[Errno 2] No such file or directory: 'C:\\Users\\AShea\\Downloads\\ComfyUI_windows_portable\\ComfyUI\\custom_nodes\\comfyui_controlnet_aux\\ckpts\\hr16/ControlNet-HandRefiner-pruned\\cache\\models--hr16--ControlNet-HandRefiner-pruned\\blobs\\41ed675bcd1f4f4b62a49bad64901f08f8b67ed744b715da87738f926dae685c.incomplete'

ComfyUI doesn't look comfy AT ALL. 😢

Couldn't dwpose or openpose do the same thing?

beard

Add Krita +comfy work flow. Please! I see a lot of video that combines the 2 and line to see how you use it

How can you do this without comfy

I look forward to the day we laugh at the complexity involved in these type of fixes...Probably a couple of months looking at the speed of current developments.

great video, would you please consider making a tutorial for automatic 1111?

great tuto as always ! i would like to see how it works on img to img and with sdxl !

I encountered some challenges. Initially, I had to update the Python library MediaPipe to the latest version manually, as the automatic install/update process was unsuccessful. I experimented with various models and noticed mixed outcomes. Specifically, I used a newer checkpoint named "realisticVisionV60B1_v60B1VAE.safetensors" from the Realistic Vision series. Despite adjusting the depth control, the joint representation was peculiar. Before applying latent upscaling, the model accurately depicted the correct number of fingers according to depth perception. However, post-upscaling, the fingers appeared deformed or melted.

A little late comment, but you don't need to do a vae decode -> encode. There's a node called "Remove latent noise mask" that removes the mask so you can keep working on the same latent. (Every time you go between latent and pixel space you lose a little quality, as the decode/encode process is not lossless).
Also, you would probably get a little less sausage like hands if you lowered the denoise a bit to somewhere in the 0.7-0.9 area.

only for comfy?

Unfortunately I don't have the device requirement to set up comfyui. Please do you have any advice for me?

Are all your future videos going to be using Comfy? Need to know so i can unsub beforehand

Why is AV_ControlNetPreprocessor node red? Egadz!

Also you can use simple Midas Depth map instead ComfyUI's ControlNet Auxiliary Preprocessors

That's great! Would love to see some examples of more complicated hand positions or hands that are partly covered by some objects. Does it still work then or is it unusable in those scenarios ?

thank you very much, great as always. Can you do Ultimate SD Upscale instead?

If you are getting the error on the hands CN with KeyError: 'control_model.out.2.weight', update Comfyui (everything) and it should fix it.

Wait for SDXL

terrible tutorial. There's no point talking really slowly if the content of what you're saying is useless. I'm sure the people that are saying it's a great tutorial probably didn't need half the steps, but, if you're going to make a tutorial explain what you're doing. You zoom in and talk about one thing then say, 'you don't need to do that', then go on to something else without explaining what connects to what and how to find the nodes you're talking about! We want to follow along, but you make that impossible. I find your tutorials lack enough content. You take a long time to say very little and what you do say is disjointed an incoherent which makes following along seriously tedious!

Great video. Please note that Preprocess is requiring Node "Segment Anything" for some reason and without it can not be loaded! Thank You for sharing!

Instead of VAE decode and encode before your latent upscale, you could use a use 'get latent size' node, create an empty mask injecting width/height as input, and apply a blank mask as the new latent mask. Especially with larger images it will save you time versus going through the VAE pipeline, but also, since the VAE encoding/decoding is a lossy process, you actually lose quality between samples (not that an upscaled latent looks any better unless done iteratively) -- I prefer to upscale in pixelspace, then denoise starting at step 42, ending at step 52, then another sample after that from step 52 to carry me to 64 steps. I find three samples before post is my optimal workflow.

cannot install controlnet preprocessor, has this error Conflicted Nodes:
ColorCorrect [ComfyUI-post-processing-nodes], ColorBlend [stability-ComfyUI-nodes], SDXLPromptStyler [ComfyUI-Eagle-PNGInfo], SDXLPromptStyler [sdxl_prompt_styler]

Hello and have an awesome 2024 <3 Greetings from Greece!

hello, , i have tried to reach you on discord but i couldnt, i wanted to ask you a very important question.. once we upload our workflows in open ai .. we cant delete any of the workflows ? why ?

Thanks!
I have a problem when I launch the Workflow, this warning appears: "the following node types were not found: AV_ControlNetPreprocessor"

I downloaded and placed the "ControlNet-HandRefiner-pruned" file in this folder: ComfyUI_windows_portable\ComfyUI\models\controlnet.

Can you please tell me how to fix this?

huggingface_hub.utils._errors.LocalEntryNotFoundError

One suggestion for vids like this: a section at the end which shows 3 example prompts and results. Prompt can be on screen, dont have to read it out

How can we fix hands in automatic 1111?

Any idea why my "comfyui-art-venture" custom nodes have an "import failed" issue? Its is required by this workflow for the "ControlNet Preprocessor". I already made sure that all conflicting custom nodes are uninstalled.

Why it is called Comfy and why it is UI? Coz it is NOT comfy and not even UI lol