Have you ever found yourself wanting to adjust a photo during the retouching process? Perhaps you’ve thought, “It would be better if it were a little more to the left!” or “It would be better if it were a little wider!” In the past, there was only one manual way to make these adjustments. However, thanks to the new Inpaint model launched by Stable Diffusion + ControlNet, you can now not only paint the picture but also extend and complete it (outpainting)! The capabilities of this model are truly impressive, and I invite you to see it in action!
Table of Contents
– Preparation Material
– Downloading the Inpaint ControlNet Model
– Experiment Image Preparation
– Collecting Prompted Words using Interrogate
– Painting with ControlNet Inpaint
– Basic txt2img Settings
– ControlNet Settings
– Further Reading
Preparation Material
To begin utilizing the Inpaint ControlNet Model, you will need to download the control_v11p_sd15_inpaint.pth and control_v11p_sd15_inpaint.yaml files from HuggingFace. These files should be placed in the stable-diffusion-webuiextensionssd-webui-controlnet folder. Once this is done, restart the StableDiffusion WebUI. If you see the ControlNet v1.1 block and Inpaint Model, then the installation is complete!
Note: If you had ControlNet installed previously, it must be updated to ControlNet v1.1 or a higher version in order to access the inpaint functionality.
Experiment Image Preparation
In order to test Stable Diffusion’s ability to expand outwards, it is recommended to deliberately use the entire image as a crop and remove both sides. By doing so, you can compare the AI-generated completion of the central part with the original image. Please cut out the test pattern from both sides.
Collecting Prompted Words using Interrogate
To enhance the perfection of the picture, you can use the built-in Interrogate CLIP or Interrogate DeepBooru in the Stable Diffusion img2img tab. This feature allows you to deduce the prompt from the picture and make manual adjustments accordingly. By doing so, you can achieve faster results. Here are the detailed steps:
1. Drag the image into the source block of the img2img tab.
2. Click the orange Interrogate CLIP button on the top right (DeepBooru is also available and will generate keywords).
3. Wait for a short calculation time, and the prompt words and collected data will appear in the Positive Prompt backup input box.
Painting with ControlNet Inpaint
Now that everything is ready, let’s proceed with the operation in the txt2img Stable Diffusion tab:
1. Paste the prompt.
2. Increase the Negative Prompt Sampling Method appropriately. Choose Euler because it has a faster calculation speed and is used for preliminary composition.
3. Set the Sampling Steps to 20 by default.
4. Adjust the width and height to 784 and 512 respectively. These values can be freely adjusted, but it’s important to ensure that the proportions are different from the original picture. For example, if the original image is in a vertical format, the width should be set to be greater than the height. Keep the other settings at their default values.
ControlNet Settings
Next, let’s focus on the ControlNet settings:
1. Check the enable box.
2. Drag the image to the source area.
3. Select “inpaint_only” as the Preprocessor.
4. Choose “control_v11p_sd15_inpaint” as the Model.
5. Select “Resize and Fill” as the Resize Mode. This step is crucial as it ensures that the blank areas will be filled correctly during the outpainting process. Choosing the wrong mode may result in incomplete painting.
Further Reading
For more information and resources on this topic, please refer to the following:
– [Provide the link to further reading resources here]
Note: Please ensure that the provided prompt is unique and different from the original text.
When retouching photos, have you ever come across “It would be better if it was a little more to the left!” “It would be better if it were a little wider!” In the past, there was only way to P manually that I could find. out, but depending on the new Inpaint model launched by Stable Diffusion + ControlNet , in addition to painting the picture, it can also be extended to complete the picture (outpaintng)! It’s too strong! Come and see how it works!
Table of contents
preparation material
Download the Inpaint ControlNet Model
Inpaint ControlNet Model Download control_v11p_sd15_inpaint.pth and control_v11p_sd15_inpaint.yaml from HuggingFace, and place them in the stable-diffusion-webuiextensionssd-webui-controlnet folder.
Go back to StableDiffusion WebUI, restart it, if you see the ControlNet v1.1 block and Inpaint Model, it means the installation is complete!
Note: If ControlNet has been installed before, it must be updated to ControlNet v1.1 or higher to get the inpaint functionality!
Prepare pictures of the experiment
To “contrast” Stable Diffusion’s ability to expand outwards, I deliberately used the entire image as a crop to remove both sides, leaving the central part for AI to complete, and finally compared to the image original
Cut out the test pattern on both sides
Collect prompted words using Interrogate
In order to make the picture more perfect, you can use the built-in Interrogate CLIP or Interrogate DeepBooru in the tab Stable Diffusion img2img to deduce the prompt from the picture, and then adjust it manually according to the situation, which is faster . The detailed steps are as follows:
Drag the image into the source block of the img2img tab and click the orange Interrogate CLIP button on the top right (DeepBooru is also available, it will be generated as a keyword) Wait for a short calculation time, and the prompt words and collected appears in the Positive Prompt backup input box
Paint with ControlNet Inpaint
Basic txt2img settings
Everything is ready, go back to the txt2img Stable Diffusion tab page and start the operation:
Paste the Prompt now, and increase the Negative Prompt Sampling Method appropriately: Choose Euler and, because the calculation speed is relatively fast, and it is used for preliminary composition Sampling Steps: First use the default 20 Width / High : set to 784, 512 respectively, and the values can be freely adjusted, but in order for it to be able to make an external painting, the proportion must be different from the original picture! For example, the original image is a vertical format, and to make it a painted out banner, the width must be set to be greater than the height. Others keep the default
ControlNet Settings
Next, the focus is on ControlNet settings:
Check enable and drag the image to the source area Preprocessor: Select inpaint_only Model: Select control_v11p_sd15_inpaint Resize Mode: Select Resize and Fill, very important! Because it is necessary for him to fill in the blank, if he chooses wrong, there will be no painting!
Further reading
#expand #pictures #outwards #Stable #Diffusion #ControlNet #painting
