Why short prompts fail in Stable Diffusion
45sReveals a common beginner mistake and promises a fix, hooking viewers who struggle with AI art.
▶ Play ClipThis video provides a comprehensive guide to crafting effective prompts in Stable Diffusion, covering techniques for specificity, using negative prompts, leveraging scripts for variations, and integrating ChatGPT for prompt generation. The presenter demonstrates methods to control image output, from subject and environment to lighting and style, using Stable Diffusion Forge UI and the Juggernaut XL model.
The video aims to show how to think about prompting in Stable Diffusion, methods to create prompts faster, and uses Forge UI with Juggernaut XL v10 model.
Simple prompts like 'portrait of a woman' leave too much freedom to the AI. Specify image type (photo, illustration, painting) and details like 'modern photo' to get desired results.
Using a fixed seed allows for consistent experimentation with prompts, enabling comparison of changes.
Place the subject in an environment (forest, beach, studio with black background) and specify attributes like hair color, clothing, and lighting (rim light, golden hour).
Instead of broad terms like 'painting portrait', specify 'oil painting', 'watercolor', or 'pencil drawing'. Also consider cinematic fashion portrait.
List unwanted elements in negative prompt (e.g., police badge, blue light). It doesn't work perfectly; some words are stronger. Use different seeds or inpainting if needed.
Use XYZ plot script with prompt search and replace to test variations (e.g., different hair colors) and generate a grid of results.
Giving the subject a name (random or mix of celebrities) helps maintain facial consistency across generations.
Adjust sampling steps or CFG scale for subtle variations. Use XYZ plot with CFG scale values (e.g., 5-7) to generate and choose the best result.
Use ChatGPT to adapt a working prompt for different jobs (doctor, chef) by describing the desired changes.
Ask ChatGPT to write descriptive prompts for lazy prompting; guide it in the right direction.
Upload an image and use CLIP interrogate to get a prompt. Alternatively, upload to ChatGPT for a more accurate description.
If no internet or GPT, use pre-saved art styles from the internet. Art styles add a saved prompt to the original prompt.
Use round brackets to increase word weight (e.g., (blue house)). Use Ctrl+Up/Down to adjust weight. Avoid values above 1.9 to prevent artifacts.
Typical prompt order: art style/medium, subject, description, environment, extra info (colors, lighting, mood).
With GPT-4o, simply ask for a Stable Diffusion prompt (e.g., 'watercolor painting of a cute bunny with glasses') and test it.
Use 'Generate Forever' (right-click generate) or batch slider for multiple images. Use prompts from file/text box for multiple different prompts.
Effective prompting in Stable Diffusion requires specificity, use of negative prompts, and tools like XYZ plot and ChatGPT for variations. The video provides a structured approach to achieve desired image outputs efficiently.
"Title accurately reflects the content: a practical guide to prompting in Stable Diffusion."
What is the recommended prompt structure in Stable Diffusion?
Art style/medium, subject, description, environment, extra info (colors, lighting, mood).
9:12
How can you increase the weight of a word in a prompt?
Use round brackets around the word, e.g., (blue house). Use Ctrl+Up to increase weight, Ctrl+Down to decrease.
7:47
What is the purpose of a negative prompt?
To list elements you don't want in the generated image.
2:52
How can you generate multiple variations of a prompt efficiently?
Use the XYZ plot script with prompt search and replace, or use batch generation with prompts from a text file.
3:25
What tool can be used to get a prompt from an existing image?
CLIP interrogate (paperclip icon) in image-to-image, or upload to ChatGPT for a description.
6:06
What is the recommended maximum weight value to avoid artifacts?
Avoid going above 1.9 or 2.0.
8:50
How can you maintain facial consistency across generations?
Give the subject a name (random or mix of celebrities) and use a fixed seed.
4:00
What does CLIP stand for?
Contrastive Language-Image Pre-training.
6:16
Be Specific
Emphasizes that simple prompts give too much freedom to AI; specificity is key.
0:18Negative Prompt Limitations
Explains that negative prompts don't work perfectly and some words are stronger.
2:52XYZ Plot for Variations
Demonstrates a powerful script for testing prompt variations systematically.
3:25CLIP vs ChatGPT for Prompt Extraction
Compares two methods for getting prompts from images, noting ChatGPT yields more consistent results.
6:06Word Weighting Shortcuts
Introduces Ctrl+Up/Down for quick weight adjustment, a time-saving tip.
7:47[00:00] hey today I want to show you how I
[00:01] prompt in stable diffusion how I think
[00:03] about it when prompting and what methods
[00:05] I use to create prompts faster I'm using
[00:08] stable diffusion Forge UI and Juggernaut
[00:11] XL version 10 model but you can use any
[00:13] model you prefer with the settings that
[00:15] work best for that model many beginners
[00:18] simply prompt something simple like
[00:19] portrait of a woman while many short
[00:22] prompts still work in interfaces like
[00:24] Focus or AI like mid journey and chat
[00:26] GPT you're leaving too much freedom to
[00:29] the AI
[00:30] you can be more specific to get
[00:32] something closer to what you have in
[00:33] mind for example you can specify the
[00:35] type of image you want such as a photo
[00:37] illustration painting and so on as you
[00:40] can see even by mentioning photo the
[00:42] possibilities are still too many I got a
[00:44] vintage photo so maybe you can add an
[00:47] extra word like modern to specify that
[00:49] you want a modern photo instead of a
[00:52] vintage one to experiment better with
[00:54] the prompt you can use a fixed seed you
[00:57] can either use the last seed or just
[00:59] type number and generate again so we
[01:03] have the type of image and a subject but
[01:06] let's also place the subject in an
[01:08] environment maybe we can put her in the
[01:10] forest or on a beach or perhaps we want
[01:12] a studio photo with a black background
[01:14] behind maybe I don't want the woman to
[01:17] have a random hair color so I add blonde
[01:20] hair in the prompt perhaps I want her
[01:22] shirt to always be white so it stands
[01:24] out against the black background maybe I
[01:26] want the image to pop out more so I can
[01:29] add a rim light coming from behind
[01:32] visible in the hair alternatively I
[01:34] might want a golden hour type of light
[01:36] to make it look more natural if you have
[01:39] a specific hairstyle in mind you can add
[01:41] that to the prompt if you don't know the
[01:43] name you can search on Google for
[01:45] hairstyles women names for example if
[01:48] you like bangs you can add that to the
[01:51] prompt Additionally you can use chat GPT
[01:54] to provide lists for anything such as a
[01:56] list of women's clothing so I can use
[01:58] like a red cardigan instead of a white
[02:00] shirt let's try different types of art
[02:03] styles for the photo instead of just
[02:05] saying painting portrait which is a
[02:07] broad category we can specify further
[02:10] for example it could be an oil painting
[02:12] maybe with an old look or perhaps it's a
[02:15] watercolor painting we could also
[02:17] consider a pencil drawing but in this
[02:19] case we'd need to adapt the colors and
[02:21] background to match that style uh let's
[02:24] play around more with photography a
[02:27] cinematic fashion portrait could look
[02:29] cool
[02:30] we can also mention the nationality of
[02:33] the woman or the country for instance we
[02:36] could generate in African woman or a
[02:39] police woman since we've been so
[02:42] specific with clothes it might not
[02:44] include the police uniform so instead of
[02:46] a cardigan let's add a police suit or
[02:49] uniform let me quickly show you the
[02:52] negative prompt here you list things
[02:55] that you don't want to appear in your
[02:57] image it doesn't work perfectly some
[03:00] words are stronger than others but for
[03:02] example if you don't want a police badge
[03:05] you can add that to the negative prompt
[03:07] now when I generate again the badge is
[03:09] gone if you don't like the blue light
[03:11] behind you can add it to the negative
[03:13] prompt if that doesn't work you can try
[03:15] different seeds um in the worst case
[03:18] scenario you can use in painting to
[03:20] remove it let me remove the negative
[03:22] prompt and I'll show you how you can
[03:24] replace different words in the prompt
[03:25] scroll down and go to the XYZ plot on
[03:28] the X Type search for prompt Sr on the X
[03:31] values you put the words you want to
[03:33] search and replace first so for example
[03:36] I want to search for the word blonde and
[03:38] replace it with different colors like
[03:40] dark purple red and so on this way you
[03:43] can see how the girl looks with
[03:45] different hair colors as you can see now
[03:47] it started to generate the policewoman
[03:49] with different hair colors as a bonus at
[03:52] the end you have a grid with the
[03:53] searched word and all the other
[03:55] variation words that were replaced you
[03:57] can open it and see how they look in
[03:59] different
[04:00] colors let me dress her in a white shirt
[04:02] real quick let's say I want to redo
[04:04] everything but the girl's face to be
[04:06] more similar between Generations one
[04:09] thing that helps is to give her a name I
[04:11] usually use a random invented name but
[04:14] you can also use a mix of celebrities if
[04:16] you want so I'll give you a name and
[04:19] I'll generate again with that search and
[04:21] replace script active as you can see in
[04:24] the results the woman is quite similar
[04:26] between
[04:27] Generations because it has the same seed
[04:30] and the same description and name only
[04:33] the color is different let's say you
[04:35] like the results of a prompt but you
[04:37] just want a very subtle variation of it
[04:40] you can do that by adjusting the
[04:42] sampling steps or CFG scale you can do
[04:45] that automatically if you go back to the
[04:46] scripts and instead of prompt search and
[04:49] replace you choose CFG scale for the X
[04:52] values you can put between five and
[04:54] seven for example now when you generate
[04:57] you'll get those very subtle variations
[04:59] they might be harder to see on the
[05:00] screen but let's go to the folder where
[05:02] they were saved and switch from one to
[05:04] another as you can see some don't have
[05:06] badges or the light is a little
[05:08] different so you can choose the best one
[05:10] from all or you can combine them in
[05:12] Photoshop and keep what is best from
[05:15] each I also like to get variations for
[05:18] prompts using chat GPT so I have this
[05:20] prompt that I know works okay and I want
[05:23] a prompt adapted from different
[05:25] jobs I can ask GPT to adapt it for me
[05:29] and as you can see I get some nice
[05:32] variations let me test quickly the
[05:34] doctor prompt as you can see it added a
[05:37] white coat to the prompt now let's see
[05:39] how it looks like as a chef copy and
[05:41] paste The Prompt using the control plus
[05:43] C and Control Plus V shortcuts and
[05:46] generate quickly different variations
[05:49] sometimes I'm too lazy to write
[05:51] descriptive prompts so I just ask chat
[05:54] GPT to write it for me you just need to
[05:56] guide it a little in the right direction
[05:59] let's test it really quick to see what
[06:00] we get seems to work okay go to image to
[06:03] image and let me show you another trick
[06:06] that you can use when you don't know how
[06:07] to prompt but you have a photo or an
[06:09] illustration upload it here and then
[06:11] click on the paperclip icon that will
[06:14] interrogate clip which stands for
[06:16] contrastive language image pre-training
[06:18] the first time you're using it it will
[06:20] take some time to download a model you
[06:22] can see when it's ready in the command
[06:24] window but after that it should
[06:26] interrogate quicker that will give you a
[06:29] prompt for that image I'll copy this
[06:31] prompt and test it quickly the result is
[06:34] not bad could be better you can see that
[06:37] it's a policewoman but she doesn't look
[06:38] very similar to my image what seems to
[06:41] work better and get more consistent
[06:44] results for me is chat GPT I just upload
[06:47] a photo to chat GPT and ask it to
[06:49] describe it for me in a long sentence
[06:52] let's copy The Prompt and test it to see
[06:53] what we get now look at that how similar
[06:56] the photos look like with the image I
[06:58] gave it
[07:00] now you can talk with chat GPT to change
[07:02] the prompt how you want it for example
[07:04] you can adapt it for a doctor instead
[07:07] and this is the result if you don't have
[07:09] access to GPT or the internet and you
[07:12] still have problems with uh prompting
[07:14] but want to get some nice generation you
[07:17] can use art styles instead so if you
[07:20] have a short prompt you can go to art
[07:22] styles and either save your own art
[07:25] styles or download some free art styles
[07:27] from the internet I have an entire video
[07:29] with with 260 art styles and how to
[07:31] download them if you look at the prompt
[07:33] description then you look at the info
[07:35] for Generation you can see that you get
[07:38] the original prompt first then the art
[07:40] style is just another already saved
[07:42] prompt that is added to the original
[07:44] prompt plus the negative prompt let's
[07:47] talk a little bit about how to add more
[07:49] weight to certain words I have this
[07:52] prompt where I added green blue house
[07:54] next to each other to show you how I can
[07:56] make one word more important than the
[07:57] other so this is what I get for this
[08:00] prompt the house has blue and green in
[08:02] it I can make the words Blue House more
[08:04] important if I add round brackets you
[08:07] can see if you forgot to close a bracket
[08:10] it shows in red in the corner where the
[08:12] tokens are once I generate you can see
[08:15] the house has more blue than green you
[08:17] can add even more weight to those words
[08:19] if you add another
[08:21] bracket but there's an easier way to do
[08:24] this faster you just select the words
[08:27] then press control and use the up Arrow
[08:29] key to add more
[08:30] weight with control and the down arrow
[08:33] you can reduce the weight it can also
[08:36] have a negative value if you want it to
[08:37] be less important to change the value
[08:40] you select only the text without the
[08:42] brackets and numbers and use the
[08:44] shortcut again alternatively you can
[08:47] just change the numbers manually you
[08:50] should avoid going too high usually if
[08:52] you go more than 1.9 or two values it
[08:56] starts to go a little crazy and you get
[08:58] something like this like an a I exploded
[09:00] brain if I add a negative value to The
[09:02] Gnome for example it will be less
[09:05] important and you can see in the results
[09:07] that it's barely visible the main
[09:10] subject becomes the
[09:12] house here's how I usually prompt I put
[09:15] the art style or medium either first or
[09:17] last then comes the subject followed by
[09:19] the description then the environment and
[09:22] finally any extra information like
[09:23] colors lighting and mood while I was
[09:26] working on the video chat GPT released a
[09:29] new model version GPT 40 so now it's
[09:32] even easier to just ask for a stable
[09:35] diffusion prompt for what you want to
[09:36] create so if I want a watercolor
[09:38] painting of a cute bunny with glasses I
[09:40] can get a prompt quickly now I can test
[09:43] it in stable diffusion and the result is
[09:45] what I asked for I can then make more
[09:48] changes for example I can ask chat GPT
[09:51] to make it more minimalist and add a
[09:53] white background so now when I test it I
[09:55] get a nice cute cartoon bunny on a white
[09:58] background another thing you can do is
[10:00] to turn on generate Forever by right
[10:02] clicking on the generate button and
[10:04] choosing generate
[10:06] forever to stop it you need to
[10:08] rightclick again and choose cancel
[10:10] generate forever if you don't want to
[10:12] let it generate forever and have a
[10:14] specific number of generations in mind
[10:16] like between one and 100 you can move
[10:19] the batch slider to the value you want
[10:22] and it will generate that number of
[10:24] images alternatively if you have
[10:27] multiple different prompts you can
[10:29] choose prompts from a file or a text box
[10:32] you can paste prompts here or upload a
[10:34] text file with the prompts just make
[10:35] sure each prompt is on a separate line
[10:38] um I can once again use chat GPT to give
[10:40] me a list of different variations of the
[10:42] prompt with different animals then I can
[10:45] uh copy and paste that list of prompts
[10:47] in the text area when I hit generate it
[10:50] will start generating each of those
[10:52] prompts um as you can see I get
[10:53] different animals on a white background
[10:56] if you have any questions you can post
[10:57] them in my Facebook group pix Roma
[10:59] Community I usually post news prompts
[11:02] daily challenges design and crafts there
[11:05] I've recently passed 1,000 members so
[11:07] thank you to all who joined if you found
[11:09] something useful in this video leave a
[11:11] like thank you and have a great day
⚡ Saved you time reading this? Transcribe any YouTube video for free — no signup needed.