Prompt Weights & Punctuations - How to use it in Automatic1111- Stable Diffusion

Prompt Engineer 01 - Prompt Weights, Blending, & Punctuations Using Automatic1111

Today we're diving into a topic that's been on my mind a lot lately: Prompt Weights. Now, you might have heard this term thrown around here and there, but let's be honest, there isn't one definitive guide that really breaks it down for us in great detail. So, I took matters into my own hands, scoured the vast corners of the internet, and decided to create a comprehensive blog series that will unravel the mysteries of Stable Diffusion, using Automatic1111's Web UI and try to understand what prompt weights and what the punctuations mean and how to use it.

Learn Everything!

We're going to learn everything there is to know about Prompt Weights in this blog, and trust me, by the end of this, you'll have a full understanding of it! I've got my notepad filled with questions, and my enthusiasm levels off the charts.

We're going to explore the ins and outs of Prompt Weights like never before. I'll be sharing my findings, breaking down complex concepts into easy-to-understand language, and providing practical examples along the way. By the end of this post, you'll have a solid grasp on prompt weights & punctuations and how to use it in Automatic1111, and you'll be able to apply it to your own project like a pro.

Remember, we're all about learning, growing, and pushing our creative boundaries here. It's not just about prompting the best images, it's about understanding the tools and techniques so that you can have full control of your outputs. Get ready to level up your prompt engineering game and join me on mastering Prompt Weights. WAGMI—We're all gonna make it!

What do parentheses do in Stable Diffusion?

To adjust a model's focus on specific words, use parentheses ( ) for emphasis and square brackets [ ] to diminish attention. In Stable Diffusion, wrapping a word with triple parentheses (((word))) boosts its weight by 1.05 times. Essentially, when a word is enclosed in parentheses, the model emphasizes it more in its output. Conversely, a word inside square brackets receives reduced prominence. In Stable Diffusion, however, triple parentheses amplify a word's significance, but overdoing it can cause that word to dominate the entire output. To deep dive, keep reading!

Help Support this Page

Please click on the image below to download this helpful Cheat Sheet. By sharing this page or the Cheat Sheet on Twitter @AndyHTu, you can contribute to the expansion of this resource and ensure we can continue providing you with valuable training material for future AI Photography Workflows, including the use of Stable Diffusion and other cutting-edge tools. Your support is greatly appreciated and instrumental in our growth.

Click on Image to Download Cheat Sheet

Stable Diffusion Prompt Weights & Prompt Editing

---

Table of Contents

1. Attention/Emphasis

2. Cheat sheet

3. A Different Kind of Blend

4. For Your Information

5. Questions or Requests?

---

Attention/Emphasis

Let's talk about how to enhance the model's attention using modifiers in your prompts.

To increase the model's attention to specific words, you can use parentheses ( ). For example, (bright) will make the model focus more on the word "bright" when generating the response. On the other hand, if you want to decrease the model's attention to certain words, you can use square brackets [ ]. For instance, [blurry] will make the model pay less attention to the word "blurry" in its output.

But wait, there's more! You can also combine multiple modifiers for greater control. For example, [(vibrant) and [colorful]] will increase the attention on both "vibrant" and "colorful" while generating the response.

You can specify a weight using parentheses like this: (text:1.4). If no weight is specified, it defaults to 1.1. Like wise with square brackets, weights work to reduce the weight.

Now, if you want to use literal parentheses or square brackets in your prompt, simply use a backslash to escape them. For example, anime_\(character\) .

A [frog:turtle:0.1] on a leaf in a forest = decrease to 0.1 adds turtle features.	A [frog:turtle:0.9] on a leaf in a forest = increase to 0.9 removes most the turtle features

(Left) A [frog:turtle:0.1] on a leaf in a forest = decrease to 0.1 adds turtle features. (Right) A [frog:turtle:0.9] on a leaf in a forest = increase to 0.9 removes most the turtle features

A (frog:turtle:0.1) on a leaf in a forest = decrease to 0.1 removes turtle, retains shape	A (frog:turtle:0.9) on a leaf in a forest = increase to 0.9 adds the turtle features on frog

(Left) A (frog:turtle:0.1) on a leaf in a forest = decrease to 0.1 removes turtle, retains shape (Right) A (frog:turtle:0.9) on a leaf in a forest = increase to 0.9 adds the turtle features on frog

What happens if we swapped frog:turtle > turtle:frog ? Let's see!

A [turtle:frog:0.1] on a leaf in a forest	A [turtle:frog:0.9] on a leaf in a forest

The idea is pretty similar, but the results are different. Play around with it and see what you get!

A single dog with a (turtle shell:0.25) with (elephant:0.00) ears in the ((((art style of Greg Rutkowski))))	A single [dog] with a (turtle shell:0.25) with (elephant:0.00) ears in the art style of Greg Rutkowski	A single dog with a (turtle shell:0.25) with (elephant:0.00) ears in the art style of Greg Rutkowski
A single dog with a turtle shell with (elephant:0.00) ears in the art style of Greg Rutkowski	A single (((dog))) with a turtle shell with elephant ears in the art style of Greg Rutkowski	A single dog with a turtle shell with elephant ears in the art style of (((Greg Rutkowski)))
A single dog with a turtle shell with elephant ((ears)) in the art style of Greg Rutkowski	​A single dog with a turtle shell with (elephant:0.25) ears in the art style of Greg Rutkowski	A single dog with a turtle shell with elephant ears in the art style of Greg Rutkowski

Cheat sheet:

a (word) - increase attention to word by a factor of 1.1

a ((word)) - increase attention to word by a factor of 1.21 (= 1.1 * 1.1)

a [word] - decrease attention to word by a factor of 1.1

a (word:1.5) - increase attention to word by a factor of 1.5

a (word:0.25) - decrease attention to word by a factor of 4 (= 1 / 0.25)

a \(word\) - use literal ( ) characters in prompt

[word | word] - a different way to blend multiple prompts, weights can be used

(word | word) - a different way to blend multiple prompts, weights can be used

word AND word - a different way to blend multiple prompts, weights can be used

[word:to:word] - [lion:robot] (This blends lion and robot equally)

[word:word:step] - lion:robot:20 (This means that 20 steps in, and it will change to the robot prompt. The lower the number, the sooner it will switch to the second part of the prompt. This will be based on the Sampling Steps you set below)

[word:word:percentage] - lion:robot:0.2 (This means 20 lion and terminator are 20 percent)

These give you slightly different results and you'd have to play around with it to see what it gives you. Steps and Percentage are essentially the same idea.

You can also mix in more than 2 words.

​Note: By the way, NAI uses my pre-2022-09-29 implementation, but with a multiplier of 1.05 and curly braces { } instead of parentheses (). So, if you're converting, keep that in mind. That's it for now! Stay tuned for more updates and cool features. Keep on creating!

Now, here's some news for you. On 2022-09-29, a new implementation that supports escape characters and numerical weights. However, the old implementation had a minor flaw where it sometimes omitted characters. For instance, "a (((farm))), daytime" would become "a farm daytime" without the comma. Don't worry, the new implementation preserves all text correctly, but it means your saved seeds might generate different results. If you prefer the old behavior, there's an option in the settings to switch back.

• their {word} = our (word:1.05)

• their {{word}} = our (word:1.1025)

• their [word] = our (word:0.952) (0.952 = 1/1.05)

• their [[word]] = our (word:0.907) (0.907 = 1/1.05/1.05)

A Different Kind of Blending

There are different ways to blend multiple prompts and add weights to further customize the model's response. Let's break them down:

1. [word | word]: This format allows you to blend multiple prompts by separating them with a vertical bar ( | ). For example, [sunset | landscape] will prompt the model to consider both "sunset" and "landscape" while generating the response. Additionally, you can add weights to each prompt to adjust their importance.

2. (word | word): Similar to the previous format, this one also blends multiple prompts but using parentheses instead of square brackets. So, (vintage | black and white) will prompt the model to consider both "vintage" and "black and white" in its response. Again, you can assign weights to influence their significance.

3. word AND word: This format allows you to blend prompts using the "AND" operator. For example, vibrant AND colorful prompts the model to consider both "vibrant" and "colorful" in generating the response. As with the previous formats, weights can be applied to fine-tune their influence.

Feel free to mix and match these blending methods and experiment with different situations. They offer alternatives but achieve similar outcomes in terms of blending prompts. Adding weights to each prompt enables you to prioritize specific aspects or themes.

So go ahead, have fun playing around with these techniques, and observe how they impact the model's responses. It's all about finding the right mix for your desired results. Happy experimenting!

Let's use everything and see what you got? adorable (pigs:1.5) | fierce (alligators:1.5)] AND (playful (turtles:1.2) | energetic (frogs:0.9)

Uh... I don't even know.

​ For Your Information I am using the Hardblend Model to get these realistic looks, so if you're using the standard SD 1.5, you won't get the same results. The best way to understand how to use this, is to just experiment with it. Good Luck!

Help Support this Page

Please click on the image below to download this helpful Cheat Sheet. By sharing this page or the Cheat Sheet on Twitter @AndyHTu, you can contribute to the expansion of this resource and ensure we can continue providing you with valuable training material for future AI Photography Workflows, including the use of Stable Diffusion and other cutting-edge tools. Your support is greatly appreciated and instrumental in our growth.

• Visit Civitai Community to download fine tuned models to play with.

• Reference: Github Automatic1111 User Guide.

CIVITAI.COM is the best place for your fine-tuned Stable Diffusion Models

---

Stay up to date with what's happening with Stability AI and Stable Diffusion.

---

Learning Stable Diffusion 101:

• How to Install Stable Diffusion Web UI

• Mastering the Automatic1111 User Interface: A Comprehensive Guide for Beginners

• Prompt Weights & Punctuations - How to use it in Automatic1111- Stable Diffusion

• Words of Power: The Essential Cheat Sheet for Super Prompts Techniques

• Maximize Your Workflow Efficiency: Pre-Planning with Stable Diffusion's XYZ Plot

Learn how to use Stable Diffusion and much more. Visit the link below-

Stable Diffusion 101