--- title: Complete guide to samplers in Stable Diffusion description: Dive into the world of Stable Diffusion samplers and unlock the potential of image generation. url: https://www.felixsanz.dev/articles/complete-guide-to-samplers-in-stable-diffusion language: en published: 2023-06-30 updated: 2023-06-30 tags: - ai - stablediffusion alternate-es: https://www.felixsanz.dev/es/articulos/guia-completa-de-samplers-en-stable-diffusion --- # Complete guide to samplers in Stable Diffusion ## Introduction As we saw in the article [How Stable Diffusion works](https://www.felixsanz.dev/articles/how-stable-diffusion-works#sampling), when we ask Stable Diffusion to generate an image the first thing it does is generate an image with noise and then the sampling process removes noise through a series of steps that we have specified. It would be something like **starting with a block of white marble** and hammering it for several days **until you get Michelangelo's David**. Several algorithms come into play in this process. The one known as **sampler** is in charge of obtaining a sample from the model that we are using in Stable Diffusion on which the noise estimated by the noise predictor is applied. It then subtracts this sample from the image it is cleaning, **polishing the marble in each step**. This algorithm handles the **how**, while the algorithm known as the **noise scheduler** handles the **how much**. > If the noise reduction were linear, our image would change the same amount in each step, producing abrupt changes. A negatively sloped noise scheduler can remove large amounts of noise initially for faster progress, and then move on to less noise removal to **fine-tune small details** in the image. > > [How Stable Diffusion works](https://www.felixsanz.dev/articles/how-stable-diffusion-works#sampling) Following the marble analogy, in the beginning it will probably be more useful to give it good hits and **remove large chunks to advance quickly**, while towards the end we will have to **go very slowly to fine tune the details** and not make an arm fall off. A key aspect of the process is **convergence**. When a sampling algorithm reaches a point where more steps will not improve the result, the image is said to have converged. Some algorithms converge very quickly and are **ideal for testing ideas**. Others take longer or require a greater number of steps but usually offer **more quality**. Others never do because they have no limit and offer more **creativity**. With this article you will understand the nomenclature as well as the uses of the different methods without going into too much technical detail. > [!NOTE] > **Parameters** > The image used in the demonstrations has been generated with the following parameters: > > - **Checkpoint**: `dreamshaper_631BakedVae.safetensors`. > - **Positive prompt**: `ultra realistic 8k cg, picture-perfect black sports car, desert wasteland road, car drifting, tires churns up the dry earth beneath creating a magnificent sand dust cloud that billows outwards, nuclear mushroom cloud in the background far away, sunset, masterpiece, professional artwork, ultra high resolution, cinematic lighting, cinematic bloom, natural light`. > - **Negative prompt**: `paintings, cartoon, anime, sketches, lowres, sun`. > - **Width/Height**: `512`/`512`. > - **CFG Scale**: `7`. > - **Seed**: `1954306091`. ## Samplers available Depending on the software used you will find a varied list of possibilities. In this case we are going to analyze the samplers available in Automatic1111. It is difficult to classify them into groups, although there are clearly two main approaches: - **Probabilistic models** such as `DDPM`, `DDIM`, `PLMS` and the `DPM` family of models. These generative models are able to generate an output based on the probability distribution estimated by the model. It would be like using a **camera to photograph a landscape**. - **Numerical approach methods** such as `Euler`, `Heun` and `LMS`. In each step, the solution to a particular mathematical problem is sought and the solution is estimated bit by bit. In this case it would be like using **painting and a canvas to create the landscape** and adding new details in each step. ### DDPM `DDPM` ([paper](https://arxiv.org/abs/2006.11239)) (Denoising Diffusion Probabilistic Models) is one of the first samplers available in Stable Diffusion. It is based on explicit probabilistic models to remove noise from an image. It requires a large number of steps to achieve a decent result. It is no longer available in Automatic1111. ### DDIM `DDIM` ([paper](https://arxiv.org/abs/2010.02502)) (Denoising Diffusion Implicit Models) works similarly to `DDPM`, using in this case implicit probabilistic models. This difference produces better results in a much smaller number of steps, making it a faster sampler with little loss of quality. ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/ddim10.DeiMH9O9_2hCq6l.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/ddim30.Bq9DzMYh_Z1r9qXW.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/ddim50.Bpk4B4d7_Z18oMU5.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/ddim100.Ccdz4sex_2lH2E4.jpg) ### PLMS `PLMS` ([paper](https://arxiv.org/abs/2202.09778)) (Pseudo Linear Multi-Step) is an improvement over `DDIM`. Using a 50-step process it is possible to achieve higher quality than a 1000-step process in `DDIM`. Fascinating, isn't it? Well, read on, this is nothing. ![Distorted image](https://www.felixsanz.dev/assets/plms10.Cy2jenwB_Z1eq5Sm.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/plms30.DOv2NksY_RaSn0.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/plms50.DzNGoZ4l_15me9r.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/plms100.fh77gKqS_28wWek.jpg) ### Euler `Euler` is possibly the simplest method. Based on [ordinary differential equations](https://en.wikipedia.org/wiki/Ordinary_differential_equation) (ODE), this numerical method eliminates noise linearly in each step. Due to its simplicity it may not be as accurate as we would like but it is one of the fastest. ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/euler10.CMbvxDY3_O8l1v.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/euler30.BXUTfu_J_ZAFk6D.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/euler50.D2c9JLYr_RHgdL.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/euler100.w5V7K7IH_ZRh4NQ.jpg) ### Heun `Heun` is the perfectionist brother of `Euler`. While `Euler` only performs a linear approximation, `Heun` performs two tasks in each step, making it a second-order sampler. It first uses a linear approximation for prediction and then a nonlinear approximation for correction. This improvement in accuracy offers higher quality in exchange for a small drawback: it takes twice as long. > [!NOTE] > **A bit of history** > Karl Heun developed this numerical method more than a century ago! ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/heun10.DyRj28cz_zC2SJ.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/heun30.BdmD8Nj9_1PkrhA.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/heun50.BrI98p2S_ZAU71d.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/heun100.DfDu1-II_ZpawyM.jpg) ### LMS `LMS` or [Linear Multi-Step method](https://en.wikipedia.org/wiki/Linear_multistep_method) is the cousin of `PLMS` that uses a numerical rather than a probabilistic approach (`PLMS` - `P` = `LMS`). Moreover, unlike `Euler` and `Heun`, it uses information from previous steps to reduce noise in each new step. It offers better accuracy in exchange for higher computational requirements (slower). ![Distorted image](https://www.felixsanz.dev/assets/lms10.5eopwSkp_ZrPfNQ.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/lms30.B7mCdcsu_1NufbE.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/lms50.CHc3LSeE_Z1bpQIH.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/lms100.CDjUKbUq_Z2ufX5S.jpg) ### Family of DPM models `DPM` (Diffusion Probabilistic Models) are probabilistic models that offers improvements over `DDPM`. Hence the similar name. There is also no implementation available in Automatic1111 because it has improved versions as we will see below. `DPM2` is an improvement over `DPM`. You could say that it is version 2. ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpm210.CJkV48iK_Z2gAChA.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpm230.ft1giF_t_ZlcUv9.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpm250.De04Gv_O_Z2wYe4t.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpm2100.CIpPSH0V_ZwcgLj.jpg) On the other hand we have `DPM++`, which is also an improvement of `DPM`. It uses a hybrid approach combining deterministic and probabilistic methods for sampling and subsequent noise reduction. There is no basic implementation of this sampler in Automatic1111, but it is combined with other methods. We will see this in the next section. Thus, two versions with improvements were born from `DPM`: `DPM2` and `DPM++`. #### Faster DPM models (`DPM-Solver` and `UniPC`) Diffusion Probabilistic Models (`DPM`) are, as the name suggests, **probabilistic**. In each step, **equations are not solved by deterministic numerical methods** as in the case of `Euler`, `Heun` or `LMS`, but the problem is approached by approximation to try to sample as accurately as possible. Within these models there is a piece called **solver**, an algorithm that has an important role in calculating and approximating a probability distribution in sampling. And this is where a new technique known as `DPM-Solver` is implemented that shortens the duration of each step. In other words, models like `DPM fast` ([paper](https://arxiv.org/abs/2206.00927)) or `DPM++ 2S`/`DPM++ 2M` ([paper](https://arxiv.org/abs/2211.01095)) implement a faster solver that will save time in the sampling process. ![Distorted image](https://www.felixsanz.dev/assets/dpmfast10.jS3pT9Dm_23Hgwz.jpg) ![Distorted image of a black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmfast30.BBkKzJjy_OO1Ed.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmfast50.D9Hz-hql_Z1vhMg5.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmfast100.ipoXyfBn_ZNWEXo.jpg) In the case of `DPM++ 2S`/`DPM++ 2M` the number `2` means that they are second order. That is, they use both a **predictor** and a **corrector** to approximate the result accurately. The `S` stands for `Single step`. A **single calculation** is performed in each step, so it is faster. In contrast, the letter `M` stands for `Multi step`, an approach in which **multiple calculations** are performed in each step, taking into account information obtained in previous steps. This equates to more accurate and higher quality convergence at the cost of taking longer. In both modalities this solver is faster than the default `DPM` model solver. There is no Automatic1111 implementation of `DPM++ 2S`, only with `A`, `Karras` and `SDE` variants (more on that later). So let's see some samples of `DPM++ 2M`. ![Distorted image of a black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2m10.CkLTadxD_aXY9V.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2m30.C054w9UZ_SvLar.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2m50.DuobTGGY_1aUG0.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2m100.D2iTo67c_ZPXkWA.jpg) As for `UniPC` ([paper](https://arxiv.org/abs/2302.04867)), it is a solver that consists of two parts: a unified predictor (`UniP`) and a unified corrector (`UniC`). This method can be applied to any `DPM` model and focuses on delivering the **maximum possible sampling quality in the least amount of steps**. Remember now when `PLMS` brought down to 50 steps what `DDIM` did in 1000? Well, in some cases `UniPC` is able to generate quality images in as few as 5 or 10 steps. Thus, `UniPC` can be integrated in `DPM` models both `Single step` and `Multi step`, making it comparable to `DPM++ 2S` or `DPM++ 2M`, with the particularity of offering better results when the number of steps is very low. Even the `UniC` corrector can be integrated into these sampling algorithms to achieve higher efficiency (e.g. `DPM++ 2S` + `UniC`). ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/unipc10.Cx55oQYX_1Hj8WW.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/unipc30.nC9d6TNk_srbs2.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/unipc50.viUWpz6E_ZTuQtD.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/unipc100.Bzg6o0i1_ZYPUM.jpg) #### More accurate DPM models (`Adaptive`) The `DPM adaptive` model is an extension of the `DPM` model in which it **adapts the step size** according to the difficulty of the problem is trying to solve. In other words, it is as if the specified number of steps is ignored and the algorithm is free to sample more efficiently until the best possible convergence is achieved. It generates higher quality images at the expense of taking as long as it needs to (it is the slowest sampler). ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmadaptive.DcNEn4oj_ZP3CLm.jpg) ## Other features Only one sampling algorithm can be chosen. Either `Euler` or `DPM` can be used, but not both at the same time. Instead, when we talk about variants or extra features, these can be combined. For example, we can use the sampler named `DPM2 A Karras`. Let's see what these new values mean. ### Ancestral variants When a sampler contains the letter `A`, it usually means that it belongs to the category of **ancestral** variants. These variants add, in each new step, random variables obtained from previous steps. It is as if after cleaning up the noise in one step, some of the previous noise is added back. Samplers with this feature **never converge because of the random noise added in each step**. If there is always noise to remove, you can always go one step further. This makes them more creative samplers. An extra step does not necessarily increase the quality, but rather gives another similar result. > [!NOTE] > **Reproducibility** > If you try to reproduce an image generated with Stable Diffusion and you don't succeed even though you are using the same seed and the same parameters, it may be because you are using an ancestral sampler. This is normal! The noise that is re-added in each step is random and different implementations or versions of the sampler will almost certainly generate different results. Some examples of samplers are `Euler A`, `DPM2 A` or `DPM++ 2S A`. ![Nuclear explosion in the desert](https://www.felixsanz.dev/assets/eulera10.DO_Y5Mqh_28mU6g.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/eulera30.VS6gmAP__ZJNftn.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/eulera50.DNtZz4PJ_2vqlgq.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/eulera100.CvkXzr3k_XOq4R.jpg) ### Karras Variants Variants with the word `Karras` (or `K`) ([paper](https://arxiv.org/abs/2206.00364)) refer to work led by Nvidia engineer Tero Karras. This process introduces a series of improvements in some samplers, achieving **improved efficiency** in both the quality of the output and the computation required for sampling. Some samplers using these changes are: `LMS Karras`, `DPM2 Karras`, `DPM2 A Karras`, `DPM++ 2S A Karras`, `DPM++ 2M Karras` or `DPM++ SDE Karras`. ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2mk10.DqQv057i_13Jnbp.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2mk30.CPb17b5i_Z1R96DI.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2mk50.DK8VCqze_Z1ktOV8.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2mk100.B4jg5XS7_CKvtL.jpg) The `Karras` version has the advantage of offering better results in a reduced number of steps as can be seen in the following example: ![Distorted image of a black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2m10.CkLTadxD_aXY9V.jpg) *DPM++ 2M - 10 steps* ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmpp2mk10.DqQv057i_13Jnbp.jpg) *DPM++ 2M Karras - 10 steps* If you use a high number of steps you will have a hard time seeing the difference. ### Stochastic variants The `SDE` ([paper](https://arxiv.org/abs/2011.13456)) variants use stochastic differential equations. Without going into further detail, using this type of differential equations allows to model the noise in a more sophisticated and accurate way, using information from previous steps, which in principle would generate **higher quality images in exchange for being slower**. Being stochastic, it never converges, so the higher the number of steps they do not offer higher quality, but rather more variations, just like ancestral samplers. At the date of publication of this article we have `DPM++ SDE`, `DPM++ 2M SDE`, `DPM++ SDE Karras` and `DPM++ 2M SDE Karras`. ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmppsde10.YBAj4T_s_ZoCVVB.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmppsde30.3A3AnAkS_1VgyJH.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmppsde50.AZLoNzY6_1nd9mE.jpg) ![Black sports car in the desert running away from a nuclear explosion](https://www.felixsanz.dev/assets/dpmppsde100.Z5o-R6nh_ePtjt.jpg) ## What is the best sampler in Stable Diffusion? Is a Ferrari or a Jeep better? Well it depends on whether you're going off-road, doesn't it? Depending on what you need it's better to use one type of sampler or another. With the above information I hope it will be easy to choose, but here are some hints. ### Image quality If you are looking for quality it is a good idea to pursue convergence. That's the point at which you get the highest quality. If you don't want to sacrifice too much generation speed, forget about samplers like `DDIM` that need hundreds of steps to converge. `Heun` and `LMS Karras` offer good results but it is better to use `DPM++ 2M` or its `Karras` version. You can also try `DPM adaptive` if you are not in a hurry, or `UniPC` if you are. With these samplers mentioned above you will get good results in 20-30 steps although it doesn't hurt to try a few extra steps. ### Generation speed If you are testing prompts you don't want to spend so much time waiting for results. In this case and where you are not looking for maximum quality but to test changes quickly I recommend using `DPM++ 2M` or `UniPC` with a small number of steps. With just 10-15 steps you will get a very decent image. If you don't care about reproducibility you also have `Euler A`, a fast and good quality ancestral sampler. My favorite sampler! ### Creativity and flexibility This section is reserved exclusively for ancestral and stochastic samplers. They don't offer bad quality nor are they slow, they are just different. The problem or advantage (depending on how you look at it) of these samplers is that if you have an image generated in 40 steps, having done it in 50 steps can make the image better or worse. You will have to test continuously. And this lottery makes these samplers more creative since you can always change the number of steps to obtain small variations. Of course here `Euler A` and `DPM++ SDE Karras` stand out. Try generating images in 15 steps, 20 steps, 25 steps... and see how the result changes.