Introducing Stable Diffusion 3.5
Updated October 29th with release of Stable Diffusion 3.5 Medium
Key Takeaways:
Today we are introducing Stable Diffusion 3.5. This open release includes multiple model variantsincluding Stable Diffusion 3.5 Large and Stable Diffusion 3.5 Large Turboand as of October 29thStable Diffusion 3.5 Medium.
These models are highly customizable for their sizerun on consumer hardwareand are free for both commercial and non-commercial use under the permissive Stability AI Community License.
You can download all Stable Diffusion 3.5 models from Hugging Face and the inference code on GitHub now.
Today we are releasing Stable Diffusion 3.5our most powerful models yet. This open release includes multiple variants that are customizablerun on consumer hardwareand are available for use under the permissive Stability AI Community License. You can download Stable Diffusion 3.5 Large and Stable Diffusion 3.5 Large Turbo models from Hugging Face and the inference code on GitHub now.
In Junewe released Stable Diffusion 3 Mediumthe first open release from the Stable Diffusion 3 series. This release didn't fully meet our standards or our communities’ expectations. After listening to the valuable community feedbackinstead of a quick fixwe took the time to further develop a version that advances our mission to transform visual media.
Stable Diffusion 3.5 reflects our commitment to empower builders and creators with tools that are widely accessiblecutting-edgeand free for most use cases. We encourage the distribution and monetization of work across the entire pipeline - whether it's fine-tuningLoRAoptimizationsapplicationsor artwork.
What’s being released
Stable Diffusion 3.5 offers a variety of models developed to meet the needs of scientific researchershobbyistsstartupsand enterprises alike:
Stable Diffusion 3.5 Large: At 8.1 billion parameterswith superior quality and prompt adherencethis base model is the most powerful in the Stable Diffusion family. This model is ideal for professional use cases at 1 megapixel resolution.
Stable Diffusion 3.5 Large Turbo: A distilled version of Stable Diffusion 3.5 Large generates high-quality images with exceptional prompt adherence in just 4 stepsmaking it considerably faster than Stable Diffusion 3.5 Large.
Stable Diffusion 3.5 Medium: At 2.5 billion parameterswith improved MMDiT-X architecture and training methodsthis model is designed to run “out of the box” on consumer hardwarestriking a balance between quality and ease of customization. It is capable of generating images ranging between 0.25 and 2 megapixel resolution.
Developing the models
In developing the modelswe prioritized customizability to offer a flexible base to build upon. To achieve thiswe integrated Query-Key Normalization into the transformer blocksstabilizing the model training process and simplifying further fine-tuning and development.
To support this level of downstream flexibilitywe had to make some trade-offs. Greater variation in outputs from the same prompt with different seeds may occurwhich is intentional as it helps preserve a broader knowledge-base and diverse s in the base models. Howeveras a resultprompts lacking specificity might lead to increased uncertainty in the outputand the aesthetic level may vary.
For the Medium model specificallywe made several adjustments to the architecture and training protocols to enhance qualitycoherenceand multi-resolution generation abilities.
Where the models excel
The Stable Diffusion 3.5 version excels in the following areasmaking it one of the most customizable and accessible image models on the marketwhile maintaining top-tier performance in prompt adherence and image quality:
Customizability: Easily fine-tune the model to meet your specific creative needsor build applications based on customized workflows.
Efficient Performance: Optimized to run on standard consumer hardware without heavy demandsespecially the Stable Diffusion 3.5 Medium and Stable Diffusion 3.5 Large Turbo models.
We took a look at the hardware compatibility for running Stable Diffusion 3.5 Medium alongside other open-image base models. This model only requires 9.9 GB of VRAM (excluding text encoders) to unlock its full performancemaking it highly accessible and compatible with most consumer GPUs.
Diverse Outputs: Creates images representative of the worldnot just one type of personwith different skin tones and featureswithout the need for extensive prompting.
Versatile Styles: Capable of generating a wide range of s and aesthetics like 3Dphotographypaintingline artand virtually any visual imaginable.
Additionallyour analysis shows that Stable Diffusion 3.5 Large leads the market in prompt adherence and rivals much larger models in image quality.
Stable Diffusion 3.5 Large Turbo offers some of the fastest inference times for its sizewhile remaining highly competitive in both image quality and prompt adherenceeven when compared to non-distilled models of similar size
Stable Diffusion 3.5 Medium outperforms other medium-sized modelsoffering a balance of prompt adherence and image qualitymaking it a top choice for efficienthigh-quality performance.
The Stability AI Community license at a glance
We are pleased to release this model under our permissive community license. Here are the key components of the license:
Free for non-commercial use: Individuals and organizations can use the model free of charge for non-commercial useincluding scientific research.
Free for commercial use (up to $1M in annual revenue): Startupssmall to medium-sized businessesand creators can use the model for commercial purposes at no costas long as their total annual revenue is less than $1M.
Ownership of outputs: Retain ownership of the media generated without restrictive licensing implications.
For organizations with annual revenue more than $1Mplease contact us here to inquire about an Enterprise License.
More ways to access the models
While the model weights are available on Hugging Face now for self-hostingyou can also access the model through the following platforms:
Our commitment to safety
We believe in saferesponsible AI practices and take deliberate measures to ensure Integrity starts at the early stages of development. This means we have taken and continue to take reasonable steps to prevent the misuse of Stable Diffusion 3.5 by bad actors. For more information about our approach to Safety please visit our Stable Safety page.
Coming soon
We will also launch ControlNets soonproviding advanced control features for a wide variety of professional use cases.
We look forward to hearing your feedback on Stable Diffusion 3.5 and seeing what you create with the models. You can share thoughts directly with us through this form.
To stay updated on our progress follow us on XLinkedIn, Instagramand join our Discord Community.
