The Prompter #005
08/22/22-09/02/22
welcome to The Prompter #005!
this is a weekly newsletter written by krea.ai to keep you updated with the latest news around prompt engineering.
sorry for the delay, it has been a busy week!
📰 AI news
Stable Diffusion Public Release
finally! now we all have access to Stable Diffusion and we can use it locally in our own GPUs, in Google Colab, create our own applications with it, or use the new Stability AI API.
the release of the weights and implementation is under a license that allows for commercial and non-commercial usage of them.
they also included an AI-based Safety Classifier by default in the overall software package—although it can be adjusted if we want to.
the Stability AI API service was also released, they charge $0.1 per standard image (512x512 and 50 steps) and $0.3 in case the user does not allow Stability AI to store their API requests and response content.
[code and weights][optimized colab][hf spaces demo]
CLIP Image Embeddings with Stable Diffusion
Justin Pinkey shared some of his results after fine-tuning Stable Diffusion to generate results with images as input instead of texts.
this application is very similar to DALL-E 2 variations, where you can input an image and produce variants of it that preserve its style and content.
his implementation consisted of substituting the text encoder from Stable Diffusion with a CLIP image encoder and re-training the model using images as input.
amazing work!
Deforum Diffusion
deforum.art is a community of AI image synthesis developers, enthusiasts, and artists that work daily on improving their free Stable Diffusion notebook.
this is probably the best Google Colab notebook for using Stable Diffusion, it allows you to use any version of the model and to even generate animations with it!
[link][google colab][discord]
Hyper-parameter Optimization of Stable Diffusion Parameters
Peter Baylies shared some of his experiments where he visualized the effect of the scale and steps parameters in Stable Diffusion.
these kinds of studies are extremely useful to understand the effect of each parameter of the model in the resulting generations.
[link]
ERNIE-ViLG is out
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation is a work that Baidu published in December last year.
it consists of a 10 billion parameter autoregressive model that uses a VQGAN; very similar to Parti or dalle-mini.
[project website][paper][hf spaces demo]
Awesome AI Art Image Synthesis
@altryne created an awesome GitHub repository with a ton of information about prompt engineering tools, Google Colabs using Stable Diffusion, generative AI models, prompt helpers, theory resources, and more.
this is a must-read for anyone starting to use text-to-image AI models.
[link]
About: Image Synthesis Style Studies
the team behind the Art Studies just wrote down some background info, and a FAQs about their work.
this db is not only useful as a source of inspiration, but necessary to understand the capabilities and possibilities of text-to-image models.
[link]
🛠️ tools for prompting
@hlky__ created a browser tool based on Gradio library for Stable Diffusion that allows us to use the model locally with a visual interface [link]
Stephen Young released Prompt Parrot 2.1! this notebook allows you to train a prompt generator using distilgpt2 with your own prompts (or some of the >148 pre-loaded signature Kyrick prompts) and use it with Stable Diffusion. [link]
Grant Davis just turned Noam Chomsky into Gandalf with Wand, his new iOS native app that uses Stable Diffusion for editing images from text [link].
@nousr_ showcased a plugin for using Stable Diffusion with Google Colab in GIMP, really neat [link].
Antonio Cao shared his WIP with Ando AI, a Figma plugin that uses Stable Diffusion to help designers turn their ideas into realistic images in seconds [link].
Dreamtweet App is a Twitter bot that will reply with a Stable Diffusion generation from any of your tweets by just including the keyword “Dream” and mentioning @dreamtweetapp [link]
@altryne created an implementation of @hlky_’s web UI that works on Google Colab [link]
🎨 AI Art
a great example of what you can create with deforum.art’s notebook by @pharmapsychotic
@edbyus used init images with Stable Diffusion to turn his renders into different materials.
awesome use of DALL-E 2 + DAIN (frame interpolation) + EBSynth (optical flow) by @karenxcheng.
another similar use of outfit swapping with DALL-E 2 by @paultrillo
🦾 krea updates
we spend the last 5 days designing and implementing our new version of krea.ai.
with this new tool, you can explore >10M generations and store them in your own collections.
we explained all its features about it in our last tweet.
we have extracted >10M generations from stable diffusion.
our search bar recommends prompts while you search.
you create and organize your own collections of prompts.
use our sidebar to browse hundreds of modifiers from open-prompts.
we have extracted modifiers from each prompt, you can search them by just clicking them!
try it out and share your thoughts, ideas, and feedback with us!
we just opened out Discord too, come say hi!