AI Tutorial | Incorporate your art style in AI-generated Art
Kiki Wu
Like this project
Posted Apr 5, 2023
Learn how to incorporate your art style into AI art by using image prompts and custom model training.
When generating AI art, you can input prompts to specify the subject, aspect ratio, and art style. For instance, you could type in "impressionist portrait avatar" or "a painting about a violet garden by Vincent Van Gogh". Some AI tools even allow you to input artists' names to refine the results for accuracy.
However, there are lingering copyright concerns associated with AI tools like Midjourney and Statble Diffusion. These tools have been criticized for potentially being trained on a massive dataset of five billion images scraped from the web without obtaining consent from the original artists. As a result, it has led me to consider creating AI art with my own unique style, rather than borrowing from other artists' works.
But before delving into the technical aspects, let's first explore how we typically describe an artwork and analyze its styles. This will help identify relevant keywords that can be used as prompts for AI image generators.
Describe and analyze an artwork
1. Visual Description
the visual description of an artwork could be started with something simple. Go to a museum, and by looking at the basic information on the wall label, you know the artwork’s medium, dimension, and subject.
Further, you describe the main subjects — Landscape, portrait, figurative, animal, still life, architecture, abstract objects…. Use different adjectives to describe its color tone and textures, or think about the process of making this artwork. You can also do a formal analysis and describe the following elements in detail:
Visual elements: What attracts your eye first?
Composition: How are the components arranged? symmetrical/asymmetrical?
Line: Contour lines, widths of lines, lines of direction or movement
Material
Texture: rough/smooth, soft/hard
Colors: hue, value, saturation, primary/secondary, complementary, warm/cool?
Light and Shadow: contrast, shading, cross-hatching
Shape: organic/geometric, naturalistic / abstract
Technique
The tip is to describe the piece as if your audience has not seen it. Focus on description rather than interpretation.
2. Stylistic Analysis
In her blog “Writing About Art”, Marjorie Munsterberg quotes philosopher Berel Lang and concludes the term “style” refers to the resemblance works of art have to one another:
Enough visual elements must be shared by enough works to make their combination distinctive and recognizable to a number of people.
The idea must convey meaning to enough people to become widely used.
Style is an idea that has been created by someone rather than a quality that belongs to the objects.
Style is a key factor that sets an artist's work apart from others and can reveal commonalities within their body of work. For example, Van Gogh's unique use of paint and brushstrokes distinguish his art from others. The style also helps identify the authorship of specific artworks.
When conducting a stylistic analysis, it's useful to ask questions and make comparisons between different artworks. Think about the following questions:
Does the artwork exhibit characteristics of a particular artistic style, movement, or time period? Can you discern any influences from contemporary trends, fashions, or ideologies? How can you identify these elements within the work?
Are there any relevant connections or comparisons that can be made between this artwork and others? Have other artists explored similar subject matter in a similar manner, and did this occur prior to or after the creation of this artwork?
If you are interested in a detailed guide to artwork analysis, please read this article:
Find the keyword of your art
Now you should have a good understanding of how to describe and analyze your artwork using formal analysis and stylistic analysis.
But if you're still having difficulty using the appropriate phrases and adjectives to describe your art, I recommend showing your artwork to your friends and paying attention to the way they describe it. Alternatively, you can leverage the power of AI (once again!) by uploading your artwork to the Image to Text AI tool to uncover potential keywords.
For example, below is an illustration I draw
My description is:
đź’ˇa digital illustration of a goddess seated in a lotus position, with a symmetrical composition. The entire illustration is adorned with glowing contour lines and smooth saturated gradient colors. The goddess is filled with a rainbow gradient tone, and sacred organic lines connect seven chakras within her body. A new moon is depicted right above her head, and a pink-purple lotus is situated underneath her feet. The goddess's hair is long, reaching down to the river in the bottom part of the illustration. The background features leaves of a sacred fig tree growing under the blue-layered clouds that energize the relaxed body of the goddess. Several stars shine in the cloud.
My creative process draws primarily from psychedelic art, meditation, and the Zen lifestyle, so I consider these terms crucial to the context of this artwork:
Zen art, energy painting, visionary, 7 chakras, and psychedelic, lotus.
Use Image to Text AI tool
Then I upload my image to Clip-Interrogator, a prompt engineering tool that combines OpenAI's CLIP and Salesforce's BLIP to optimize text prompts to match a given image.
Here is the output:
đź’ˇa painting of a person sitting in a lotus position, digital art, inspired by Alex Grey, shutterstock, martial artist dryad, background image, iridescent cybernetic processor, ufotable art style, twin souls, anime art nouveau cosmic display, bright glowing translucent aura, symmetrical tarot illustration, fully body photo, medical drawing, banner, ultra high res
Umm 🤔You might wonder how this is so different from my original description.
My initial depiction focuses more on "what the image depicts" than "how the image looks like", as I describe the visual elements and their arrangement in the artwork. However, when it comes to AI-generated artwork, the goal for AI is to identify similarities and differences between styles to achieve accuracy.
Therefore, I break down the output in the following way:
The main visual elements of the image:
a painting of a person sitting in a lotus position
bright glowing translucent aura
Full body photo
What does this image look like and the possible styles related to this image (interpreted by AI)
digital art, inspired by Alex Grey
martial artist dryad
ufotable art style
anime art nouveau cosmic display
symmetrical tarot illustration
medical drawing
Others:
Banner
Ultra high res
I googled those words and entered them into Midjourney to see if they were related to my art. In my opinion, ufotable art style is a bit unrelated, while anime art nouveau cosmic display and bright glowing translucent aura seem closer to my illustration.
So let’s keep those style words, and later I'll apply them in Midjourney:
glowing contour lines, anime art nouveau cosmic display, medical drawing, bright glowing translucent aura, organic lines, smooth saturated gradient colors, visionary, Zen art, energy painting, visionary, 7 chakras, psychedelic
Implement your style in AI-generated art
1. Image Prompt + Text Prompt
The simplest way to begin incorporating your style into AI art is to upload an image as a reference.
Image from Midjourney User Guide
In Midjorney, you can upload your image to Discord and use the image weight parameterÂ
--iwto adjust the importance of the image vs. text portion of a prompt.You can find the example in the User Guide. The higher the weight number, the greater impact the image prompt will have. By default, the weight is set to 1.
Screenshot from Midjourney User Guide
I uploaded the person in lotus position illustration to the Midjourney Discord server and set the weight of the image prompt to 2 (--iw 2).
The text prompt was the same as my previous description: a painting of a person sitting in a lotus position. I wanted to see how Midjourney would incorporate my original art into its imagination.
I also try to use the Image prompt + the long text prompt I get from Clip-Interrogator to enhance the overall style, the result is quite impressive!!
Now we have an idea of how the image prompt and style words impact the outcome.
Let's try it with a different subject. For instance, I enter "A painting of a dancing angel" or "A painting of a mermaid under the sea" in the text prompt to see how the AI generates the corresponding artwork.
Prompt: a painting of an angel dancing in rose garden
Prompt: a painting of a woman standing in front of a bunch of flowers
And feel free to change image prompt weights and other parameters like aspect ratio and seed to see different results!
2. Train custom model using Leonardo AI
The advanced method is to train your custom model using Leonardo AI.
Make sure you are whitelisted, if not, you should register first.
Launch the Leonardo AI app, and click Training & Datasets > Create a new dataset.
Check this model fine-tuning guide before uploading your image and training the model.
It's recommended to use between 8-15 images, with a minimum of 5 and a maximum of 30. The ideal number of images can vary depending on what you're training the model to do. Generally, training a specific object doesn't require as many images as training a particular style, which may require more images.
Click Training & Datasets
Create New Dataset
Make sure the style of your images is consistent!!
In my case, I want to train AI to recognize my mandala art style, so I upload images that possess similar characteristics: mandala art featuring organic high saturated lines and dark background.
Upload images as Dataset
Then press the Train Model button and finish the training setting:
Select training resolution. The default setting is 512 x 512, but you can increase it to 768 x 768 if you prefer.
Set Category.
Select Base Model. Either Stabld Diffusion 1.5 or 2.1.
Enter the instance prompt. Simply type in a single word such as "building," "character," or "animal," which will later be used to generate artwork using this training model. As an example, I used "mandala" as my instance prompt.
The model training process typically takes between 10 to 20 minutes. Once the training is complete, you will receive an email notification.
Next, navigate to the AI Image Generation Tool and begin typing in your prompt to generate artwork using your custom model.
Click AI Image Generation
Select your Custom Model
Don't forget to include the same instance phrase in your prompt:
This is the final result, and I didn’t anticipate the AI-generated art would turn out so psychedelic!
Final results
Find and love your own art style
Although I enjoy blending my art style with AI's imagination, establishing my own distinct style used to be a challenge for me.
I understand artists who are just starting their careers may still be in the process of discovering their unique art style and developing a robust portfolio that sets them apart from the competition. If you're in this stage, here are some tips to help you identify and embrace your personal artistic style.
1. Create! Create! Create
As you produce more art, you gain insight into your strengths and the specific messages you aim to communicate through your work.
2. Learn from artists you admire
I curate many collections of artworks that catch my eye on Pinterest and later I can study the mediums and techniques utilized.
I enjoy experimenting with incorporating these techniques into my own work to explore new effects. However, it's important to note that while drawing inspiration from other artists is perfectly fine, copying their works is a big NO!!
My Pinterest idea board
3. Analyze your own artwork
Take time to look at your artwork and analyze them. It's important to evaluate what you like and don't like it. By doing so, you can identify areas for improvement and refine your artistic technique or creative process accordingly.
4. Consider the context and story behind it
While it's true that we can explore various techniques and produce art on any subject, I personally believe that the most impactful art is one that resonates with our hearts.
It's not just about creating something visually appealing; rather, it's about conveying a meaningful story that has the potential to make an impact on the world. Consider the story your artwork conveys and how it can have a significant impact on both you and your audience.