Create immersive artwork using AI art and STYLY VR
Kiki Wu
World-building, and creating surreal scenes for virtual events and exhibitions have been crucial parts of my artistic practice, as it enables me to bridge real-life experiences with the digital realm. This process of creating digital art in cyberspace is integral to my work, as it allows me to translate tangible moments and emotions into immersive experiences.
In this article, I will outline techniques for crafting immersive, surreal scenes using the latest AI image generators and VR platforms. The techniques presented are beginner-friendly and do not require any coding skills!
What is immersive artwork?
Although immersive artwork is commonly associated with virtual reality nowadays, the concept of immersive experiences can encompass any type of experience that fully engages the viewer's senses. This can include XR artworks or large-scale interactive installations like what Teamlab is doing, which transports the viewer into an entirely different world or environment, where they can interact with the artwork/content using all of their senses, not limited to visual senses. Here I will concentrate solely on art experiences that are based on virtual reality and web browsers.
What is driving the growing popularity of immersive artwork?
Research has shown that humans tend to learn better and retain more information in immersive environments. This is because immersive experiences are more engaging and memorable, allowing the viewer to fully interact with the content and absorb the information more effectively. This makes immersive art an increasingly important tool for education, especially in fields such as science, history, and the arts.
Innovative ways of showing art since pandemic
The global pandemic happened in 2020 may also be a contributing factor to the growing popularity of immersive artwork. With physical exhibition venues being shut down, artists have had to find new ways to present their artwork to the public. XR experiences that can be hosted online and viewed repeatedly have increased the accessibility of art while also promoting the healing effects of immersive virtual worlds that can help reduce people's anxiety during long periods of lockdown. This has made immersive artwork a particularly attractive medium for artists seeking to engage with audiences in new and innovative ways, while also providing a means of artistic expression during times of social distancing and isolation.
Is creating immersive art challenging?
Creating immersive artwork often requires possessing 3D modeling skills, as well as knowledge of game engines such as Unity and Unreal, or WebVR library like A-Frame, which are used to set up the virtual reality world and create interaction within the artwork.
Emergence of Metaverse platform and AI Image Generators
Fortunately, in 2023, artists who lack coding skills can still create immersive scenes using metaverse platforms such as STYLY, Spatial, they are mostly web browser based editors which enable creators to creating immersive artworks by simply uploading 3D models and other media.
Moreover, with the increasing popularity of AI image generators and ongoing improvements in this technology, artists can easily draft concepts for their artwork and build beautiful prototypes using these tools.
Tools for creating immersive AI art
- Blockade Labs: An AI generator that creates 360° panorama images for use as skybox textures.
- MidJourney: a well-known AI image generator that is highly regarded for its highly detailed, vibrant art style and accuracy. While it is not specifically designed to generate 360-degree panoramic images by default, it is possible to achieve this by using prompts such as "seamless equirectangular 360° panorama photo - - ar 2:1”, I will explain the prompt later in this article.
- Adobe Photoshop: Retouch the AI Image
- STYLY: no-code XR experience builder. It supports web browser, VR device and mobile AR!
- Beatoven: AI music maker
Start Creating✨
1. Generate AI 360° panorama image from Blockade Labs
Go to Blockade Lab and type in prompts to create AI 360° panorama image. We will use this images as the main scene in our immersive artwork.
It will take around 15~20 seconds to generate the image.
💡Here is my prompt: Inside mystical mushrooms, curly vines and blue lotuses, psychedelic flowers, trippy neon color, floating stars.
Explore Different Styles
You have the option to choose from 16 different art styles. I prefer the Anime Art Style, Dreamlike and Surreal, as they align with my affinity for psychedelic art, trippy color schemes, and gradient visuals. However, feel free to explore and experiment with other styles and type in extra style words until you find the one that resonates with your unique creative vision.
For your reference, take a look at these mushrooms scenes of different styles:
Anime Style
Fantasy Landscape
Dreamlike
Surreal
💡Struggle with prompts? Don’t worry, you can find inspirations on PromptHero
Option 2: Use Midjourney to create 360° panorama image
Don’t like art style the built in Blackade Lab? Let’s jump to MidJourney.
Pay attention to the prompt here:
a seamless equirectangular 360° panorama photo of Your Imagination, Art Styles - - ar 2:1
The first part “A seamless equirectangular…….” is crucial because it defines the image type and format, and the last “- -ar 2:1” is the aspect ratio of your image. Remember to include these two parts in your prompt!
💡New to MidJourney? Check the user guide here!
After you receive the output from MidJourney, download the image and upload it to Blockade Labs to check for any potential bugs.
It's important to note that while you can input prompt like "seamless," MidJourney is not specifically designed for generating 360-degree panorama images. As a result, there may still be visible seams in your image even with that prompt.
Fix the Seam using Photoshop Spot Healing Brush Tool
So let’s fix the seam! Go to photoshop and import your AI image, cut the image in half and switch both sides. Move the right side to the left and the left side to the right.
Your resulting image should look like this:
You see seam! Let’s get rid of it using Spot Healing Brush tool (Shortcut is J)
Now your AI image is ready! Hit the File > Export.
2. Create a Skybox object of your AI image
In order to make the 360° panorama image the primary scene, I will make it as the texture for a skybox object.
A skybox object is a technique in computer graphics that creates a realistic outdoor environment, like a sky or distant scenery. To create this effect, I wrap the AI image as image texture around a sphere in Blender and export a .glb file, upload it as a skybox object to STYLY.
Use Skybox template from Spatial
Luckily, the social VR platform Spatial.io offers a skybox template that streamlines the process and eliminates the need to use Blender! By following the provided steps, creators can easily generate their own skybox without any hassle.
- Click here to download Skybox template
- Unzip the file and replace the texture.jpg file with your AI 360° image. Make sure the name of the replaced file is still “texture.jpg”.
- Drag all three files simultaneously (Spatial Skybox Sample.gltf, Spatial Skybox Sample_data.bin, texture.jpeg) into http://glb-packer.glitch.me/ to make a glb file that can be uploaded.
❓Why do I choose STYLY instead of Spatial to create VR space? The main reason for my switch from Spatial to Styly is because Spatial does not support audio file uploads. However,I believe that audio, including background music and sound effects, contributes much more to immersive experiences than we realize.
3. Upload the Skybox object to STYLY
Now create a account and login to STYLY. Click your profile avatar and click “Studio”
Then click “CREATE SCENE” > VR Scene Template and input your scene title.
You should see a default scene like this, now click the gear button on the right side of the [Skybox]Default and Delete it. We will upload our own skybox.
On the top menu bar, click the Image+ icon, and select My Uploads > 3D Object. Upload your Skybox object.
Now you can view your AI art in an immersive way!
Next is to add light to your scene.
Click the Image + again, and select Environment > Light
I simply add a RooftopNight environmental light set, to make the environment looks brighter.
4. Add AI music
Go to Beatoven to generate some music for our immersive scene.
After setting up the title, duration, music genre and mood, Beatoven AI will compose the music immediately.
You should see the interface like this. Feel free to select your favorite track options, they are basically different music patterns.
Then click the top right button to download the track!
5. Final Step: Add music to the immersive scene
Go back to STYLY and upload your AI music!
Because I want the background music to continuously play on repeat, so I select Music Player(loop).
BOOM! The scene is now complete! You can immerse yourself in the art using a web browser or put on a VR headset to experience it.
👉Enjoy my trippy VR world here
Advanced techniques
Congratulations, your immersive scene is now live! To enhance the interactivity and liveliness of the immersive artworks, you might wan to add the following elements:
- Add 3D objects and animations.
- Apply visual effects, such as particle effects and camera filters.
- Create interactive elements for users to engage with, for example, some clickable objects or portals that encourage viewer to click and teleport to different worlds.
- Add movement to your AI image to make it more dynamic. Below video shows that I experiment with Runway ML to create moving image interpolation featuring four seasons of Mount Fuji.
You can experience this immersive scene here. It will take few seconds to load the scene, be patient!
AI technology frees our imaginations, I hope you enjoy creating art!!
