Skybox AI to Unreal Engine

Jun 14

Skybox AI to Unreal Engine

From 360 image to interactive playable environment or "Unleashing AI Magic: How to transform Concept Art into 3D Realms?"

/5 Author: Eva V & CHATGPT

Unleashing AI Magic: How to transform Concept Art into 3D Realms?

From the very beginning of conceptualising and crafting the story of “The Light Within,” our goal was to explore and experiment with AI tools to streamline our 3D workflow. The concept art generated by AI was awe-inspiring, but the challenge was how to translate that magic into a 3D scene seamlessly.

Unleashing Illusion: Transforming AI Concept Art into 2.5D Marvels

To overcome this, I initially ventured into creating a 2.5D scene using projection mapping with an AI-generated concept art image of a lighthouse tower. The 3D modelling tool that I used for this was Autodesk Maya. Later this scene was taken into Unreal Engine to enhance its atmosphere. This technique proved to be lightweight in terms of geometry and textures, which is ideal for real-time assets. However, it had its limitations. While 2.5D creates the illusion of depth, it may not always faithfully represent objects in true three dimensions and could encounter issues with complex details and artefacts.

Unleashing the Magic: Distortion-Free 360 Projection with Skybox AI

To enhance the visual quality without distortion, I sought to explore the same projection mapping technique but with a 360 image. After testing various tools like Midjourney and Stable diffusion, I found Skybox AI to be the optimal choice for generating accurate 360 images. Using a correct 360 image enabled us to create a projection without a distortion. which you do get with projecting a normal image.

360 Projection Magic: Creating seamless 3D scenes with Autodesk Maya

Using the 360 projection generated by Skybox AI within Autodesk Maya, I constructed the scene’s geometry based on the projected texture. This approach enabled the creation of a lightweight 3D environment that perfectly matched the atmosphere of the original 360 image. Leveraging a spherical projection material in Maya, I seamlessly applied and transferred the projected textures onto the geometry, simplifying the modelling process while preserving the desired ambiance.

Unleashing AI Brilliance: Elevating scenes with substance sampler 3D's realistic materials

To further enhance the scene, I turned to Adobe Substance Sampler 3D, a remarkable tool empowered by AI technology. It efficiently transformed the flat textures into Physically Based Rendering (PBR) materials, ensuring precise and realistic lighting within the scene. Subsequently, I meticulously refined and polished these materials using Substance Painter.

Unreal Enchantment: Crafting an immersive interactive experience with transformed scenes

Finally, the scene, now enriched with accurate geometry, projected textures, and realistic PBR materials, was brought into Unreal Engine. With Unreal Engine’s capabilities, I transformed the environment into an immersive, interactive space that was ready to be experienced and explored!

(The titles of grandeur – by ChatGPT :))