Skybox AI to Unreal Engine

From the very beginning of conceptualising and crafting the story of “The Light Within,” our goal was to explore and experiment with AI tools to streamline our 3D workflow. The concept art generated by AI was awe-inspiring, but the challenge was how to translate that magic into a 3D scene seamlessly.

To overcome this, I initially ventured into creating a 2.5D scene using projection mapping with an AI-generated concept art image of a lighthouse tower. The 3D modelling tool that I used for this was Autodesk Maya. Later this scene was taken into Unreal Engine to enhance its atmosphere. This technique proved to be lightweight in terms of geometry and textures, which is ideal for real-time assets. However, it had its limitations. While 2.5D creates the illusion of depth, it may not always faithfully represent objects in true three dimensions and could encounter issues with complex details and artefacts.

To enhance the visual quality without distortion, I sought to explore the same projection mapping technique but with a 360 image. After testing various tools like Midjourney and Stable diffusion, I found Skybox AI to be the optimal choice for generating accurate 360 images. Using a correct 360 image enabled us to create a projection without a distortion. which you do get with projecting a normal image.

Using the 360 projection generated by Skybox AI within Autodesk Maya, I constructed the scene’s geometry based on the projected texture. This approach enabled the creation of a lightweight 3D environment that perfectly matched the atmosphere of the original 360 image. Leveraging a spherical projection material in Maya, I seamlessly applied and transferred the projected textures onto the geometry, simplifying the modelling process while preserving the desired ambiance.

To further enhance the scene, I turned to Adobe Substance Sampler 3D, a remarkable tool empowered by AI technology. It efficiently transformed the flat textures into Physically Based Rendering (PBR) materials, ensuring precise and realistic lighting within the scene. Subsequently, I meticulously refined and polished these materials using Substance Painter.

Finally, the scene, now enriched with accurate geometry, projected textures, and realistic PBR materials, was brought into Unreal Engine. With Unreal Engine’s capabilities, I transformed the environment into an immersive, interactive space that was ready to be experienced and explored!

(The titles of grandeur – by ChatGPT :))