FORCES OF NATURE

UNREAL FELLOWSHIP: WORLD BUILDING (EMEA)

The Unreal World Building Fellowship is a 3.5 week intensive course that focuses on building environments in Unreal Engine, with a special eye on telling a compelling story. This page shows the building process and end result of the world I built, as I witnessed about a hundred super talented artists doing the same thing. My main goal was to use the tools available in Unreal Engine 5.1 to build a believable large open world using existing assets from the Quixel Megascans library, Unreal Marketplace and TurboSquid.

CHALLENGE

Early 2023 I was invited by Epic Games to be a part of the acclaimed “Unreal Fellowship: World Building EMEA”, where I received 3.5 weeks of super intensive training from some of the best realtime 3D professionals in the world. The overarching mission of the course was to build an engaging environment following a random ‘dice roll’ in the Fellowship World Building Scene Configurator, that would give each fellow a unique set of attributes and modifiers for building their unique worlds.

The ‘dice role’ resulted in the following set of attributes and modifiers for building my world:

Setting: Pure Landscape / No Structures > Swamp
Genre: Action / Thriller
Visual Style: Realism / Hyper Realism > Color Palette - Black and White; Applied to - Multiple Subjects
Environment Conditions: Tornado / Hurricane > Time of Day - Dawn; Climate Type - Water World
Surprise Twist: Electro Magnetic

After playing with the flexibility we had in locking in our attribute options, I decided to build a big tropical lagoon with a mangrove swamp. Even though the sheer number of trees in such a large landscape could potentially pose a challenge to my computer, this was the ideal setting to dive deep into Unreal’s set of world building tools and experience its strengths and weaknesses.

Flythrough video (with audio) of the environment I made during the World Building Fellowship (February 2023)

CONCEPT AND MOODBOARD

In the first week we were asked to pitch our environment concept and back story and support it with a mood board and possibly a sketch or a blockout. An important part of the whole fellowship experience is to share the world building process with other fellows and give each other feedback. This took place on a ‘smaller’ level in scrums with your team of 13 fellows, lead by a mentor and a technical assistant, and on a ‘bigger’ level in weeklies with everyone in the fellowship.

In order to imagine what my world should look like, I started looking for photo references and quickly locked into the idea of a mangrove swamp because I love mangrove trees. When I found pictures of the Asmat tribe in Indonesia that uses black and white body paint, live around mangrove swamps and do spear fishing all my attributes and modifiers quickly fell into place. This is the backstory I came up with:

FORCES OF NATURE
The stage is set around a coastal mangrove swamp in eastern Indonesia. The canoe of a tribal fisherman is rowing out of the mangrove forest at dawn. The camera is positioned on the front of the boat and witnessing the beautiful clear waters with fish and coral.
Then we see the canoe from behind rowing out of the mangrove and we notice a threatening storm (hurricane) is coming in. It's a magnificent sight because the (dynamic) dark clouds are only covering one side of the sky, and the sun is rising above the horizon just below. The tribal fisherman with black and white marks on his face and spear decides to go fishing anyway.
We see the fisherman standing up in the boat besides some small remainders of the mangrove forest against the dark and dangerous sky. The fisherman jumps up and throws the spear in the water while he puts his entire body behind it. But as soon as the spear hits a fish in the water, the sky explodes in a spectacle of lightning within the dark clouds.
The sequence ends with an aerial view looking over the mangrove forest towards the ocean and the dark clouds with lightning above a rising sun.

To visualize the story I collected the most interesting references in a mood board, and drew a rough top down sketch of the environment with some initial ideas for interesting camera angles. This gave me an good starting point which parts of the environment would become most important.

One of the things my team pointed out to me after the pitch, was to try to tell the story through the environment first (one of the most important aspects of world building), and only add characters later when you still have time. This put me on the right track to focus on the most important aspects first, as there were enough challenges ahead already in the short time we had to create our worlds.

ASSET Search

The second part of my research was to get a clear overview of what type of assets (especially trees and plants) would be needed exactly to build my environment. I organized my references in a separate asset reference board and made a clear distinction between must-have and nice-to-have assets.

In my search for assets I quickly found the really good ‘Tropical Jungle Pack’ by Kaizen Digital Interactive in the Unreal Engine Marketplace with optimized Nanite jungle trees in the pack ready to be used. However finding good mangrove trees appeared to be a bigger challenge, as most asset packs were either limited to a single tree or not prepared for use in game engines. Eventually I went for the beautiful ‘Mangroves 2.0 3D Model’ by Robert de Wit on TurboSquid, but every tree needed manual preparation for real-time from a humongous Maya scene and a complete material rebuild in Unreal as well. For the rest of the assets I mostly relied on the Quixel Megascans Library.

BUILDING THE LANDSCAPE

The extensive classes about the landscape and water tools in Unreal 5.1 at the end of week one, provided me with the knowledge to start shaping my landscape. At the start, the most difficult thing to imagine was how big my world should become. Looking up the dimensions of real lagoons in the Raja Ampat region and other bays I visited in the past, helped me a lot in getting the right feel for scale. This lead me to decide to create a 2x2 km landscape in a world partition level.

What was also really helpful was a technique, suggested in one of the classes, to draw a landscape schematic in Photoshop first, and use that as the base color texture for a temporary ‘sculpting’ landscape material in Unreal. For this I made sure to:

match the resolution of the schematic in pixels to the resolution of the landscape,
use height contour lines and colors to visualize the shape of the landscape,
overlay a grid with known dimensions to keep a sense of scale.

Since my main goal was to learn more about the world building tools in Unreal, it was a logical decision to use Landmass. I started with a first pass for the mountain shapes, and then used the experimental Water Body Ocean actor to add the water surface. On this foundation I continued to build up the underwater landscape in layers from the deepest part of the ocean working upwards to the sand banks and islands rising above the water. I finished the landscape build with a second pass for the mountain shapes and some hand painted touch-ups for the coastal areas.

The great thing of using Landmass for shaping a landscape is the enormous flexibility you keep all the way to the end of a project to move an island, lower a mountain or add other little tweaks that would be much more cumbersome with using an external package for landscape creation.

Landscape Setup with Landmass and Water Tools in Unreal 5.1

The landscape materials class at the end of week two introduced us to advanced concepts like auto texturing based on height, slope, curvature, normal and ao, cell bombing and distance fading to hide texture repetition, triplanar projection to hide texture stretching on sharp angles, macro variation with noise textures, virtual texture blending for seamlessly blending assets with the landscape, turning your landscape into a Nanite asset and procedural foliage placement via the landscape material.

This class also revealed a major disadvantage of using Landmass for creating landscapes, because while you would get both height, curvature, normal and ao maps automatically when using an external package for landscape creation, you don’t get those directly using Landmass. In that case you would have to export the landscape as a mesh, reimport it again and bake the maps using Unreal’s modeling tools (which currently is still quite a slow process). Sadly this totally negates the advantage of flexibility in using Landmass, since you have to redo the round trip every time you make a change to the landscape.

Since my time was becoming limited by the end of week two, I decided not to setup this advanced landscape material from scratch myself but to use the great ‘Auto Landscape Material’ by Unreal Sensei instead, and see how far I could get with only using the landscape itself as an input (so without baking the additional maps).

Using 4 textures from the Quixel Megascans Library (rock, forest ground, coral beach sand and rippled sand) this turned out to work pretty good already, even without baking the additional maps. The Auto Landscape Material provided a lot of flexibility in customizing the controls and combining automatic features with manual landscape painting.

SCATTERING THE FOLIAGE

The foliage and grass class at the end of week two introduced us to two incredibly powerful methods of procedural foliage placement, both via the landscape material and through the use of foliage volumes.

I decided to use the procedural foliage placement feature in the Auto Landscape Material to scatter some seagrass and corals underwater across the seabed, to have some more texture variation on top of the rippled sand texture. With the material ready to use, this was as simple as loading the Landscape Grass Type actor connected to the seabed material layer with the desired static meshes and adjusting density, scaling variation, fall off, etc.

Underwater foliage scattered via Auto Landscape Material

Because of the vastness of my landscape it was a no-brainer to use procedural foliage volumes for scattering both the jungle and mangrove trees. Together with the foliage blocking volumes and the fact that you can shape volumes in Unreal however you like, this provided an enormous flexibility for scattering foliage. Also being able to create different sets of trees in procedural foliage spawners (to be used on different foliage volumes), and setting limits for tree placement to certain heights (for example above water) and slopes as well as setting density, clustering and numerous other parameters on a per tree level, appeared to be incredibly powerful.

Thousands of Nanite trees scattered via Foliage (Blocking) Volumes

Besides procedurally scattering trees across the landscape, I hand placed only about 150 mangrove trees, water lilies and underwater rock formations in specific areas to control the look on camera. In the end the level contained about 9.300 mangrove trees (ranging from 1 to 22 million triangles per tree) and 58.000 jungle trees (ranging from 1K to 37K triangles per tree), while I was still being able to work smoothly inside the scene (with scalability set to high). It’s amazing how powerful Nanite is!

Nanite Visualization for the scattered Nanite foliage

LIGHTING AND CINEMATICS

If I had to mention one take away from all scrums and weeklies sessions during this fellowship, it’s: Lighting is King !!! Great lighting is essential to sell any environment!

For this reason we were treated with a lot of interesting sessions on lighting and rendering in week two, and our team was blessed with a TA specialized in lighting as well.

The base of my ‘dawn’ lighting setup consists of the default environment lighting components: A low angle Directional Light for the sun, SkyLight to control the bounce light, SkyAtmosphere to create add atmospherics (with a higher Mie absorption), a subtle ExponentialHeightFog to create more depth and VolumetricClouds to create the base cloud layer. Additionally the PostProcessVolume eliminates unwanted effects like too much exposure compensation, sets the color temperature and most importantly controls all the Lumen parameters that have the capacity to turn a render from plain right ugly to pretty darn good with a few changes.

On top of that some fill lights were added to lift the shadows on the mangrove trees and islands, as well as to brighten up the water lilies. Material Parameter Collections were created for the jungle tree materials and mangrove tree materials to be able to control their appearance easily on a shot by shot basis without having to put extra light on them. Lastly I added more depth by placing fog cards using the ‘EasyFog’ Blueprint by William Faucher, and brought more life to the environment by adding some birds.

A rendering issue that I still have no good solution for are the blocky shadows and rendering artefacts caused by the Virtual Shadow Maps when the shadows are really long, in this case because the sun is just above the horizon. If you look good you can notice it in some renders, but I tried to hide it in the edit as much as possible.

Throughout the entire world building process I worked with the initial camera angles from my concept sketch. However after I had setup the cinematics in Sequencer and realized the ‘dawn’ lighting, my mentor advised me to have another ‘photo’ walk around my environment to try to find new and interesting camera angles where I could profit from the local lighting conditions. This resulted in a bunch of great new shots, that improved the way I was selling my environment (and to be honest I kept finding new angles even until after the festival and final revisions deadline).

Another big revelation for me from other fellows was the benefit of using an OCIO color workflow in Unreal (sRGB or ACES) to safeguard the color output to Davinci Resolve (or any other editor) and all the way towards the color graded video output.

KITBASHING A THUNDERSTORM WITH LIGHTNING

To be honest, when I wrote my concept and put together my mood board, I hadn't really thought through how you could realize a thunderstorm with lightning in Unreal Engine. So after experimenting with different volumetric cloud solutions, I came to the conclusion that none of them offered the flexibility I needed or would be feasible within the couple of days I had left.

The golden tip came from a fellow two days before the accountability deadline, VDB clouds! Apparently the Canadian video game developer Eidos-Montréal released a non-official Unreal plugin that could read OpenVDB and NanoVDB files in Unreal. You can find it here. And on top of that there was a wealth of free downloadable VDB cloud assets available online. I ended up using the free VDB Cloud Pack and a Looping Tornado from JangaFX (created with EmberGen). With about 160 of these VDB volumes I kitbashed the thunderstorm I had imagined together in no time, including a moving tornado! A happy accident revealed itself when I browsed through my cameras again with my wife to review the thunderstorm: I had accidently created an angry face in the clouds, which matched my concept so well!

After finishing the thunderstorm shape, I continued adding the lightning effects, but ran into some limitations of the current version of the VDB plugin: Only ambient light and the first directional light were supported at this point. This meant that it would be impossible to have a dynamic lightning effect interacting with the volumetric clouds!

Another happy accident revealed the solution when I started playing with the shading parameters on the VDB cloud actor: moving the anisotropy-slider produced a similar effect on the VDB cloud as a real cloud would show when lighting up during a lightning flash. So again, I kitbashed together 6 additional cloud groups on the VDB thunderstorm to simulate different cloud sections lighting up during a series of lightning flashes.

Next, I downloaded one of the ‘3D Lightning’ models by Constantin Os from TurboSquid and cut it up with the Unreal modeling tools to create various smaller lightning bolts. I positioned them within and between the 6 additional cloud groups I created, and set up the animation for the whole lightning effect in Sequencer by grouping similar parameters on multiple clouds into a single parameter in Sequencer (which made animating so many assets together a lot easier).

Finally, I added some ambient sound with a couple of sound effects which brought the whole scene to life even more!

CLOSING WORDS

My Unreal Fellowship experience was amazing! The amount of knowledge that was shared in the classes, during live labs, by guest speakers and through the Fellowship community on Slack was incredible! It was really inspiring to work alongside so many talented artists and witness their worlds growing from concepts to incredible pieces of art. The long days were exhausting but I would do it again anytime! I learned a lot about Unreal, but also realize there’s so much more that I have yet to learn about this powerful piece of software.

I’d like to close this post with expressing my gratitude. I’d like to thank Brian Pohl, Julie Lottering, Cathy Cheo-Isaacs and Epic Games for the opportunity to be a part of the Fellowship community. I’d like to thank all instructors, TA’s and guest speakers for sharing their knowledge, and especially my mentor Cordula Hansen and TA Daniel Langhjelm who were always there to provide help and assistance when I needed it. I’d also like to thank my team mates and all other fellows for being so kind and for sharing this experience together. And finally my family, thanks for putting up with this crazy schedule for 4 weeks.