Sign-up for Email Updates:

Help Survive Development:

Development Blog
July 25, 2015 By: David Tse

This week I worked on improving high level interstate generation and then the reorganization of the generation pipeline. I made an important shift on the world generation development plan. Previously my focus was on trying to get each step to be as realistic as possible before moving on to other steps. This made me really focus and agonize over generation algorithms before I can actually see how they look/feel in game. The new, correct, approach is to get this pipeline fully functional from start to finish. From generating content all the way to loading and walking through it in-game.

This means that many of the gen algorithms will be simplistic and very far from generating realistic worlds. I sort of knew this was the right approach from the beginning, but found it hard to move on to other steps when the gen algorithms were in the lowest level of functionality like I should have. With this reorganization and the distinct definition of the different stages of the pipeline, their inputs, and their outputs I've decided to fully commit to getting everything flowing with the simplest algorithms.

Once the pipeline is functioning I can release the build to you guys with simple algorithms then improve and iterate on them much more quickly with real time feedback. So you guys will get a bare bones update with simple gen algorithms and I can do updates to the generation fairly frequently as aspects of it are improved. Instead of you getting a semi-decent generated world later down the road.

For this weekly update I’m just going to lay out the generation pipeline by showing the inputs/outputs for the stages, the purpose of each stage, and the current state of each stage. Writing this weekly has also really helped me to fully flesh out the pipeline instead of it all just existing in my head and random notes. So first let me quickly define some terms that will make the pipeline a little easier to read:

GP: Generation Parameters – Inputs to the stage usually defined in the .ini files. Stuff you all will be able to tweak for mods or custom worlds in the future.

HL: High Level – I use this a lot talking about some of the different stages. It just means the scale of the data is very large and in the kilometer range (mountain ranges) while the low level stuff is in the meter range (small hills).

World Generation Pipeline



High Level Continent Generation Stage:

Description:

Uses plate tectonics simulation to generate continents and large terrain features like mountain ranges, lakes, and high level elevation. This stage also distributes moisture and temperature to generate different biomes.

Current State:

High level elevation generation is in a decent state that generates good looking continents, but loads more work still needs to be done. Moisture/biome classification is very basic at the moment.

Inputs:

  • GP(number of continents, ocean to land ratio, elevation collision spread ratios, etc.)

Outputs:

  • HL Continent Heightmap split into chunks
  • HL Biome Map with moisture and temperature split into chunks

Initial Test
Screenshot of basic continent generation with mountain ranges, lakes, and high level elevation.

City Placement Stage:

Description:

Uses biome data to distribute cities across the land. It favors locations near water and that are relatively flat, but will create sparse cities in less than optimal locations like in real life. This stage also decides what cities need to be connected to each other.

Current State:

Right now it just does semi random city locations that favor high moisture and does a good job of determining which cities need to be connected.

Inputs:

  • HL Continent Heightmap
  • HL Biome Map
  • GP(biome distribution factors, desired large to small city distribution, population based prune distance, etc.)

Outputs:

  • City Data(Positions, List of Neighboring Cities, Population Count)
  • HL Population Map, created using noise based on city locations/population

Initial Test
Shows example of early city distribution. Ignore terrain its on that is from very old build.

High Level Interstate Generation Stage:

Description:

Goes through city data and creates interstates that make all the necessary connections between cities. It uses the high level continent elevation and the algorithm for low level elevation so it can make accurate decisions about where it can and cannot pass through (hills, rivers, etc.). It also makes sure interstates naturally pass through cities instead of all going through the same city center point.

Current State:

This is one of the stages that needs the most work. It currently tries to do some pathfinding between cities and generally it works, but roads frequently overlap due to the lack of pathfinding resolution. This system also ignores low level elevation. I did a little work on it this week before deciding to focus on the pipeline as a whole, but this system still needs to be heavily revised once I get the whole pipeline functioning. The plan is to use a growth system similar to the city roads instead of high level pathfinding.

Inputs:

  • HL Continent Heightmap
  • HL Population Map
  • City Data
  • Low Level Terrain Gen Algorithm/Sampler, needed to generate local obstacles like small (few hundred meters) hills that need to be avoided.
  • GP(terrain change tolerances, max allowed slope/turn, interstates sharing tolerance, etc.)

Outputs:

  • Interstate Splines.

Initial Test
Screenshot of the current state of high level interstate generation.

Major City Road Generation Stage:

Description:

Uses population map to grow major city roads based on desired city shape and type. Then take those roads and classify them into city blocks.

Current State:

I'm pretty happy with this stage so far. Right now it only has one city shape which just follows population with not high level target shape, but in the future there will be different city types/shapes. The city block classification is fully functional. The city generation video blow is basically the current state with a much larger scale.

Inputs:

  • HL Interstates
  • HL Population Map
  • City Data
  • GP(all the growth parameters for roads, city shape, etc.)

Outputs:

  • Major City Roads
  • City Blocks created by major roads


City Block Generation Stage:

Description:

Takes the city blocks enclosed by the major city roads and fills them with the various buildings and roads needed in a city. There are many different city block types that are filled with different types of buildings. Think retail, residential, schools, business, etc. There can also be a mix of different types like a neighborhood with a school in it or a business park with a large retail store.

Current State:

This is the stage I have probably put the least amount of work into so far. Right now it generates a very simple grid like neighborhood. It also only generates roads at the moment, but adding simple house generation will not take too long. This is one of the stages I am leaving in its current very unfinished state until the pipeline is fully functional. The current state is shown at the end of the city generation video above.

Inputs:

  • HL Interstates
  • HL Population Map
  • Major City Roads
  • City Blocks
  • GP(some controlling params on frequency of different block types and their tendency to be near population, type of city based on population(rural, medium pop, high pop with skyscrapers, etc.), desired population density, etc.)

Outputs:

  • Low Level city roads
  • List of all builds with their position, orientation, size, income/wealth, and type
  • List of city entities(light poles, telephone poles, traffic lights, mail boxes, etc.)


Real Time/On Load:



Building Generation Stage:

Description:

This stage and all subsequent stages are done when you load the game and in the background as the world is streamed. This stage takes the boundaries for the buildings from the block generation stage and creates procedural building layouts. It then fills those houses with all the base entities they need (furniture, doors, windows, trimming, ect.).

Current State:

This is currently version 1.0 of basic house generation. It is in the same state as the build generation video below that talks about it. This algorithm has many problems and gets some fundamental things about how buildings are constructed wrong. The biggest is that they are largely rectangular and don’t waste space. Anyway there will be a new version of house generation once the pipeline is set up. For now those houses will be put in a box and thrown into the world as is.

Inputs:

  • Building Data
  • GP(List of params for each building type being generated, will be a large data file for min/max values for room types and sizes depending on house income/wealth)

Outputs:

  • Custom building layouts for every building needed, determines walls, doors and windows.
  • List of entity data needed per house, location of sinks, couches, beds, door frames, etc.


Geometry Creation Stage:

Description:

This stage just takes all the building layouts and road data then turns them into actual geometry and textures the game can render.

Current State:

It is sort of good enough for now. It can create roads from splines and generate most of the walls for buildings. There are some holes like above the door ways, but like everything else its good enough to get the pipeline set up.

Inputs:

  • Building Data
  • All Road Segments

Outputs:

  • Generated geometry needed to render all the roads and building walls.

Initial Test
Shows early interior of a house with some test balls.

Final In-Game Stream and Render Stage:

Description:

Takes the current position of the player and streams in/generates the needed data in the background. It aggressively culls said data and then renders it. This is basically the real time component that takes all the generated data and makes use of it. The above two stages actually happen inside this stage for the most part. As the generated stuff is streamed in the player save games files are loaded and merged with the generated data in this stage.

Current State:

I have spent a lot of time on this stage so far. All the terrain streaming is done and the backend data structure for streaming chunks of world gen data is all planned and saved out. The only work that needs to be done is on the actual loading part. Once this stage is done and we can see generated stuff in-game then the gen pipeline will be fully functional. Then we can go back and make the world more dense, realistic and interesting.

Inputs:

  • Chunked Building Data
  • Chunked Road Segment Data
  • Chunked Entity Data

Outputs:

  • The World

Initial Test
Only screenshot I have of a building with terrain in-game. Its pretty old and just shows building without its exterior sitting in the terrain.



So that is the general plan for world generation and the steps involved. Like I said before my focus is now on getting this pipeline functional with the absolute bare minimum generation algorithms. This should mean the next Pre-Alpha update is only a few weeks out, but we will see.

That's all I have for this weekly. Hopefully it gave you a better picture of the current state of world generation and where it's headed. If you have any questions about the process or anything else please let me know and don't forget if you want to see my daily development updates you can follow me on twitter!

July 17, 2015 By: David Tse

You may have noticed that I missed the last two weekly updates. There are a couple of different reasons for that. Primarily my focus on getting the mac build of Atlas to the new artist and the fact that I was out of town for a week. All that really matters is we are back on track and updates will return to the normal weekly schedule. This update is a day late from the target release day of Thursday, but that’s because my 21st birthday was on Wednesday and my capacity to work on the following Thursday was greatly diminished...

Anyway on to the actual update. These last 3 weeks were focused primarily on getting a working build of Atlas fully useable on Mac so the new artist could start previewing his work in-engine. On top of that I did get some work done on city generation, but nothing of note. Also keep in mind that one of these weeks I was out of town and only got minimal work done.



The Fall of QT

One thing I wasted a lot of time on was trying to get QT to function correctly with OpenGL contexts on Mac. After trying so long to get that to work in addition to dealing with QT lag spikes and uncontrollable crashes I decided to just completely ditch it.

Initial Test
Screenshot of Atlas running on Mac with new custom menu bar.

In the amount of time I wasted getting QT to work with my engine on Windows, learn how to use all the different QT widgets, and then trying to get it to work again on Mac I could have created a robust QUI system myself. So I decided to just cut my losses and invest future Atlas development time into my own system.

This will take a little more time to get up to the same functionality as QT, but I can build it up one part at a time only creating the GUI widgets I actually need. Having full control of the GUI system and not having to worry about figuring out how to create/get working exactly what I want in QT will allow for much faster development in the long run.

Initial Test
Render of new Front door created by our awesome new artist Erik!

I actually already created a pretty strong GUI foundation in the engine for all the menus in the game. So getting the editors functionality up will not take too long. To start I created a custom menu bar using these foundations that is used to load models, load materials, and toggle different things in the engine. Like I said before I can do the editor GUI one step at a time and currently that menu bar is all I need to implement right now. When Atlas needs more functionality I can easily implement the widgets it needs.

Initial Test
Render of new Front door created by our awesome new artist Erik!

Another big advantage of doing the GUI myself is that it helps to advance the engine’s GUI system which is going to be used for all the in-game GUI for things like inventories and other menus. This means work on Atlas is also directly improving the game itself. Something that definitely cannot be said about using QT which usually results in me wasting my time trying to get it to not suck.

Mac Port

Once I ditched all QT dependencies in Atlas finishing up the port to Mac was more straight forward, but there were a few more challenges to tackle. In addition to getting rid of QT I also had to stop using Xcode because every time I would run Atlas it would just crash Xcode which is completely unacceptable. So I was off to look for a reliable command line build system. Something I previously had little experience with.

After a little while looking and researching I ended up going for Meson with the Ninja backend. Meson is an open source build system created by Jussi Pakkanen that is super fast and really easy to set up. I really love it and if you are looking for a C++ build system there really is nothing better.

Initial Test
Render of new backyard fence created by our awesome new artist Erik!

Once I had the app building and running fine on OSX I had to package it correctly and get it running on the artist’s non development machine. I’ve never deployed a Mac app before so there was a little bit of a learning curve to figure out how OSX deals with dependencies, but I eventually got it packaged in a .app and running perfectly on the new artist’s computer. Meson actually comes with a nice utility to do the packaging which is another big plus.



On top of getting Atlas to the artist I also did some significant restructuring to the perforce server. I moved all the game data from the main depot with the main working directory and the code into another depot so artists can focus completely on the assets. This is also where new builds of the game will be staged for release. This all just means development and deploying builds to you all will get much easier meaning releases can happen more often.

Then like I mentioned I also did some more work on city generation, but there is nothing of note to mention. All my focus is now back on City generation. That is all for this catch-up update and you can start to expect consistent weekly updates starting next Thursday. Don’t forget I’m doing daily updates on Twitter and continued to do them even when missing the last two weeklies so if for some reason I miss a week in the future there will still be news over on the twitter: @Subsurfacegames.

June 25, 2015 By: David Tse

This week I started working on proper scale for roads and a bunch of other less cool stuff. I spent time improving the editor, Atlas, and porting it to Mac so the new artist could preview his work in-engine. This week I'm also going to try something a little new. I'm not sure exactly how its going to work out or what you guys think about it, but I could really use your support.



Patreon

I'm officially launching a Patreon Campaign to improve the game and help speed up development! Patreon is a service where you can contribute X amount of dollars per month to support someone or something, in this came me and the development of Survive! You can check it out here: https://www.patreon.com/survive

I think I do a good job of explaining the reasons for the campaign on the actual patreon page so you should go read that, but here is the jist of it: I have enough money to support development for a long time, but if you contribute to the patreon I can spend more money on new assets for the game and get to the steam launch much faster.

There will be no exclusive rewards or content for this patreon so you don't have to worry about missing out. It's just a way for you guys to further support development and help me out. If you already have the game and can spare a few dollars to make the game better for all of us please head over to the patreon and take a look. Now onto this week's development news!

Atlas

Like I mentioned before I spent most of my time this week with atlas, the entity editor for Survive. There is not a ton of news here, I just got the model and material loading/preview working enough for the new artists to check out his work.

Initial Test
Screenshot of Atlas previewing a model and material!

These improvements were actually a small part of my work this week, but this artist for some unfathomable reason uses a mac. So I spent a good amount of time porting Titan Engine and Atlas to Mac. Theoretically this should be easy because all libraries are multi-platform, but its still a HUGE pain in the ass.

Less fun stuff

I figured since I was recompiling 3rd party libraries for mac I might as well dive straight into it and do a bunch of maintenance stuff I've been meaning to do for a long time. The first thing on the list was upgrading to visual studio 2013. It's not a huge change, but comes with faster compile times and is really easy to do while doing the next change.

One big bottleneck when working on world generation algorithms is memory. When I'm testing new stuff and trying to get all these cities, roads, and buildings to generate correctly memory is a real pain in the ass. I just don't want to have to worry about optimizing memory while working on rapidly changing algorithms. So I decided to switch the game exclusively to using 64-bit binaries.

This change does mean you need a 64-bit operating system and OS to run the game now, but the fraction of people who don't meet those requirements and can run the game already are minuscule. With a standard 32-bit binary the game is limited to only 4GB of memory regardless of how much a computer has. Like I said before this is really annoying when developing the game and it limits players who have more RAM than that. Hugh from Unknown Worlds wrote a really good article on their switch to 64-bit that will give you some more insight: http://unknownworlds.com/subnautica/64-bit-not-64-bit/

Initial Test
Screenshot of city roads with better scale, but still way to big. The roads are slightly shifted to the right because of a bug.

On top of all that I decided to just go even further into the rabbit hole and reformat my computer and reinstall windows. I usually do this every couple of years to keep my computer fresh and it had already been a few years since the last time. I also decided to go with the Windows 10 preview build instead of sticking with Windows 8.1 because i'm impatient and stupid.

While reinstalling windows it was also the perfect opportunity to setup my two 250GB SSDs in RAID 0! RAID 0 combines two drives so to windows they look like one single drive. This is really great for read/write performance and it gives me double the space on my C drive something I was constantly running out of. This read/write performance is also helps to improve compile times. I didn't do this before because it's also more risky. With this setup if one drive fails you lose everything. To "fix" this issue I just setup my external 4TB drive to do daily backups.



So this week I spent a lot of time doing a bunch of stupid house keeping shit. I also didn't finish the mac build just yet so expect some more work on that next week. Other than that the maintenance stuff is mostly finished and I can start working on proper scale city generation with all 32GB of my RAM soon! Don't forget to go check out the Patreon and consider further supporting Survive, I would really really appreciate it!

June 19, 2015 By: David Tse

This week I worked on classifying all the roads into city blocks, figuring out which blocks have the highest population and using the border roads to fill each block with basic grid like neighborhood layouts.

This weekly update is also going to be a little different and is all going to be in a video showing the city generation so check it out!

City Generation!

So like I said in the video this next week I'm going to integrate the cities with the high level interstates and then move back into the actual game world loading and rendering these new roads. Once everything is in the world I'm going to see all the problems with the road generation and most likely have to go back and tweak it to make them work, but we will cross that bridge when we come to it.

As for the progress of the game as a whole its going really well. These roads were a huge milestone and the long awaited Pre-Alpha 5 is getting much much closer to release. That's it for this week and don't forget if you want daily news on what's going on with development you can go follow me on Twitter!

June 13, 2015 By: David Tse

This week was about refining and tweaking the local constraints and global goals so they create the road networks I want or at the very least are acceptable for now. I also started a little on actual city roads that connect large city blocks. The roads need more work and variation, but are a decent start so far. I also made some changes to how all this generation stuff is going to work.



New design

Last week I mentioned that the city connecting roads try to follow population maps that were generated using the old high level highways and city locations. This week I decided that was stupid. So I'm going back to using those actual roads, but then delete the parts that are close to the actual cities. Then the road growth takes over generating the city roads using new population maps created with perlin noise masked by city locations. This way we get all cities connected and each city can handle connecting these interstates.

Road growth refinements

A lot of this has to do with the probes that keep roads from being too close to each other. There were a number of bugs with the circle line segment intersection function that had to be fixed. These fixes alone greatly improved the road placement. Once those core bugs were fixed I redid the way the probes decide which way to turn when encountering a road segment. By getting the normal of the hit segment and projecting it out from the new segment we are rotating all segments in the direction of least resistance. Another important improvement in road placement.

Initial Test
Current best city highways, smaller city roads will be filled into the blocks created by these roads. The green is the population map so spots with lot of green are cties. I made the cities really really huge so I could generate them faster for testing. This screen is showing 4-5 cities really zoomed in. So imagine the old high level roads I showed before and at each intersection point there will be one of these cities.

I added a second probe to the tip of new segments along with the one already in front of the tip so roads that are growing at the same time in parallel will also try to avoid each other. This was generally the only case were roads could end up being really close together so this fix means that roads should NEVER be too close to each other.

This week I also extended the probe system to the environmental awareness. Now in the local constraints each prospective road segment will sample a number of uniformly distributed points inside a specific radius. If some of these samples are in water or are on a mountain it will rotate to avoid it using the same system as road avoidance probes. This is more robust than the old system that just did one sample in front of the segment meaning roads more closely follow the coasts and mountains.

Initial Test
Example showing how the roads would be generated WITHOUT these probes. You can see how the roads look terrible because they are all jumbled together.

I mentioned last week that roads would try to reach water so they could follow the coast instead of hitting it and bouncing in another direction. This week I actually implemented that. So if the environmental probe hits water or some other impassible obstacle it gets marked and it will only try to seek the water or other obstacle it hit. Otherwise roads just try and follow population and maintain a certain shape. So now when a road encounters water it will aggressively seek it out while also avoiding it, creating perfect coastal highways. This system needs a little more tweaking of min max values and angles, but it's working ok for now.

Initial Test
Showing off ocean locking roads with all other constraints turned off.

City generation

I did some of the base groundwork for city gen which is actually the stuff I mentioned earlier. For city generation to work right the roads need these local constraints so roads don't get to close to each other or water. I also started on the algorithm that finds these large city blocks and fills them with neighborhoods, but it's not finished so I'll show that next week.



I didn't have time to make a video of the world generation this week, but there should be a cool one showing the real time growth next week. I think I will only need one more week of working on the road generation before it will be good enough to start loading and placing houses on them in game. If you want to see the advancement of road growth and city generation as it happens you can follow me on Twitter for daily updates!

Previous Page