Greetings! I'm Caleb, developer of Harwell. As part of update 0.18 Harwell got a huge update - actual, functional multiplayer, and it involved an ever so slight complete rework of the multiplayer system in the game. Here's what it entailed!

The main change in the main menu is this new panel on the left side which displays your Steam and Nakama IDs and lets you update your display name:

You'll also get a little setup screen when starting the game which asks if you want to link Steam to your Nakama account, if you click no it will just use Nakama and store a device ID which is anonymous but can be used by Nakama to know if you're playing from the same device. This does mean that if you don't sync it to Steam and then move to another computer your multiplayer progress may be lost! By identifying using Nakama IDs it means people can play multiplayer outside of Steam or if Steam is down.

If you link the two, it will just set your Nakama ID to your Steam ID to make things easier. When you move to a new computer, it can then just pull the account using the Steam ID.

Maybe in the future Harwell will have its own account system so your data doesn't rely on Steam and can still follow you between devices, but currently it doesn't feel like it's worth the hosting costs.

I intend to expand this panel in the main menu when features are added later - company traits, HQ skins, stats, and more. For now, it's just used to tell you your connection status to Steam and Nakama.

As part of the update the lobby screen got an overhaul, I tidied it up, redesigned it to be clearer, and made it look a bit prettier. There's a good chance this might go through some more redesigns, but for now it's functional and looks pretty good.

The update also updated the static background stars to a dynamic shader that creates stars that twinkle, just for an extra bit of pizazz ✨

Nakama is an open-source multiplayer system used by many popular games which I use in Harwell to manage matchmaking - putting players together into multiplayer games. There's no game logic server-side, Harwell still uses peer-to-peer hosting so you can host your own games, Nakama just manages the lobbies and makes sure you can find each other (and see all open lobbies!).

Previously, my basic (broken) implementation of multiplayer was done using GodotSteam which allowed me to utilise Steam's matchmaking, but I realised that if Harwell was to played outside of Steam this wouldn't work, so I needed something more flexible. In comes Nakama!

So now, when you go to host a game of Harwell the game:

1. Checks if you have Steam open

2. If you do, hosts a lobby on Steam's matchmaking

3. It then hosts a Nakama lobby and matches it to the Steam lobby (if it exists) in it's logic and treats them as the same lobby.

   If you don't have Steam running, it will just use Nakama by itself and work exactly the same!

The lobby list that shows when you click 'Join Game' shows all of the Nakama lobbies, when joining a lobby Harwell automagically then pretends they're both the same thing so people joining from Steam and from Nakama both end up in the same place - your lobby.

Anyone can join via Nakama using the game ID or using the lobby list, but players can also join using Steam friends and can see/join the lobby in the Steam friends list. No matter which way you join, Harwell will treat you the same, both services are just there to get you into the game, actual logic and functions are synced up using RPC calls for the most part!

Remote Procedure Calls! Godot makes it really easy to sync functions between players, I can add @RPC before a function in GDScript, define some parameters, and Godot will make sure the function runs on certain peers, all peers, or specific peers depending on the parameters you provide.

This update involved a pretty heavy restructuring of Harwell's existing multiplayer system, this bit is going to be very technical, but I thought I'd add it for anyone interested. Harwell's multiplayer was not functional previously, but everything was in place for it. Technically, you could host and join games before, Harwell just wouldn't do anything, so you'd join a lobby and the game would recognise it (and say you've connected in the game logs) but it would just... keep you at the main menu. I had to make sure everything including the UI state is kept in sync.

Previously, the lobby system was it's own script called host_manager, and all of the in-game multiplayer syncing (that sync the market, buildings, map, stocks, player information, etc.) was all in one big multiplayer_manager script.

That was awful for organisation and readability, the script was over 3000 lines long! Therefore, I split it into what I'm calling 'agents'. So now the multiplayer_manager oversees the in-game syncing system and any data that needs to move between players has been put into an agent:

• steam_sync_agentManages all Steam syncing - making sure Steam IDs are synced, that the game properly reacts when joining through Steam, and manages everything to do with Steam lobbies.

• nakama_sync_agent
  Same as the Steam agent, but this one focuses on handling all Nakama syncing.

• init_sync_agent
  This ones a bit different, this is the initialisation agent and this ensures that when loading into the lobby and into the game, that everything is properly initialised. This agent has a 'checklist' of sorts that it runs through each time a game is started and checks things off as it makes sure the player is initialising all UI items, game items, and more. It interacts with the other agents as well as the multiplayer_manager.

• player_sync_agent
  This one handles everything to do with players. This ensures all clients receive all Player classes that have been created so that you can see everyone that's in your game. It also manages syncing player inventories, HQ building, financial information, and stock information along with a host of other things.

• world_sync_agentThis one is in the name as well - this syncs the world! This one makes sure that not only is the map data synced efficiently, but also the central colony as well.

• building_sync_agentSurprisingly (not), this one controls all data syncing for buildings. Every time you place a building, this agent makes sure that the other players can see it.

So, the multiplayer_manager and it's agents make sure everything syncs properly between peers, but how do we ensure that game data stays in sync? How do ensure that players aren't de-syncing without them knowing, or de-syncing and causing crashes, or other random problems? The most annoying thing in any game is when you de-sync mid-game or run into obvious connection issues constantly.

This is where the sync_manager comes in - this runs throughout the game on both the host and peers and constantly checks that each player is staying in sync. It does this in the best way I could think of:

1. Generate a checksum for each category of data based on the data the game has stored (the market, stocks, buildings, the map, etc.) The host then generates its own set of checksums.
   
   Why checksums? Network performance! For a little bit more processing power on the player's computer, we can make the data sent between players much smaller. Syncing a checksum is much easier than syncing all of the data the game needs!

2. The peers then send their checksums to the host player, with the frequency depending on how critical the data is. Something like the game state (Intro, Win/Loss Screen, Gameplay, etc.) is critical and gets checked every second, something like the other players in the game get checked every 30 seconds (unless something happens like a player leaves/joins).
   If the checksums match, great! The game continues on.

3. If the checksums don't match, then the sync_manager speaks to the multiplayer_manager and its agents to trigger a full sync for the de-synced category (so it's only syncing the market for example). Usually the game makes incremental syncs, but if the sync_manager detects a de-sync it re-syncs the entire category of data.

4. The checksums are then checked again to make sure the player recovered from the de-sync.

If the Sync Manager fails three critical consecutive checks in a row then it will pause the game, notify all players that someone has de-synced, and trigger a full re-sync across all categories to make sure the player is up to date. This is the last line of defense against a possible de-sync, and something I hope shouldn't trigger very often at all!

It's been a long one, but that pretty much covers the multiplayer changes made and the latest version of the game's multiplayer systems. If anyone is curious about any particular parts I'm happy to explain it!

Anyways, stay tuned for the next one.

See ya!

Caleb