Gossip has been rolling around in my head for a while while working on ExVenture, but there has always been more pressing matters. About a month ago we were talking on the MUD Coders Guild about cross game chat and that inspired me to kick this off.
Gossip is similar to the I3 Network except it uses more standardized technologies. Secure WebSockets are the transport layer and all events are in JSON. You can see the documentation on Gossip.
ExVenture Remote Channels
The first big feature of Gossip is remote channels. You can flag a channel as a Gossip channel and it will try to send all communications up to the network.
ExVenture Remote Player Status
When your game is configured for Gossip, all signed in players are pushed up to Gossip. This lets other connected games see your players sign in and out. Right now all notifications are displayed to users on ExVenture. This is an optional but highly suggested feature for games that are not based on ExVenture.
This should help make your game feel more alive by letting your small pool (maybe just 1!) of players see others on the network.
The local who list will also display remote players, so users can see who is on the network.
ExVenture Remote Tells
The final big feature for Gossip right now is remote tells. If you're syncing your local players up to Gossip (and ExVenture does) then remote games and send tells to your local players. ExVenture lets you initiate tells and also handles reply for remote players.
This degrades nicely for remote games that do not support tells. As part of connecting a Gossip client says what features they support. For games that are not built on ExVenture remote tells are optional, but highly suggested.
Gossip Games List
I definitely suggest you check out the Gossip Games list. So far we have 5 games connected, more seem to be checking out Gossip every day!
The Gossip client from ExVenture has been pulled out into it's own Elixir hex package. If you're developing your own Elixir MUD (and there are a lot of you out there) then come join the fun and add the package.
Right now you do need to implement the full set of callbacks for all of the features of Gossip, but I would like to let you dictate which features your game supports and slowly add them in.
There is also a Ranvier bundle that supports remote channels and player status updates.
Next month I hope to start back on ExVenture more and leave Gossip to sit for a bit. I think Gossip has a good enough feature set to let other games implement what's there and get some more feedback from the community.
I got to try out a new OTP Supervision tree pattern in ExVenture, that I am going to call a tether.
I started a new service, named Gossip, that ExVenture creates a persistent websocket connection to.
When this service dies or is not alive when the ExVenture server starts, the websocket connection will crash or refuse to start entirely. It crashes immediately and then cannot reconnect. This ripples up the supervision tree taking the entire application down in short order.
To get around this, I added a layer of supervision trees for just that socket process. My tree looks like this when fully booted:
The tether supervision starts with no childspecs. After boot the Monitor (or the cluster leader) will start the Socket inside the tether. When the socket process dies, it will eventually crash the tether supervisor causing it to restart.
When the tether supervisor restarts, it will restart with no children breaking the restart loop ("cutting the tether".)
Eventually the monitor process will try to restart the Gossip socket which may crash, causing the tether to crash and so on. Either way the crashing process was contained and the application stayed up.
The code for this was pretty simple, you can see it on GitHub in the gossip folder.
The overworld is a new extremely large feature for ExVenture. Admins can "paint" an overhead map that players can wander around. Exits can be made to rooms in the "old world" for travel back and forth.
I was very happy to get this working, even if there is still a large TODO list.
There was some conversation this week on the MUD Coders Slack that made me finally get around to a cross game chat network. I started the Gossip project for this. It's a simple Phoenix app that has a cowboy websocket handler to get around not using Phoenix channels.
I went with this route so others didn't have implement the Phoenix channels protocol in other languages. I also wanted something that was extremely stable, since Phoenix Channels has bumped versions fairly recently I don't want to have my library bump and break any existing connected game.
When connecting via Mudlet, the package down auto downloads the world map. This should save a lot of wandering time for players to get a fully functional mapper.
I also got a patch into Mudlet based on some functionality I pulled out of the ExVenture package. This is available as part of Mudlet 3.10.
Mudlet has been working on Discord integration, and ExVenture is ready whenever that ships. I have basic status setting functionality set up at the moment.
To sort of build myself up to what I thought I would need for the overworld, I started adding some nicer interfaces to the admin panel via React components. The first two I did were effects and quest scripts.
Effects are now done via a nice form that dumps to raw JSON. No longer do you need to edit that huge textarea by hand!
Quest scripts are now filled out by a nice form instead of of a text area and I also have a script tester right in the panel. You can see how the NPC would react without having to head in game.
Room exits got a massive overhaul in preparation for the overworld (and continued afterwards.) They are now one way, but the admin panel makes and destroys both directions. This drastically simplified the usage of exits inside of the code base.
The only tricky part was getting the reverse exit to be created for the overworld, but I will take one tricky spot over it being littered throughout the whole app.
As a bonus, this let me finally add in diagonal movement. You can go north east now.
I revamped the character stats. Movement points are gone, replaced with Endurance Points. They aren't used yet, but I would really like to let skills use Skill Points or Endurance Points. I might also rename Skill Points sometime in the future to better reflect the split. I want Endurance Points to be more physical based, and Skill Points to be more "mind" based.
The basic attributes have also been reworked. The new six attributes are:
They all go in pairs of two to boost the HP, SP, and EP. Strength and Agility boost HP, Intelligence and Awareness boost SP, and Vitality and Willpower boost EP. This feels a lot better than how it was previously and has at least 100% more thought put into what they are and how they're named.
The web client got a major face lift over the weekend. It now has a sidebar chat to track channel messages, so they don't get lost in the flow. I also tweaked the bottom bar to feel better, paddings and margins are now similar, along with font and font size. Some panels also got squeezed and moved around. Hopefully the new client looks much nicer to you as it does to me.
I hope to continue with the overworld and Gossip integration. These are both big features that will be fun to continue with in the coming months.
I am in the process of reading The Art of Game Design by Jesse Schell so I expect some changes to come from that. Namely I want to start thinking of how I want people to experience the game and start tweaking it to better reflect that. I expect this to be a slow process but highly worth the thought put into it.
This week I wrapped up the absolute basics of ExVenture's largest new feature, the overworld.
This was a huge feature that's barely been scratched.
First up was I wanted to make a nice way to edit the overworld in the browser. This was my first big React app. You can see it in action below:
I have a lot of ideas on how to make this nicer, but the editor works for now. Things I want to add are: a nicer color picker, which will display the color instead of just using the word; and some way to paint bucket large areas with symbols and colors.
I may also set up a terrain swatch to bundle together a symbol and color (and eventually cell attributes) into a nice picker.
After the map is created, you need to hook it up with rooms so players can move in and out of the map. This is another react page that does live API updates.
Once exits are created, players will be able to move around freely between the room based zones and the overworld based zones.
Here is movement on MidMUD, going from the room based town of Milay to the countryside around it:
Overworld zones are set up as a pool of Sector processes. Each sector handles a portion of the grid of cells in a map, 10x10 right now. The calling or casting process determines which sector to talk to so this should scale well.
Only minor updates for the commands were needed to get the overworld working. Mostly the updates were stripping functionality and making sure they did less on the overworld. Commands like involving items in particular were broken on the overworld, because I didn't want to implement item storage on the overworld quite yet.
I've started a GitHub issue to track all the things that are still missing. The big ones right now are:
Items in the overworld
NPCs in the overworld
Say communication should go further than your current cell
Seeing other players on the map
I am looking forward to seeing the overworld slowly fill out now that the basics of being able to get to it and move around are done. It is a huge relief to have this part over.
Mudlet is a client that can be used to connect to MidMUD or any other MUD. They have a way for the server to push a client package down to the client. This lets Mudlet be customized for MidMUD and any ExVenture server.
Through the server push mechanism, you can also auto-update the package. This lets the package stay lock-step with the server as new things are added and changed.
The package now contains and auto-mapper, stat gauges, and a tabbed chat.
Telnet Connection Flow
To go together with the Mudlet additions, I worked on the telnet connection flow. Previously you needed to get a one time password and enter that into you chat.
I changed it up to work more like how Netflix authorizes a remote device. The telnet connection gets an ID and registers it as a session waiting to be authorized. You then click the authorize link the connection is giving you. On this page you can authorize the connection and be signed in when you switch back to your client.
This is so much more convenient than copy pasting a password and I think its just as secure.
Large Scale Metrics
I spent an afternoon a few weeks ago trying to figure out how well ExVenture will scale as more and more data is pushed in and thousands of processes spin up. ExVenture performed wonderfully in my testing. I only tested the number of NPCs performing movement.
I set up roughly 14,000 rooms with 20,000 NPC processes wandering around them. This was on a quad-core Macbook Pro. The 99 percentile for movement was 350 microseconds. I am still in a slight shock about how much Erlang did not care about data size.
As expected, turning ExVenture into a distributed app came with a significant number of bugs. Luckily I had some help on the ExVenture Discord in finding them. I set up Sentry to start recording new exceptions as they came. This lets me see what was going on and record them better than a log I never look at.
A summary of the bugs I found so far:
Channel communication was not spanning the cluster
Keeping players out of the game until it is online
Rebalance zones as nodes drop out
Current connected players were locking up
Lots of functions were not expecting bad data back
Strange transient data errors, I think due to processes not rebooting properly
A room can crash often enough to tank the entire supervision tree and kill a node
Session recovery exponential back off
Not all of these are fixed, namely the room tanking the entire tree, but most are. Getting in session recovery to be exponential is a big win. A player connected could generate thousands of exceptions by how I had it before.
The MUD Coders Guild had a Game Jam going on during the beginning of May. In the last moments I joined with MidMUD in the cheapest feature I could think of: a random name selection for players signing up.
This is a simple feature, but one that I am glad to get in. Picking a name has to be the hardest part of making a character, so simplifying this for new people is a good win.
Home Page Updates
The home page now includes a web chat client! This was fun to add since it was the prototypical phoenix application. Phoenix channels are finally being used "right" in ExVenture!
The admin UI also got an update. I had been watching [Refactoring UI][refactoring-ui] and became inspired to make the admin look less crappy. There are now flash messages on all actions, and the forms should look a lot better.
If you attach an email to your account, you can trigger a password reset.
If you try using a skill that exists but you don't have, you get a nicer message for it
Get all items in a room at once
Delete rooms from the admin
Handle web client disconnects, by reconnecting
Send mail from the home page
Disable user accounts
Bug: Adding an item spawning did not trigger its timer
Bug: Session recovery did not start regen
Quests always show their progress as 100% after completion
Bug: several bugs in the telnet protocol surrounding IAC
Bug: web page redirects after signing in was slightly broken
NPCs can have delayed actions
NPCs can have mutliple actions in a single event, queued up
Channel chatter is recorded for replay when connecting to the web client
scan command to view the surrounding area
Who list tweaks
Commands can be marked admin only
Automatic balancing of NPCs, change their level and the stats boost to the minimum for that level
Continuous effect for stat boost
Skills can default target yourself
Target yourself via self
Hide yourself when looking at a room
Format chat messages by capitalizing and adding punctuation
Bug: NPCs targeted you before entering a room
GMCP heartbeat message
Lock movement when skills are cooling down
GMCP skill status message
Next month I am going to continue with bug fixing while the app is in clustered mode. I have been getting some suggestions about what to tackle next so I will probably continue with that. I also wouldn't mind getting to in game forums in the next month.
I might also start switching these to a faster update cycle if I continue to have this many changes.