Recent Posts

ExVenture Updates for November 2018

Posted on 28 Nov 2018

The last month of ExVenture has been pretty good. ExVenture was on an Elixir podcast and did an introduction meetup with UtahElixir. There were also a lot of Gossip related updates.

Links for MidMUD & ExVenture:

Podcasts

I was on the Elixir Mix podcast, episode 27. On it, I talked about all of the different projects that ExVenture encompasses. I also went into some of the topics I covered in the Going Multi-Node ElixirConf talk I gave. Definitely give it a listen!

I also talked with UtahElixir about ExVenture and introduced it to the group. Mark Ericksen is an organizer for UtahElixir and wants to use ExVenture as a way of teaching various elixir topics and wants to go over various parts of it over the next few meetups. How cool!

Multiple Characters

I finally wrapped up the split apart from a single user account, to a user account that has characters. This was a fairly long process as it touched a lot of pieces of the code. I have been slowly going through everything so as to not feel like it was a suffocating refactor and the work finally paid off.

I was very happy with how easy this final refactor ended up being after the positioning ExVenture for this with previous refactors.

You can see this in Pull Request #90.

Grapvine Login

I have been talking with the MUD Coders Guild for a what's next thing for Grapevine and we came up with the idea of Grapvine being an OAuth provider. I have previously written one so I figured it'd be fun to do again. With multiple characters in place, now was the time to add it into ExVenture as well.

I couldn't use existing OAuth code because they ship with their own backing tables, and I wanted to reuse the games table. If you update ExVenture and are connected to Gossip, you now support Grapevine authentication. It takes some small extra configuration on Gossip's end to get redirect URIs in place, but otherwise that's it.

You can see this in Pull Request #91 and Pull Request #4.

Gossip

Gossip has gotten some work as well. I rewrote the underlying sync code to push around versioned payloads. This lets me capture deletes in addition to creates and updates.

With this in place, I was able to add in in-game events. You can add events to your game and they will be displayed on Grapevine. I would like to get a nice calendar view for this, but I went with simple until some data starts getting added in.

Gossip and Grapevine also got some minor styling tweaks. The homepages for each should better explain what they are.

Gossip also tracks player count over time now and generates nice looking charts. This is viewable in Grapevine.

Grapevine Player Counts

Gossip Refactors

The Gossip server and clients both got major refactors to their socket code. I was able to do a cool event router of sorts for the server. This can be seen here.

The elixir client for Gossip was also updated and recieved a 1.0 version. I blogged about this recently, check it out.

Small Tweaks

  • Grapevine - display game connection status
  • Refactor Game.Format into smaller sub modules
  • Fix the overworld map editor to allow for adding exits
  • Gossip - delay sending game offline messages
  • Grapevine - password resets
  • Update raisin deps
  • Gossip client - telemetry events
  • Gossip - migrate to telemetry
  • Gossip - require players to be online to send tells
  • Social emotes now actually broadcast to the room you're in

Social Updates

This month was pretty good for ExVenture on the social front. Lots of new stars across all of the projects!

I went to The Big Elixir earlier this month and gave my Going Multi-Node talk again. I met a lot of new people who were previously into MUDs or thought they seemed cool and are new to MUDs. Welcome everyone I met from The Big Elixir.

There are several new patrons over at the Patreon as well. Welcome and thanks for supporting the project!

The discord group has several new members in the last month.

Next Month

I am planning on refactoring the event system behind NPCs next. I am currently scoping out what I would like it to look like so this should happen. It will be a big undertaking but I think well worth the effort.

I am still looking at achievements as well. Progress is slow but it is happening. I am still trying to figure out some sync issues with them but they should be do-able. With the new sync in place for Gossip -> Grapevine these should be pretty easy to get going as well.

Writing an Evented WebSocket Client

Posted on 15 Nov 2018

I recently did a pretty big refactor for the Gossip Elixir client. I'd like to show off what I did for that. For a brief background, Gossip sends events over the websocket connection. A sample event might be:

{
  "event": "channels/broadcast",
  "ref": "89036074-446f-41ab-b87a-44ef1f962f2e",
  "payload": {
    "channel": "gossip",
    "message": "Hello everyone!",
    "game": "ExVenture",
    "name": "Player"
  }
}

You can see all of the events over at the Gossip Docs.

Client Flow

The client connects to Gossip via the Gossip.Socket server. This is a Websockex process, which is a little different than your standard GenServer. It might eventually be swapped out to be a Gun client.

The flow of data goes as such:

New websocket frame

Full code

defmodule Gossip.Socket do
  # ...

  def handle_frame({:text, message}, state) do
    case Events.receive(state, message) do
      {:ok, state} ->
        {:ok, state}

      {:reply, message, state} ->
        {:reply, {:text, Poison.encode!(message)}, state}

      # other return values
    end
  end

  # ...
end

When the client gets a new websocket frame, the socket process calls down to a lower level module.

Handling the event

Full code

defmodule Gossip.Socket.Events do
  # ...

  def receive(state, message) do
    with {:ok, message} <- Poison.decode(message),
         {:ok, state} <- process(state, message) do
      {:ok, state}
    else
      {:reply, message, state} ->
        {:reply, message, state}
    end
  end

  # ...
end

The frame is decoded to an event and then processed.

Pattern matching on the event

Full code

defmodule Gossip.Socket.Events do
  # ...

  def process(state, message = %{"event" => "channels/" <> _}) do
    Core.handle_receive(state, message)
  end

  def process(state, message = %{"event" => "players/" <> _}) do
    Players.handle_receive(state, message)
  end

  # ...
end

Each event has a general scheme of noun/verb, such as channels/broadcast. This pattern matches on the just the noun part and pushes it lower into the module for processing.

Pattern matched on the specific event

Full code

Full code

defmodule Gossip.Socket.Core do
  # ...

  def handle_receive(state, message = %{"event" => "channels/broadcast"}) do
    process_channel_broadcast(state, message)
  end

  def process_channel_broadcast(state, %{"payload" => payload}) do
    message = %Message{
      channel: payload["channel"],
      game: payload["game"],
      name: payload["name"],
      message: payload["message"],
    }

    core_module(state).message_broadcast(message)

    {:ok, state}
  end

  # ...
end

Here the event is fully pattern matched and calls to the internal function. I like to do it this way to keep it similar to GenServers and keeping the handle_* functions skinny.

Testing

With the client broken up into fairly small pieces, it helps encourage testing and keeps it simple to test.

Prior to this refactor, the Gossip client had almost no tests. This is due to each event module being broken up into separate elixir modules and having separate processing functions for each event.

See testing in action.

Conclusion

I hope you poke around the rest of the client. I also recommend taking a look around the server side of Gossip since that got a big refactor as well. One really cool section is the "event router" macro I set up to generate the server side receive functions.

Nginx TLS Socket Termination

Posted on 15 Nov 2018

For MidMUD, I have a secure endpoint that you can connect to the game with. This is simply a TLS wrapper around a standard telnet connection. It's very simple to set up, and assumes that you already have Let's Encrypt set up on your main domain.

You can also load balance with this, which MidMUD uses to balance across the cluster.

This must be set up at the top level nginx.conf as there are no virtual host semantics for raw sockets.

nginx.conf

# other config

stream {
  upstream telnet {
    server game-01.example.com:5555;
    server game-02.example.com:5555;
    # add as many or as few of these as you need
  }

  server {
    listen [::]:5555;
    listen 5555;

    proxy_pass telnet;
  }

  server {
    listen [::]:5443 ssl;
    listen 5443 ssl;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    proxy_pass telnet;
  }
}

This has nginx listening on 5555 and 5443 as a plain text TCP socket and a secure TLS socket. Both of which proxy to the local TCP socket which should be in your secure network, and may also be on the same host.

Hopefully this helps you configure your own plaintext TCP socket to be more secure!

ExVenture Updates for October 2018

Posted on 26 Oct 2018

The last month of ExVenture was a lot of minor refactors and other chores that needed to happen plus a migration back to working on Gossip.

Links for MidMUD & ExVenture:

Gossip

I started working on Gossip again. There is a new support flag that you can support, which loads basic information about the games that are online, games. You can also receive events for when a game connects and disconnects now. This should help you keep local caches more up to date. See the Gossip Docs for more information about the new events.

Gossip also got a HUGE refactor of it's socket code. It had started getting fairly spaghetti like so I spent a weekend refactoring it to be much nicer, this pull request shows how that went.

Grapevine

I also launched a new site called Grapevine that is for players on the Gossip network. Games have a profile page now, which shows the basic game information plus current players and how to connect and play yourself.

Players also have a profile which shows the in game characters you've registered. Grapevine is a full Gossip application which means you can send a tell to it. This allows grapevine to know who your player is and can link the character to your profile.

There is a lot of work on this site that I want to do in the future. The biggest of which will be achievements. See this issue on Gossip's GitHub if you're interested in helping shape what these look like.

Gossip Elixir Client

The Gossip elixir client also got a major refactor thats about to wrap up. It was getting large enough to finally have some patterns take root and be able to refactor to be nicer (it might have also been drifting towards spaghetti code as well oops.) This pull request shows this refactor in action.

The client also got updated to include all of the current features that Gossip the server supports. You can also not support every feature that Gossip and the client can handle by providing only the callback modules that you wish to support. I need to update the docs for this but expect a 1.0 client soon!

Raisin

Games supporting Gossip is starting to have a minor upswing and I want to be ready for when this takes off. To help with that I started Raisin to handle moderation. Raisin is a network application like Grapevine and logs the goings on of the Gossip network. This is all it does at the moment, but this is the place where network admins will be able to moderate the community.

Backbone Sync

To support Grapevine and Raisin, Gossip needed a new level of connecting game which I've called a network application. This application gets socket super powers when it connects and starts to receive sync events. When Grapevine loads it gets a set of sync/games and sync/channels events, and will continue to get them as anything changes.

To keep code duplication minimal, when I started Raisin I pulled out the common sync code into a separate OTP application. It manages its own repo and keeps sync code in one spot. gosisp-backbone is up on GitHub.

Small Tweaks

  • API representers to handle outputting in various formats
  • Remote erlang QR code library for an elixir version
  • Player saves get updated via a common function
  • Create a character struct for a user, continuing the migration to multiple characters per account
  • Gossip update indexes to use the SQL lower() function
  • Gossip can have user agent repo links
  • Gossip blocks certain game names
  • Gossip can have connections for your game
  • Gossip's your game page got a spruce up

Social Updates

This month was pretty good for ExVenture on the social front. Lots of new stars across all of the projects!

There are several new patrons over at the Patreon as well. Welcome and thanks for supporting the project!

The discord group has several new members in the last month.

I was also promoted to be an admin of The MUD Coders Guild this month. I look forward to helping make Gossip and co a more official part of the guild. It was already unofficially official since there is a #guild-outreach channel where we all talked about new features, but now we can take it to bigger and better heights!

Next Month

Over the next month I will most likely keep going with Gosisp, Grapevine, and Raisin. In particularly Raisin needs to be able to have some power on the network. That will be fun to write the events for.

I also want to keep moving on achievements. I will most likely add them into ExVenture first, before starting on Gossip itself. I would like to be ahead of the network for Gossip instead of letting ExVenture play catch up.

Gossip Sync Websocket Protocol

Posted on 16 Oct 2018

In the last few weeks I started on a new project that is tied to Gossip, the chat network for text-based games (aka MUDs.) This project is called Grapevine. Grapevine is a player site where you can register your characters across any game that is connected to Gossip.

Links for Gossip & Grapevine:

Sync Protocol

For Grapevine, I wanted to have it connect with the standard Elixir client for Gosisp which meant it needed to live on the same socket as a standard game. I went with adding in a Gossip network-level application that pretends to be a game for most parts, but when one connects it gives the socket super powers.

The biggest of those super powers at the moment is recieving sync/* events. When the socket connects it gets a set of sync events for all of the channels and games that Gossip knows about. Each event is bundled up into 10 channels or games, to help out a little bit with network overhead.

A sync event looks similar to this:

{
  "event": "sync/channels",
  "payload": {
    "channels": [
      {"id": 1, "name": "gossip", "description": "...", "hidden": false}
    ]
  }
}

When Grapevine receives this it creates or updates a local version of that remote record. I called this the backbone sync, you can view it here on Gossip's side.

These events also trigger for any creates or updates while the socket is connected. When a new game is made or a game is edited, Gossip broadcasts an internal sync event that the socket is subscribed to. This immediately pushes out and keeps the remote applications up to date always.

Raisin

One of the main reasons for wanting to head this socket route, is that I also wanted other higher privileged applications to live on the network. The next one being Raisin, which will be a moderation tool for the network.

With the start of Raisin, I wanted to reuse the same sync code across both projects. I had previously done this for Rails applications via Rails Engines, but I wasn't sure how to accomplish this. After talking with some fine folks over at the MUD Coders Guild, I think I stumbled across something that works.

I pulled out all of the common code into a new library that Grapevine and Raisin depend on. This library hosts its own ecto repo so it can internally save the data. Configuration wise, it's the same database as Grapevine or Raisin. That lets you use the backbone internal schemas outside of the backbone library.

I'll have more to say about this as I continue to explore it.

Conclusion

It is extremely exciting to watch these sync events propagate. The closest I've seen to this are webhooks, but being connnected to a duplexed socket is another thing entirely.

I hope you all check out the code backing all of this, and maybe even decide to join the network yourself as a player!

Creative Commons License
This site's content is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License unless otherwise specified. Code on this site is licensed under the MIT License unless otherwise specified.