M39cybXZRMOz2dTiGFIk_build-a-realtime-chat-server-with-go.png.jpg

Modern web applications are becoming more and complex. Users are often greeted with an experience that is both reactive and engaging. Pages update in real time without the user having to initiate calls to the server or refreshing their browser. In the early days, developers relied on AJAX requests to create applications that were pretty close to realtime. Now they're able to use the power of WebSockets to create fully realtime applications.

In this tutorial we'll create a realtime chat application using the Go programming language and WebSockets. The frontend will be written using HTML5 and VueJS. A basic understanding of the Go language, JavaScript and HTML5 are assumed. It's also assumed that you have a little bit of experience working with VueJS.

To get started with Go, you can check out the excellent interactive tutorial on the official Go website. https://tour.golang.org/welcome/1

And for Vue you can check out the excellent free video series by Jeffrey Way at Laracasts. https://laracasts.com/series/learn-vue-2-step-by-step

What Is a WebSocket?

Normal web applications are served up using at least one or more requests to an HTTP server. A piece of client software, normally a web browser sends a request to the server and the server sends back a response. The response is usually HTML which the browser then renders as a web page. Stylesheets, JavaScript code and images can also be sent back in a response to complete the entire web page. Each request and response is part of its own separate connection and a large website like Facebook can actually yield hundreds of these connections just to render a single page.

AJAX works the exact same way. Using JavaScript, developers can make requests to the HTTP server for little pieces of information and then update a single part of the webpage based on the response. This can all be done without having to refresh the browser. This still has some limitations though.

Each HTTP request/response connection is closed after the response and to get any new information another connection must be made. The server has no idea the client is looking for new information without a new request to tell it so. One technique to make AJAX applications seem realtime is to run AJAX requests in a timed loop. After a set interval, the application can rerun a request to the server to see if there has been any update which needs to be reflected in the browser. This works fine for small applications but is not very efficient. This is where WebSockets come in handy.

WebSockets are part of a prosposed standard created by the Internet Engineering Task Force (IETF). The full specification of how WebSockets should be implemented are detailed in RFC6455. Here is how the document defines a WebSocket.

The WebSocket Protocol enables two-way communication between a client running untrusted code in a controlled environment to a remote host that has opted-in to communications from that code.

In other words, a WebSocket is a connection that is always open and allows a client and server to send messages back and forth unprompted. The server can push new information to the client whenever it deems it necessary and the client can do the same to the server.

WebSockets in JavaScript

Most modern browsers have support for WebSockets in their JavaScript implementation. To initiate a WebSocket connection from the browser you can use the simple WebSocket JavaScript object like this.

    var ws = new Websocket("ws://example.com/ws");

The only argument you need is a URL to where the WebSocket connection is going to be accepted by the server. The request is actually an HTTP request initially but we use "ws://" or "wss://" for a secure connection. This lets the server know that we are trying to create a new WebSocket connection. The server will then "upgrade" the connection to a persistent two-way connection between the client and server.

Once a new WebSocket object is created and the connected is successfully created we can use the "send()" method to send text to the server and define a handler function on our WebSocket's "onmessage" property to do something with messages sent from the server. This will be explained later in our chat application code.

WebSockets In Go

WebSockets are not included as part of the Go standard library but thankfully there are a few nice third-party packages that make working with WebSockets a breeze. In this example we will use a package called "gorilla/websocket" which is part of the popular Gorilla Toolkit collection of packages for creating web applications in Go. To install it, simply run the following.

$ go get github.com/gorilla/websocket

Building the Server

The first piece of this application is going to be the server. It will be a simple HTTP server that handles requests. It will serve up our HTML5 and JavaScript code as well as complete the setup of WebSocket connections from clients. Going a step further, the server will also keep track of each WebSocket connection and relay chat messages sent from one client to all other clients connected by WebSocket. Start by creating a new empty directory then inside that directory, create a "src" and "public" directory. Inside the "src" directory create a file called "main.go".

The first part of the application is some setup. We start our application like all Go applications and define our package namespace, in this case "main". Next we import some useful packages. "log" and "net/http" are both part of the standard library and will be used to log (duh) and create a simple HTTP server. The final package, "github.com/gorilla/websocket", will help us easily create and work with our WebSocket connections.

package main

import (
        "log"
        "net/http"

        "github.com/gorilla/websocket"
)

The next two lines are some global variables that will be used by the rest of the app. Global variables are usually a bad practice but we will use them this time for simplicity. The first variable is a map where the key is actually a pointer to a WebSocket. The value is just a boolean. The value isn't actually needed but we are using a map because it is easier than an array to append and delete items.

The second variable is a channel that will act as a queue for messages sent by clients. Later in the code, we will define a goroutine to read new messages from the channel and then send them to the other clients connected to the server.

var clients = make(map[*websocket.Conn]bool) // connected clients
var broadcast = make(chan Message)           // broadcast channel

Next we create an instance of an upgrader. This is just an object with methods for taking a normal HTTP connection and upgrading it to a WebSocket as we'll see later in the code.

// Configure the upgrader
var upgrader = websocket.Upgrader{}

Finally we'll define an object to hold our messages. It's a simple struct with some string attributes for an email address, a username and the actual message. We'll use the email to display a unique avatar provided by the popular Gravatar service.

The text surrounded by backticks is just metadata which helps Go serialize and unserialize the Message object to and from JSON.

// Define our message object
type Message struct {
        Email    string `json:"email"`
        Username string `json:"username"`
        Message  string `json:"message"`
}

The main entry point of any Go application is always the "main()" function. The code is pretty simple. We first create a static fileserver and tie that to the "/" route so that when a user accesses the site they will be able to view index.html and any assets. In this example we will have an "app.js" file for our JavaScript code and a simple "style.css" for any styling.

func main() {
        // Create a simple file server
        fs := http.FileServer(http.Dir("../public"))
        http.Handle("/", fs)

The next route we want to define is "/ws" which is where we will handle any requests for initiating a WebSocket. We pass it the name of a function called "handleConnections" which we will define later.

func main() {
    ...
        // Configure websocket route
        http.HandleFunc("/ws", handleConnections)

In the next step we start a goroutine called "handleMessages". This is a concurrent process that will run along side the rest of the application that will only take messages from the broadcast channel from before and the pass them to clients over their respective WebSocket connection. Concurrency in Go is one of it's greatest features. How it all works is beyond the scope of this article but you can check it out in action for yourself on Go's official tutorial site. Just think of concurrent processes or goroutines as backround processes or asynchronous functions if you're familiar with JavaScript.

func main() {
    ...
        // Start listening for incoming chat messages
        go handleMessages()

Finally, we print a helpful message to the console and start the webserver. If there are any errors we log them and exit the application.

func main() {
    ...
        // Start the server on localhost port 8000 and log any errors
        log.Println("http server started on :8000")
        err := http.ListenAndServe(":8000", nil)
        if err != nil {
                log.Fatal("ListenAndServe: ", err)
        }
}

Next we need to create the function to handle our incoming WebSocket connections. First we use the upgrader's "Upgrade()" method to change our initial GET request to a full on WebSocket. If there is an error, we log it but don't exit. Also take note of the defer statement. This is neat way to let Go know to close out our WebSocket connection when the function returns. This saves us from writing multiple "Close()" statements depending on how the function returns.

func handleConnections(w http.ResponseWriter, r *http.Request) {
        // Upgrade initial GET request to a websocket
        ws, err := upgrader.Upgrade(w, r, nil)
        if err != nil {
                log.Fatal(err)
        }
        // Make sure we close the connection when the function returns
        defer ws.Close()

Next we register our new client by adding it to the global "clients" map we created earlier.

func handleConnections(w http.ResponseWriter, r *http.Request) {
    ...
        // Register our new client
        clients[ws] = true

The final piece is an inifinite loop that continuously waits for a new message to be written to the WebSocket, unserializes it from JSON to a Message object and then throws it into the broadcast channel. Our "handleMessages()" goroutine can then take it can send it to everyone else that is connected.

If there is some kind of error with reading from the socket, we assume the client has disconnected for some reason or another. We log the error and remove that client from our global "clients" map so we don't try to read from or send new messages to that client.

Another thing to note is that HTTP route handler functions are run as goroutines. This allows the HTTP server to handle multiple incoming connections without having to wait for another connection to finish.

func handleConnections(w http.ResponseWriter, r *http.Request) {
    ...
        for {
                var msg Message
                // Read in a new message as JSON and map it to a Message object
                err := ws.ReadJSON(&msg)
                if err != nil {
                        log.Printf("error: %v", err)
                        delete(clients, ws)
                        break
                }
                // Send the newly received message to the broadcast channel
                broadcast <- msg
        }
}

The final piece of the server is the "handleMessages()" function. This is simply a loop that continuously reads from the "broadcast" channel and then relays the message to all of our clients over their respective WebSocket connection. Again, if there is an error with writing to the WebSocket, we close the connection and remove it from the "clients" map.

func handleMessages() {
        for {
                // Grab the next message from the broadcast channel
                msg := <-broadcast
                // Send it out to every client that is currently connected
                for client := range clients {
                        err := client.WriteJSON(msg)
                        if err != nil {
                                log.Printf("error: %v", err)
                                client.Close()
                                delete(clients, client)
                        }
                }
        }
}

Building the Client

A chat application wouldn't be complete without a pretty UI. We'll create a simple, clean interface using some HTML5 and VueJS. We'll also take advantage of some libraries like Materialize CSS and EmojiOne for some styling and emoji goodness. Inside the "public" directory, create a new file called "index.html".

The first piece is pretty basic. We also pull in some stylesheets and fonts to make everything pretty. "style.css" is our own stylesheet to customize a few things.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Simple Chat</title>

    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.97.8/css/materialize.min.css">
    <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons">
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/emojione/2.2.6/assets/css/emojione.min.css"/>
    <link rel="stylesheet" href="/style.css">

</head>

The next piece is just the interface. It's just a few fields to handle choosing a username and sending messages along with displaying new chat messages. The details of working with VueJS are beyond the scope of this article but check out the documentation if you're unfamiliar.

<body>
<header>
    <nav>
        <div class="nav-wrapper">
            <a href="/" class="brand-logo right">Simple Chat</a>
        </div>
    </nav>
</header>
<main id="app">
    <div class="row">
        <div class="col s12">
            <div class="card horizontal">
                <div id="chat-messages" class="card-content" v-html="chatContent">
                </div>
            </div>
        </div>
    </div>
    <div class="row" v-if="joined">
        <div class="input-field col s8">
            <input type="text" v-model="newMsg" @keyup.enter="send">
        </div>
        <div class="input-field col s4">
            <button class="waves-effect waves-light btn" @click="send">
                <i class="material-icons right">chat</i>
                Send
            </button>
        </div>
    </div>
    <div class="row" v-if="!joined">
        <div class="input-field col s8">
            <input type="email" v-model.trim="email" placeholder="Email">
        </div>
        <div class="input-field col s8">
            <input type="text" v-model.trim="username" placeholder="Username">
        </div>
        <div class="input-field col s4">
            <button class="waves-effect waves-light btn" @click="join()">
                <i class="material-icons right">done</i>
                Join
            </button>
        </div>
    </div>
</main>
<footer class="page-footer">
</footer>

The last piece is just importing all the required JavaScript libraries to inclue Vue, EmojiOne, jQuery and Materialize. We also need an MD5 library to grab URLs for avatars from Gravatar. This will be explained better when we tackle the JavaScript code. The last import, "app.js", is our custom code.

<script src="https://unpkg.com/vue@2.1.3/dist/vue.min.js"></script>
<script src="https://cdn.jsdelivr.net/emojione/2.2.6/lib/js/emojione.min.js"></script>
<script src="https://code.jquery.com/jquery-2.1.1.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.2/rollups/md5.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/materialize/0.97.8/js/materialize.min.js"></script>
<script src="/app.js"></script>
</body>
</html>

Next create a file called "style.css" in the "public" directory. Here we'll just place a little bit of styling.

body {
    display: flex;
    min-height: 100vh;
    flex-direction: column;
}

main {
    flex: 1 0 auto;
}

#chat-messages {
    min-height: 10vh;
    height: 60vh;
    width: 100%;
    overflow-y: scroll;
}

The final part of the client is our JavaScript code. Create a new file in the "public" directory called "app.js".

As with any VueJS application we start by creating a new Vue object. We mount it to a div with the id of "#app". This allows anything within that div to share scope with our Vue instance. Next we define a few variables.

new Vue({
    el: '#app',

    data: {
        ws: null, // Our websocket
        newMsg: '', // Holds new messages to be sent to the server
        chatContent: '', // A running list of chat messages displayed on the screen
        email: null, // Email address used for grabbing an avatar
        username: null, // Our username
        joined: false // True if email and username have been filled in
    },

Vue provides an attribute called "created" which is meant to be a function you define that handles anything you want to as soon as the Vue instance has been created. This is helpful for any setup work you need to do for the application. In this case we want to create a new WebSocket connection with the server and create a handler for when new messages are sent from the server. We store the new WebSocket in our "ws" variable created in the "data" property.

The "addEventListener()" method takes a function that will be used to handle incoming messages. We expect all messages to be a JSON string so we parse it so that it is an object literal. Then we can use the different properties to format a pretty message line complete with an avatar. The "gravatarURL()" method will be explained later. Also we are using a nifty library called EmojiOne to parse emoji codes. The "toImage()" method will turn those emoji codes into actual images. For example if you type ":robot:" it will be replaced with a robot emoji.


    created: function() {
        var self = this;
        this.ws = new WebSocket('ws://' + window.location.host + '/ws');
        this.ws.addEventListener('message', function(e) {
            var msg = JSON.parse(e.data);
            self.chatContent += '<div class="chip">'
                    + '<img src="' + self.gravatarURL(msg.email) + '">' // Avatar
                    + msg.username
                + '</div>'
                + emojione.toImage(msg.message) + '<br/>'; // Parse emojis

            var element = document.getElementById('chat-messages');
            element.scrollTop = element.scrollHeight; // Auto scroll to the bottom
        });
    },

The "methods" property is where we define any functions we would like to use in our VueJS app. The "send" method handles sending messages to the server. First we make sure the message isn't blank. Then we format the message as an object and then "stringify()" it so that the server can parse it. We use a little jQuery trick to escape HTML and JavaScript from any incoming message. This prevents any sort of injection attacks.

    methods: {
        send: function () {
            if (this.newMsg != '') {
                this.ws.send(
                    JSON.stringify({
                        email: this.email,
                        username: this.username,
                        message: $('<p>').html(this.newMsg).text() // Strip out html
                    }
                ));
                this.newMsg = ''; // Reset newMsg
            }
        },

The "join" function will make sure the user enters an email and username before they can send any messages. Once they do, we set join to "true" and allow them to start chatting. Again we strip out any HTML or JavaScript.


        join: function () {
            if (!this.email) {
                Materialize.toast('You must enter an email', 2000);
                return
            }
            if (!this.username) {
                Materialize.toast('You must choose a username', 2000);
                return
            }
            this.email = $('<p>').html(this.email).text();
            this.username = $('<p>').html(this.username).text();
            this.joined = true;
        },

The final function is a nice little helper function for grabbing the avatar URL from Gravatar. The final piece of the URL needs to be an MD5 encoded string based on the user's email address. MD5 is a one way encryption algorithm so it helps keep emails private while at the same time allowing the email to be used as a unique identifier.

        gravatarURL: function(email) {
            return 'http://www.gravatar.com/avatar/' + CryptoJS.MD5(email);
        }
    }
});

Running the Application

To run the application, open a console window and make sure you are in the "src" directory of your application then run the following command.

$ go run main.go

Next open a web browser and navigate to "http://localhost:8000". The chat screen will be displayed and you can now enter an email and username.

To see how the app works with multiple users, just open another browser tab or window and navigate to "http://localhost:8000". Enter a different email and username. Take turns sending messages from both windows.

Conclusion

This just a basic chat application but there are many more improvements you can make to it. I challenge you to play around with the source code and add some other features, see if you can implement private messaging or notifications when a new user joins or leaves the chat. The sky is the limit!

I hope you found this helpful and are now inspired to start creating your own realtime applications using WebSockets and Go.

How to Trace Execution of Commands in Shell Script with Shell Tracing