In a past post I wrote about distributed systems, but I intentionally omitted the subject of leaderless replication since I consider it a topic that deserves a little more focus.

In this post I’m going to explore how a leaderless system works and explain a little about how the Raft algorithm helps us achieve consensus.

Leaderless replication

As the name suggests, there are no leaders (or masters) in a leaderless setup. In this configuration all instances can receive reads and writes. If you read my previous post, you might be thinking that this sounds like master-master replication, but it is actually very different.

I mentioned two main problems in a master-master setup: Replication lag when you write to one master and then read from the other, and conflicts when you modify the same record in both masters and they try to sync. Leaderless replication doesn’t have these problems (it has others that I’ll explore soon). On top of not having those problems, a leaderless system can stay up even when instances are down (like the master-master configuration).

In computer science a distributed system is a software system in which different parts of it communicate by passing messages through a network. The different parts could be running in the same machine or distributed across the globe; as long as they communicate through an unreliable channel (a network), we can classify them as distributed and consider the challenges that come with it.

With this definition in mind, we could think about many examples of distributed systems. A single monolithic application communicating with a database could be considered a distributed system if the application communicates through a network protocol.

Although in practice, local networks can be pretty reliable, they are still vulnerable. There are two condition that can cause a plethora of problems to a system: the network being down or the network being slow. These two conditions can put the system in a wide variety of states that may give results we don’t expect.

Before we look at how these problems can affect a distributed system, lets look at a distributed system whose failure mode is mostly understood and accepted to this date: a stateless system.

I want to start using Kubernetes to manage my services. Before I go all-in I will do some playing around with their easy-to-install version that can run on a single machine.

Requirements

Before we can run Minikube locally we need to have a virtualization solution installed in our host. Since my computer supports KVM, I decided to go with it. You can check if your computer supports KVM with this command:

1
egrep -c '(vmx|svm)' /proc/cpuinfo

In a previous post I talked a little about creating installable web apps. One important thing that I didn’t cover is how to automatically prompt a user on your web app to add it to their home screen.

Serve over https

One of the requirements for chrome to show an app install banner is that the app is served over HTTPS. This can be done in a few ways depending on your server. I wrote a post on how to get free SSL certificates using Let’s encrypt that should help you get started.

To modify the names of your Desktop and Downloads folder you just modify ~/.config/user-dirs.dirs:

1
2
XDG_DESKTOP_DIR="$HOME/desktop"
XDG_DOWNLOAD_DIR="$HOME/downloads"

I just learned that my load balancer is vulnerable to the POODLE attack due to SSL 3. The recommended solution is to disable SSL 3.

I explained my HAProxy setup in a previous post, and also how I do SSL termination.

The section from my configuration I care about is:

1
2
3
4
5
6
frontend https-in
        bind *:443 ssl crt /certs/ncona.pem

        acl ncona-web-frontend hdr(host) -i ncona.com www.ncona.com

        use_backend ncona-web if ncona-web-frontend

This mode is called SSL offloading in HAProxy terms. Fixing it is as simple as adding a keyword (no-sslv3):

1
        bind *:443 ssl crt /certs/ncona.pem no-sslv3

As I move forward with my Beego project, I have reached a point where I need to authenticate my users. In the flow that I’m looking for, the client (web app, mobile app, etc…), will communicate directly with the Auth provider (Google, Facebook, etc…) and get a JWT. The only thing the server needs to do is validate the JWT. If the validation succeeds, it means the user is logged in.

If you are not familiar with JWT, you can read my previous article that explains how JWT works.

Authentication flow

Since the authentication with Google is going to happen entirely on the client, the server logic becomes a lot simpler. For my application, all endpoints will require the user to be logged in, so I will create a middleware to verify this. The middleware will expect a valid JWT in the Authorization header. If this requirement is not met, the server will return a 401.

Before we start creating models, we need to make sure our app can connect to our database. At the time of this writing Beego supports MySQL, PostgreSQL and Sqlite3.

We can have the ORM connect to our database from our main.go file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
package main

import (
    "os"
    "strings"

    _ "app/routers"

    "github.com/astaxie/beego"
    "github.com/astaxie/beego/orm"
    _ "github.com/go-sql-driver/mysql"
)

func init() {
    orm.RegisterDriver("mysql", orm.DRMySQL)
    parts := []string{os.Getenv("MYSQL_USER"), ":", os.Getenv("MYSQL_PASSWORD"),
            "@tcp(", os.Getenv("MYSQL_HOST"), ":3306)/", os.Getenv("MYSQL_DATABASE")}
    orm.RegisterDataBase("default", "mysql", strings.Join(parts, ""))
}

func main() {
    if beego.BConfig.RunMode == "dev" {
        beego.BConfig.WebConfig.DirectoryIndex = true
        beego.BConfig.WebConfig.StaticDir["/swagger"] = "swagger"
    }
    beego.Run()
}

A few weeks ago I wrote an article showing how to start a simple Beego server. Today I’m going to go a little further and explain how to start building your database using migrations. For my examples I will use a MySQL server with an empty database (no tables) named beego.

It is possible to just log in to a database server and type the SQL commands necessary to create or modify tables when we need to, but using migrations helps keep track of the changes made to the database over time. This provides something similar to version control at the database level.

If we need to create a new table, we would start by creating a new migration file using the bee tool:

1
bee generate migration create_user_table

I tried GoLang for the first time a little less than two years ago and I didn’t get a very good first impression of it. One of the things that I disliked the most is that back then the community seemed to be overly focused in performance even when that meant creating unreadable code.

One of the things that bothered me the most was that the community was against creating web frameworks with opinions. Because of this lack of standardization, I had to relearn how things are done every time I looked at a new web project written in Go.

Today I decided to look again into the state of web frameworks and there seems to be a few promising options out there. I saw a good amount of people recommending Beego, so I decided to give it a try.