In a previous post I covered the basics of Terraform. In this post I’m going to cover a few more things that I find necessary in most infrastructures I create.

The machines

I’m going to start with a couple of machines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
// Configure Google Cloud
provider "google" {
  credentials = "${file("credentials.json")}"
  project = "ncona-1504"
  version = "~> 1.13"
}

// Machines
resource "google_compute_instance" "us-central1-c--f1-micro--001" {
  name         = "us-central1-c--f1-micro--001"
  machine_type = "f1-micro"
  zone         = "us-central1-c"

  boot_disk {
    initialize_params {
      image = "ubuntu-1604-xenial-v20170815a"
    }
  }

  network_interface {
    network = "default"
  }
}

resource "google_compute_instance" "us-central1-c--f1-micro--002" {
  name         = "us-central1-c--f1-micro--002"
  machine_type = "f1-micro"
  zone         = "us-central1-c"

  boot_disk {
    initialize_params {
      image = "ubuntu-1604-xenial-v20170815a"
    }
  }

  network_interface {
    network = "default"
  }
}

Google uses the concept of Virtual Private Cloud (VPC) to refer to their capability for creating your own private network withing their infrastructure. There are a few terms that will allow us to create a network of our design:

• Network – This is a virtual (because everything is virtual in the cloud) network that can span across the globe
• Subnet – This is an IP range that can be used by machines in a single region
• Firewall – Used to limit communication between machines in the same network

Network

A network (or VPC) is just a name used to group your network infrastructure. Subnets are defined inside a network and each host is part of one subnet.

Subnets

Subnets can be defined by region. You can choose any IP range defined as private as specified in RFC-1918 (basically anything inside these ranges: 10.0.0.0 – 10.255.255.255, 172.16.0.0 – 172.31.255.255 and 192.168.0.0 – 192.168.255.255).

By default ctrlp will look for the root of your repo (by looking for a .git, .hg, .svn or .bzr file) and then start searching for files in that folder. For the project I’m currently working on that has too many files (Probably millions) I prefer that ctrlp only searches inside the folder were I started vim. This can be done with this setting in .vimrc:

1
let g:ctrlp_working_path_mode = 'a'

A few days ago, while playing with Terraform I realized that I want Terraform to manage some instances that I had already created in Google Cloud. Because these instances existed before I was using Terraform, it doesn’t konw anything about them.

The first thing I had to do to take over them, was to add them to the configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
resource "google_compute_instance" "us-central1-c--f1-micro--001" {
  name         = "us-central1-c--f1-micro--001"
  machine_type = "f1-micro"
  zone         = "us-central1-c"

  boot_disk {
    initialize_params {
      image = "ubuntu-1604-xenial-v20170815a"
    }
  }

  network_interface {
    network = "default"
  }
}

Once I had the machine in my configuration, I just had to tell Terraform, which machine is that:

1
terraform import google_compute_instance.us-central1-c--f1-micro--001 ncona-179804/us-central1-c/us-central1-c--f1-micro--001

From here terraform can manage it as if it had created it.

Terraform is a tool for managing infrastructure. It works with most cloud providers out there, but it can also (although with considerably more work) integrate with any custom solutions a company might have.

Managing infrastructure

What does it mean to manage infrastructure? Although different infrastructure management systems have different features, there are a few features that are present in most systems:

• Asset lifecycle – Adding or removing assets (machines, load balancers, switches, etc…)
• Networking – Managing how the assets communicate (networks, routes, permissions, etc…)
• Security – Secure the network and machines so it can only be accesses by the intended people
• Updates – Update the system software when necessary for security or stability reasons

Why Terraform?

There are many tools out there that allow you to manage your infrastructure (Chef, Ansible, Salt, etc…). I have worked in companies that run successfully using Chef, Ansible and Salt, so I’m sure any of the tools can be made to work. I’m writing this post about Terraform because I heard good things about it from people I admire. Two things that usually come up are an easy way to manage your whole infrastructure using configuration files, as well as making optimal changes that can be applied quickly without affecting dependent services.

Linux (and most other operating systems) provide a transparent layer of caching for auxiliary storage (hard drives, etc…). This layer allows fast access to frequently used files on disk by keeping their content in memory and reading from there when necessary.

The kernel can use any free space in RAM as page cache. If the system requires more memory, the kernel might free space used by this cache and provide it to the application that needs it.

Inspecting page cache

Page cache is stored in RAM, but the space can be reclaimed by the kernel for applications whenever necessary. In Linux we can see how many bytes of RAM are used for page cache:

1
2
3
4
~ $ free -h
              total        used        free      shared  buff/cache   available
Mem:            19G        5.4G        7.8G        1.5G        6.2G         12G
Swap:           19G          0B         19G

I’ve heard a few times during my career references to file descriptors without knowing much about them. Today I’m going to try to understand what they are.

File descriptors are a Unix concept. They refer to a way of referencing a file or other resources (stdin, stdout, etc…). They are referenced by unsigned integers. Some old versions of Unix used to limit the number of file descriptors per process to 20 (0 to 19), but nowadays, there is no actual limit on the number of file descriptors a process can have.

Each running process is assigned a file descriptors table. The table for a process can be found at /proc/PID/fd. We can see the file descriptors used by a process like this:

This is a refresher of sorting algorithms since I recently realized that I don’t remember how a lot of the most common sorting algorithms work. I’m only going to focus on arrays on this article since it is the most common structure for these kind of problems.

Bubble sort

This is the first algorithm we learn at school. It is not very efficient(O(n ^ 2) in most of the cases) but it is pretty easy to implement.

• Grab the first two elements(0 and 1) in an array and compare them
• If the element in the left is higher than the elements in the right then swap them
• Grab the next two elements(1 and 2) and do the same
• Repeat until the greatest element is in the far right
• Do the same starting from the first two elements(0 and 1) but ending before reaching the last element(which is already sorted)
• At the end the array will be sorted

Since I’ve been working in C++, I’ve noticed the necessity to get familiar with concepts about computer architecture and how the operating system works. One thing that has been bothering me for a few weeks is hearing people talk about the heap and stack memory, while I don’t really know what those things mean.

I decided to write this post to try to explain to myself what is the difference between stack memory and heap memory and hopefully understand why people keep talking about them.

The names

Stack and Heap are both names of data structures. This can lead to some confusion, because although the stack is called like that because it works similar to a stack (I’ll explain more about this), the heap is not related at all to the heap data structure. The exact origin of the heap word in this context is unknown to me, but from a little research it seems like its use is based on the English language definition for that word:

1
An untidy collection of things piled up haphazardly

Which actually describes pretty well what a heap of memory is.

The time has come for me to start writing some tests for my C++ code, and I have to admit I’m a little nervous. The company I’m working for uses Google Test as their test framework, so I will trust their expertise and use it too.

Set up

Lets start by creating a folder for our project:

1
2
mkdir ~/project
cd ~/project/

The next step is to download and unzip google test:

1
2
3
wget https://github.com/google/googletest/archive/release-1.8.0.tar.gz
tar -zxf release-1.8.0.tar.gz
rm release-1.8.0.tar.gz