CoreOS from scratch - Part 2: etcd

In previous post you have learned how to install three CoreOS instances on VirtualBox. Today we'll focus on creating etcd cluster.

Etcd is distributed key value store. Using etcd, CoreOS cluster shares configuration parameters. Etcd is used as well for sharing configuration information's between deployed applications.

There are several ways of how to create CoreOS cluster https://coreos.com/etcd/docs/latest/clustering.html '. We'll use public etcd discovery service one.

Start all CoreOS instances

First step is to boot all three CoreOS instances. After instances are up and running use ssh client to connect.

Obtain etcd token

Since we'll be using public etcd discovery service, we'll need to obtain unique token which will be used for our cluster members. Token can be obtained at:

https://discovery.etcd.io/new?size=3

Save returned token. We will use the same one for each cluster member.

Edit user_data file

During installation, your cloud-config file is coppied to:

/var/lib/coreos-install/user_data

On every of the three nodes edit file using sudo and look for the section

#  etcd2:

#    discovery: "https://discovery.etcd.io/<token>"

#    listen-peer-urls: "http://10.160.1.1X:2380,http://10.160.1.1X:7001"

#    initial-advertise-peer-urls: "http://10.160.1.1X:2380"

#    listen-client-urls: "http://0.0.0.0:2379,http://0.0.0.0:4001"

#    advertise-client-urls: "http://10.160.1.1X:2379"

  units:

#    - name: etcd2.service

#      command: start

Uncomment lines and replace X on all four places to match inner network IP on each host. For first CoreOS instance after modification of above lines, section will contain following lines. Replace <token> with the one you have obtained from etcd discovery service.

  etcd2:

    discovery: "https://discovery.etcd.io/7fd8daaee..."

    listen-peer-urls: "http://10.160.1.11:2380,http://10.160.1.11:7001"

    initial-advertise-peer-urls: "http://10.160.1.11:2380"

    listen-client-urls: "http://0.0.0.0:2379,http://0.0.0.0:4001"

    advertise-client-urls: "http://10.160.1.11:2379"

  units:

    - name: etcd2.service

      command: start

Save configs and exit editors. You are done with configuration.

Reinitialize cluster

Now on each node enter:

sudo coreos-cloudinit --from-file /var/lib/coreos-install/user_data

This must be made on each node of cluster or else cluster initialization will not be successful ! 

You should see output similar to next one:

2016/05/02 20:01:50 Calling unit command "start" on "etcd2.service"'

2016/05/02 20:01:59 Result of "start" on "etcd2.service": done

core@localhost ~ $

If you see output similar to above one, on all three cluster nodes, all went well. Now just briefly check discovery address url:

https://discovery.etcd.io/7fd8daaee...

As you can see output contains all your three nodes, which is another sign of successful installation.

etcd test

To see cluster functionality enter:

etcdctl cluster-health

member 9fdb79133bec4a5 is healthy: got healthy result from http://10.160.1.12:2379

member a9f53f93fdfb47a1 is healthy: got healthy result from http://10.160.1.11:2379

member de52eec7a95fe2ba is healthy: got healthy result from http://10.160.1.13:2379

cluster is healthy

Which again is a good sign.

Now create key/value pair on one cluster node:

core@localhost ~ $ etcdctl set /key value1

value1

and read it on another one

core@localhost ~ $ etcdctl get /key

value1

Congratulation. You have successfully clustered three CoreOS instances.

CoreOS from scratch - Part 1: OS installation