Running Kubernetes on Amazon Web Services with Rancher

As everyone is aware, Amazon has EC2 Container Services, the Amazon solutions for running docker containers. I haven't had much luck with this, so now I'm testing Rancher and Kubernetes on Amazon Web Services.

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

Rancher is a complete platform for running containers within Enterprises, providing enterprise-level access control and container orchestration.

I started first with creating a new Virtual Private Cloud, using the default wizard. This vpc is being used for the complete Rancher farm. Next thing is to create one or more new hosts, where the Rancher documentation is saying to install docker manually, I've chosen the Rancher OS AMI itself. That AMI is especially built for this purpose and very minimal.... I mean really minimal, even curl isn't installed.

The security group enables traffic to tcp port 8080 for my ip only. This is important, as there is no authorisation enabled by default. It is also a good idea to add traffic between the several hosts itself, that can be done by saving the security group (then it will get an sg- identifier), and add All Traffic permissions to that security group.

When the new instance is ready, just connect using ssh to it and start the rancher server docker image using sudo docker run -d --restart=always -p 8080:8080 rancher/server. The rancher applications will run within docker images themselves.

$ sudo docker run -d --restart=always -p 8080:8080 rancher/server
Unable to find image 'rancher/server:latest' locally  
latest: Pulling from rancher/server  
Status: Downloaded newer image for rancher/server:latest  

You can tail the logs using the docker logs -f {containerid} command:

[rancher@ip-10-0-0-29 ~]$ sudo docker logs -f 3baee4a975200c3eb4b18fd1351b07393cdd09455cf7917d46879a278399c02e
160901  8:40:30 [Note] /usr/sbin/mysqld (mysqld 5.5.49-0ubuntu0.14.04.1) starting as process 25 ...  
Uptime: 8  Threads: 1  Questions: 2  Slow queries: 0  Opens: 33  Flush tables: 1  Open tables: 26  Queries per second avg: 0.250  
Setting up database  
Importing schema  

Perfect. Rancher Server is starting, and wil be finished soon. You can browse to the ip of the instance, by default Rancher Server will run on port 8080. Important is that the public ip will be reachable for all hosts, this will be an issue when having the hosts in a vpc, so you can use the private ip instead. This can be changed at Admin and Settings.

When Rancher Server is up and running, we need to add some docker hosts. You can launch a new host (the More Like This option is really awesome, it just starts another instance of the same configuration), or add the same host as server (but that's not a best practice.)

Next add a new host within Rancher Server, click on Infrastructure, and next on Add Host.

When the host(s) have been added, you can add the Kubernetes Environment. Kubernetes will then be deployed automatically to the new host(s).

This will take a while, and you can monitor the current status by selecting the environment.

As the kubernetes environment finishes, you navigate to Catalog and deploy a container. For example select the K8s Example Ghost container. The container will be up and running in no time.

During the deployment of Kubernetes the stack is created with the following instances:

  • Rancher Ingress Controller 1 (rancher/ingress-controller): The Rancher Ingress Controller will leverage the existing load balancing functionality within Rancher and convert what is in Kubernetes ingress to a load balancer in Rancher.
  • Rancher Server (rancher/server): The Rancher Management Server, this will run the web frontend and api.
  • Rancher Agent (rancher/agent): Each node will get an individual agent, which will manage the node.
  • Rancher Kubernetes Agent 1 (rancher/kubernetes-agent): Agent responsible for handling communications between Rancher and Kubernetes .
  • Rancher Agent Instance(rancher/agent-instance): Agent instance image for Rancher.
  • Kubernetes KubeCTLD 1 (rancher/kubectld): An embarrassingly simple microservice to expose kubectl create/apply/get logic.
  • Kubernetes Kubernetes 1 (rancher/k8s): Kubernetes provides the Kubernetes API Server.
  • Kubernetes ETCD 1 (rancher/etcd): Etcd is the highly-available key value store used for persistent, safe and distributed storage of objects and metadata.
  • Kubernetes Proxy 1 (rancher/k8s): The Kubernetes network proxy runs on each node.
  • Kubernetes Scheduler 1 (rancher/k8s): The Kubernetes scheduler is a policy-rich, topology-aware, workload-specific function that significantly impacts availability, performance, and capacity.
  • Kubernetes Controller Manager 1 (rancher/k8s): The Kubernetes controller manager is a daemon that embeds the core control loops shipped with Kubernetes.
  • Kubelet 1 (rancher/k8s): The kubelet is the primary “node agent” that runs on each node.

When adding an extra host the following will be added as well:

  • Kubernetes Proxy 2
  • Kubernetes Etcd 2
  • Kubelet 2

Because I don't want to maintain my own registry, I've created a Amazon Container Registry, and registered the registry within Rancher. You can just copy the address, username AWS (note, this is case sensitive) and password (the long base64 encoded string).

You can get the credentials using:

aws ecr get-login --region us-east-1  

Then build the docker image:

docker build -t dutchcoders .  

Tag the image and register it:

docker tag dutchcoders:latest {url}/dutchcoders:latest  

Push the image to the registry:

docker push {url}/dutchcoders:latest  

If you want to read more about running Kubernetes with Rancher, check this and join our Kubernetes LinkedIn group here.

Let me know if you've got any questions regarding running Rancher with Kubernetes on Amazon AWS, you can reach me on twitter @remco_verhoef