Rails on Kubernetes - Part 1

Overview

This is a lengthy topic that is split into multiple parts:

  • Part 1: Rails and Docker Compose
  • Part 2: Kubernetes
  • Part 3: Deployments, Rolling updates and Scaling (Coming soon)

Do I need this ?

The short answer is: It depends™.

The traditional way to deploy Rails looks more or less like this:

  • Provision your servers with your favourite automation tool like Puppet, Chet, Ansible or Bash :)
  • Capistrano deploys
  • Nginx as a reverse proxy
  • Puma, unicorn, thin etc. as your app server

Capistrano makes it easy to deploy on multiple servers but you still need to provision those manually.

If you throw in a queue for background work, use caching as much as possible and create proper DB indexes you'll have a legit setup.

Complete deployment

If you want to skip straight to the code you can access the sample Rails app on Github.

Kubernetes

I won't spend time explaining what Kubernetes is in this post as there are plenty of resources out there. Using Docker or rkt to run your applications makes your infrastructure much more portable. Instead of having to provision servers and deploy the code there, you package your system libraries and your application code in a container image. There are so many official and community images to choose from you will not have to worry about configuring your own servers from scratch. Need an Nginx reverse proxy ? Easy: FROM nginx:latest. Handling things like major security updates for your OS can be as easy as pulling from a newer image.

Since you will now need to package your app in a Docker image you will have a few more moving parts. You will need a private registry and the tooling around building and pushing new images.

This is definitely a trade-off: you will now have to use kubectl for your deployments instead of cap deploy. But look at the pros:

  • Development using containers guarantees a consistent environment for every developer.
  • Need to upgrade Ruby ? It's as easy as packaging your app using the latest ruby base, test that and use rolling updates to push it live.
  • When you're scaling horizontally to more Rails servers you don't need to worry about updating Nginx configs. Ingresses and Ingress Controllers will handle that.

Take this sample setup for example:

Rails on Kubernetes

We have a few separate services here: Rails application, Redis instance, Sidekiq workers and a Postgres instance.

Depending on what your bottlenecks are you may want to easily spin up more Sidekiq workers for example. Or maybe you get a lot authenticated traffic to your main Rails app and want more app servers. With Kubernetes this is as easy as running:

kubectl scale --replicas=3 rc/rails  

The change in the numbers of replicas will propagate all the way to the Ingress Controller. Taking an Nginx IG as an example, the update will mean adding the new services to the upstream config block in the virtual host.

The rest of this article will go through a hypothetical setup step-by-step.

Prerequisites

If you intend to follow the steps here on your machine here's what you need to have installed:

Rails and Docker Compose

You could skip this step if you prefer to keep your current development workflow. You can always run the Rails app using the bundled puma server locally. Having said that, if you get the docker-compose setup running, switching to Kubernetes is going to be a breeze.

I will use ruby-2.3.1 and Rails 5.1.1 for this write-up. The simple Rails app I will create for this article looks like this:

  • Devise for authentication
  • Postgresql database
  • Redis instance
  • Sidekiq worker for sending emails in the background.

Let's go ahead and create the app:

rails new rails-kube-demo --database=postgresql  

And add devise to our Gemfile:

gem 'devise'  
gem 'sidekiq'  

We can run bundle install locally, followed by the default Devise setup steps:

rails generate devise:install  
rails generate devise user  

This will create the user.rb model along with the required migrations and routes. We don't need to run the migrations now as we will configure that in the next steps.

Docker-compose

First let's define the Dockerfile for our Rails application. This is take straight from the docker-compose guides on Rails. I added the netcat package to help us detect when services inside containers are up and running (more on that later).

FROM ruby:2.3.3

RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs netcat  
RUN mkdir /myapp

WORKDIR /myapp

ADD Gemfile /myapp/Gemfile  
ADD Gemfile.lock /myapp/Gemfile.lock

RUN bundle install  
ADD . /myapp  

Now we'll define docker-compose.yml and add the simple postgresl section:

version: '3'  
services:  
  db:
    image: postgres
  web:
    build: .
    command: bundle exec rails s -p 3000 -b '0.0.0.0'
    volumes:
      - .:/myapp
    ports:
      - "3000:3000"
    depends_on:
      - db

Before building our images let's create a .dockerignore file and add our log and tmp folders:

tmp/*  
log/*  

Let's test this out:

docker-compose build && docker-compose up  

We can now fire up a browser and navigate to http://localhost:3000. We should see the Rails default route.

Database config and migrations

Now that we know the images are building properly let's configure the database connection.

The first thing I always to is to check the image's documentation for the environment variables I can pass through. The ones we are interested in are:

  • POSTGRES_USER
  • POSTGRES_PASSWORD
  • POSTGRES_DB
  • PGDATA
version: '3'  
services:  
  setup:
    build: .
    depends_on:
      - db
    environment:
      - RAILS_ENV=development
    entrypoint: ./setup.sh
  db:
    image: postgres
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=mysecurepass
      - POSTGRES_DB=rails-kube-demo_development
      - PGDATA=/var/lib/postgresql/data
  db_data:
    image: postgres
    volumes:
      - /var/lib/postgresql/data
    command: /bin/true
  web:
    build: .
    command: bundle exec rails s -p 3000 -b '0.0.0.0'
    volumes:
      - .:/myapp
    ports:
      - "3000:3000"
    depends_on:
      - db

We created a volume-only image that we mount on the postgresql container to persist the db data.

Now we update our database.yml file correspondingly:

default:  
  adapter: postgresql
  encoding: unicode
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  username: postgres
  password: mysecurepass
  host: db
  port: 5432

development:  
  <<: *default
  database: rails-kube-demo_development

test:  
  <<: *default
  database: rails-kube-demo_test

We now need a way to run our migrations when the db container is up. Our setup container depends on the db container but but don't have a guarantee that Postgres will start before we attempt to run the migrations. You can use something like wait-for-it or roll your own bash script. We'll do the latter:

#!/bin/bash

set -e

echo "Waiting for Postgres to start..."

while ! nc -z db 5432; do sleep 0.1; done

echo "Postgres is up - executing command"

exec bin/rails db:migrate

Make this file executable and note that we are referencing it in the entrypoint: ./setup.sh of the setup container:

chmod +x setup.sh  

Let's run docker-compose build once again so that our setup.sh script is copied over. Then we can fire up the stack:

$ docker-compose up
setup_1    | Waiting for Postgres to start...  
setup_1    | Postgres is up - executing command  
setup_1    | == 20170524183835 DeviseCreateUsers: migrating ================================  
setup_1    | -- create_table(:users)  
setup_1    |    -> 0.0294s  
setup_1    | -- add_index(:users, :email, {:unique=>true})  
setup_1    |    -> 0.0166s  
setup_1    | -- add_index(:users, :reset_password_token, {:unique=>true})  
setup_1    |    -> 0.0152s  
setup_1    | == 20170524183835 DeviseCreateUsers: migrated (0.0613s) =======================  
setup_1    |  

Note that the sample output is out of order. I just wanted to emphasize that our script and migrations worked.

We should now be able to register a new account at: http://localhost:3000/users/sign_up

Sidekiq and Redis

Let's move on to the Sidekiq queue. I configured the defaulturloptions so that ActiveMailer doesn't complain about the host and port to my config/environments/development.rb:

config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }  

Next, let's create config/initializers/sidekiq.rb and configure our Redis connection. Since our sidekiq worker will not sit on the same server with Rails we have to configure both the client and the server here. We're using two ENV variables to pass on the Redis URL and PORT:

Sidekiq.configure_client do |config|  
  config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0"}
end

Sidekiq.configure_server do |config|  
  config.redis = { url: "redis://#{ENV['REDIS_URL']}:#{ENV['REDIS_PORT']}/0"}
end  

This is my config/sidekiq.yml that we will start the sidekiq worker with. Pretty standard except the mailers queue. ActiveJob will push jobs in the mailers queue and this is how we tell Sidekiq to also listen in for jobs.:

---
:concurrency: 25
:pidfile: ./tmp/pids/sidekiq.pid
:logfile: ./log/sidekiq.log
:queues:
  - default
  - mailers

I configured ActiveJob to use Sidekiq in my config/application.rb:

config.active_job.queue_adapter = :sidekiq  

And added an override to force Devise to send emails through the queue (in app/models/user.rb):

class User < ApplicationRecord  
  # Include default devise modules. Others available are:
  # :confirmable, :lockable, :timeoutable and :omniauthable
  devise :database_authenticatable, :registerable,
         :recoverable, :rememberable, :trackable, :validatable

  def send_devise_notification(notification, *args)
    devise_mailer.send(notification, self, *args).deliver_later
  end
end  

I updated the routes.rb to include the Sidekiq Dashboard engine:

  require 'sidekiq/web'
  mount Sidekiq::Web => '/sidekiq'

This will make it easier to test once we deploy everything.

We also need to start the worker as soon as Redis becomes available. To do that we'll use a similar bash script: sidekiq.sh. Don't forget to chmod +x.

#!/bin/bash

set -e

echo "Waiting for Redis to start..."

while ! nc -z redis 6379; do sleep 0.1; done

echo "Redis is up - executing command"

exec bundle exec sidekiq -C config/sidekiq.yml  

And finally, update the docker-compose.yml with the the sidekiq and redis containers:

version: '3'  
services:  
  setup:
    build: .
    depends_on:
      - db
    environment:
      - RAILS_ENV=development
    entrypoint: ./setup.sh
  db:
    image: postgres
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=mysecurepass
      - POSTGRES_DB=rails-kube-demo_development
      - PGDATA=/var/lib/postgresql/data
  db_data:
    image: postgres
    volumes:
      - /var/lib/postgresql/data
    command: /bin/true
  sidekiq:
    build: .
    environment:
      - REDIS_URL=redis
      - REDIS_PORT=6379
    depends_on:
      - redis
    entrypoint: ./sidekiq.sh
  redis:
    image: redis:3.2
    ports:
      - "6379:6379"
  web:
    build: .
    depends_on:
      - redis
      - db
      - setup
    command: bundle exec rails s -p 3000 -b '0.0.0.0'
    environment:
      - REDIS_URL=redis
      - REDIS_PORT=6379
    volumes:
      - .:/myapp
    ports:
      - "3000:3000"
    depends_on:
      - db

We can run docker-compose build && docker-compose up to update our deploy.

Check out http://localhost:3000/sidekiq for the Sidekiq dashboard.

The easiest way to test that Sidekiq is running those tasks is to create a user at http://localhost:3000/users/sign_up and then request a new password at http://localhost:3000/users/password/new. You should be able to see the job processed in the Sidekiq Dashboard.

Next Steps

We now have a working Rails/Sidekiq application running in Docker. In Part 2 we will take our docker-compose.yaml file and translate it into Kubernetes resources.