Running #Gollum on #Azure Web App on #Linux


Introduction

A couple of years ago we started to use Gollum as our internal Knowledge Base system. Even if we’re heavy users of Sharepoint, we found the native wiki feature to be unsatisfactory, third party solutions as well as the idea of developing a custom solution a pain to maintain. We started running it on a VM on Azure, but then came the Azure App Services for Linux preview, so why don’t try to containerize Gollum and run it as an App Service?
This article tells the journey to have Gollum running on Azure App Services, including:

  • build on the v5.x branch
  • integration with Azure Active Directory for authentication
  • use of Azure Storage to backup the wiki repository
  • use of Azure Storage to inject on the fly customization and startup parameters
  • be perfectly integrated with Azure App Services, so that for example we can ssh into the container
  • use a VM on Azure as the building environment
  • push the image to a private container registry on Azure and use it from there

You may ask why use App Services when Azure has Azure Containers, Fabric Services, support for container orchestrators (Kubernetes, DC/OS, Swarm). The answer is easy, for Gollum we don’t need any of those, it’s a Ruby on Rails web app that uses a git repository to keep wiki pages edited by users.

As a side note, I’m new to containers, so feel free to comment and emend and make this a community project.

Current limitations

Azure App Services for Linux are still in preview so they’re not feature completed, if something on the cloud can ever be feature completed. The biggest stopper I found is the lack of custom storage. While the solution has storage mapped in /home and shared between all the instances of the App, it’s not possible to bring your own storage: the initial idea was to map an Azure File Share to host the wiki repository, alas this is not possible. Fortunately in this case we don’t really need any scaleout capability, but still we had to add some sort of backup of the repo because the sahred storage is tied to the App Service, deleting it not just deletes the container but even the shared storage.

Step 1 – preparing the docker VM

As my dev machine I used an Ubuntu VM on Azure with the Azure >Docker Extension that basically installs docker on the VM for you. The process is really straight forward unless you want to use your own certificates and not the self signed ones. The entire certificate story is getting ridiculous, certificates management is not easy per se, if the industry defines different formats and ways to use them, well this is something that cannot last long. Anyway, if you, like me, want to use a certificate from a Windows CA, you must:

  • enroll the .pfx files (pkcs12 format) for your docker machine fqdn name. The certificate must include all the intermediate CAs in the certificate chain.
  • install openssl
  • run openssl pkcs12 -in [certificate].pfx -clcerts -nokeys -out [certificate]-crt.pem and openssl pkcs12 -in [certificate].pfx -nocerts -nodes -out [certificate]-key.pem to get the public certificate and the private key
  • export your root CA certificate in base 64 encoded X.509 format with a .pem extension

Once the development VM is ready, you can start having fun.

Step 2 – preparing the docker image

The project with the resources to build your image is hosted on github.
The dockerfile, starting with the base ruby image implements the following steps:

  • defines the environment variables to be used to customize the container, see the paragraph on container customization
  • installs the gollum prerequisites
  • build gollum from the 5.x branch, this can be changed if you want to build form master. This is neat so that every time I rebuild the image I have the latest commit on v5.x. Ad the same time I decided to include it in thee image and not build it on the fly in the init container script.
  • adds Azure cli2 support to be able to leverage azure storage for customizations and backup
  • adds ssh server support
  • adds cron support and schedules daily backups to an azure storage account. If and when the App will need multiple instances this needs to be adjusted to run just on one instance. It copies save_wiki.cron and save_wiki.py into the image.
  • adds Omniauth/Omnigollum support to be able to integrate with Azure AD. The Omnigollum files are included in the repo becase the gem file includes gollum and we want to control our Gollum deployment. In the future we can clone directly from the original github repository, patch on the fly the gem file and build omnigollum directly
  • lastly runs a python script to init the container

The init_container.py script, performs the initialization needed by the container and then applies the customization if any:

  • starts the ssh server
  • starts the cron daemon
  • if the wiki repository is not yet initialized git init it (/home/wiki)
  • if customization storage account has been specified, copies everything in the wiki root and git commit it (for example to inject custom css and js files)
  • if a Gollum startup file has been specified, downlods it and copies it in /home/conf and lastly starts Gollum with the config file, if not just starts Gollum
FROM ruby

LABEL MAINTAINER daniele.grandini@live.it
ENV PORT=80

# Accessing Azure File Share is not yet supported as any other external storage commenting out the relevant code
#ENV SHARE="azurefsshare"
#ENV SHAREPWD="azurefssharekey"
#ENV SHAREACCT="azurestorageaccountname"

ENV AZURE_STORAGE_KEY=""
ENV AZURE_STORAGE_URL=""
ENV GOLLUMCONF=""
ENV GOLLUMCONF_KEY=""
ENV GOLLUMCUSTOM=""


#Gollum prereq layer
RUN apt-get -qq update \
  && apt-get -y --fix-missing install libicu-dev cmake build-essential make ruby-dev libicu-dev zlib1g-dev \
  && rm -rf /var/lib/apt/lists/* \
  && gem install bundler 

#Gollum binaries from branch 5.x
RUN git clone https://github.com/gollum/gollum /gollum
WORKDIR /gollum
RUN git checkout -b 5.x origin/5.x \
  && bundle install

#RUN gem install github-linguist
#RUN gem install gollum
#RUN gem install org-ruby  # optional

#here we must use a file share
#RUN apt-get -qq update \ 
  #&& apt-get -y install cifs-utils \
  #&& mkdir /wiki \
  #&& git init /wiki

#WORKDIR /wiki

# ------------------------
# Azure cli2 support
# ------------------------

RUN  apt-get update -qq && apt-get install -y apt-transport-https \
  && echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ wheezy main" | tee /etc/apt/sources.list.d/azure-cli.list \
  && apt-key adv --keyserver packages.microsoft.com --recv-keys 417A0893 \
  && apt-get update -qq && apt-get install -y azure-cli

# ------------------------
# SSH Server support
# ------------------------
RUN apt-get -qq update \ 
  && apt-get install -y --no-install-recommends openssh-server \
  && echo "root:Docker!" | chpasswd

COPY sshd_config /etc/ssh/

# ------------------------
# CRON support
# here we can have a race condition with more than one node, how can I check if I there are other nodes?
# ------------------------
RUN apt-get update -qq && apt-get -y install cron
COPY save_wiki.py /usr/bin/
ADD save_wiki.cron /etc/cron.daily/save_wiki
RUN chmod 0644 /etc/cron.daily/save_wiki

# ------------------------
# Ominiauth support
# Just copy the binaries of the modified omniauth and build the gem
# ------------------------

COPY omnigollum /omnigollum/
WORKDIR /omnigollum
RUN gem build omnigollum.gemspec \
  && gem install omnigollum*.gem \
  && gem install omniauth-azure-oauth2

# ------------------------
# Init container
# ------------------------
#COPY init_container.sh /usr/bin/
COPY init_container.py /usr/bin/

#RUN chmod 755 /usr/bin/init_container.sh 
RUN chmod 755 /usr/bin/init_container.py 
EXPOSE 2222 $PORT

CMD ["/usr/bin/init_container.py"]

To build the image, you can just copy or clone the github repo and copy them for example in /gollum. You can then easily build the image running: sudo docker build -t gollum:latest /gollum

Step 3 – publish the image to the Azure docker repository

Azure Web Application for Linux takes three kind of images:

  • builtin, directly managed and built by Microsoft
  • from a public repository
  • from a private repository

The natural choice was a private repository and Azure provides the infrastructure you need with “Azure Container Registry”, My advice is to go with managed registries, but for this Gollum solution a basic one is enough. It’s really easy and in a few seconds you have the registry set up for you.

2017-07-22_15-09-08.png

Once the registry is ready in Azure you must push the image to it, to do this you need first to login: _**sudo docker login .azurecr.io -u -p **_

2017-07-22_15-29-56.png

You can also use Azure AD accounts to login to the registry

And then you can push the image let’s say in a repository called gollum. First create an alias for the remote image: sudo docker tag gollum:latest .azurecr.io/gollum/gollum:latest, then just publish the image sudo docker push .azurecr.io/gollum/gollum:latest
In this example I’m tagging this image with ‘latest’ so if in the future I want to keep multiple versions of the image I can use ‘latest’ to address the ahem latest version.
And here we are ready to start our container on Azure Web App for Linux.

Step 4 – create and run your Azure Application

To create a Web App On Linux, you can just follow the common Ibiza portal experience:

  1. Create the Web App On Linux
  2. Give it a unique name .azurewebsites.net
  3. Select a Subscription
  4. Set or Create a Rosurce Group
  5. Create the App Service Plan, this is what you’re billed for
  6. Select the image you just uploaded
    a. the image name must be in the format .azurecr.io/gollum/gollum:latest
    b. the server url is https://.azurecr.io
    c. login and password are the one you used to push the image

2017-07-22_16-02-07.gif

Now you can start your Web App and your Gollum site is ready to go. Obviously you cannot stop here, because by default Gollum is not authenticated and thus wide open to the entire Internet, not a good choice.

2017-07-22_16-08-38.png

Before going any further I suggest to bind your own domain name and an SSL certificate to protect the communication to rou web site. The steps are really easy and well explained in the Ibiza portal in the app settings:

2017-07-22_16-11-52.png

Customizing the container

My idea was to create a Gollum image with the latest bits, but leave all the customization outside the image itself and inject them at container startup. Since we’re running on Azure I chose Azure Storage as the repository of the customization. To keep it secure I used the following environment variables:

  • GOLLUMCONF – when specified must contain the url of an Azure Storage Blob container where the customization files are copied. This is the root container where the init_container.py script searches for config.rb. This file is the passed to Gollum as a parameter. For information on Config.rb check the official Gollum documentation and the Omnigollum Example and some more sampleshttps://gist.github.com/roachhd/f2cf0f10cd478adbaf4b). Example https://gollum.blob.core.windows.net/config
  • GOLLUMCUSTOM – if specified contains the folder inside GOLLUMCONF with the files to be copied into the wiki repository. This can be used to add custom js and css to Gollum and a custom template page for your wikis, too. Example: wiki
  • GOLLUMCONF_KEY – must contain the key for the storage account
  • AZURE_STORAGE_URL – if specified enables wiki pages backup to an Azure Storage Blob container. Example: https://gollum.blob.core.windows.net/wiki
  • AZURE_STORAGE_KEY – must contain the key for the Azure storage used to backup the wiki

Enabling OminiGollum

A stated before by default Gollum is not authenticated and this is obviously not good if you expose it to the public Internet. Since we’re running on Azure the natural choice is to integrate with Azure AD. To do this we use a customization of OmniAuth for Gollum: Omnigollum.

To enable Omnigollum you must first create an App in your Azure AD. From the Azure portal in the Azure Active Directory blade select App registrations and then New:

  • give a proper name to your application
  • choose the Web/API application type
  • type in the login url for your application

Once the application is saved you must edit the properties and:

  • change the reply url settings to https://%5Bmy url]/omnigollum/auth/azure_oauth2/callback
  • get the Application Id and a key you must specify in the Gollum configuration file

2017-07-22_17-11-03.gif

Once you have the app credentials, it’s time to create a c config.rb file, with the following settings (the config.rb file in the github repo has a complete example)

# Setup Omniauth via Omnigollum.
require 'omnigollum'
require 'omniauth-azure-oauth2'

options = {
  # OmniAuth::Builder block is passed as a proc
  :providers => Proc.new do
    provider :azure_oauth2, :client_id => 'YOURID', :client_secret => 'YOURKEY', :tenant_id => 'YOURNAME.onmicrosoft.com'

  end,
  :dummy_auth => false,
  # Make the entire wiki private
  :protected_routes => ['/*'],
  # Specify committer name as just the user name
  :author_format => Proc.new { |user| user.name },
  # Specify committer e-mail as just the user e-mail
  :author_email => Proc.new { |user| user.email }

  # Authorized users
  #  :authorized_users => ENV["OMNIGOLLUM_AUTHORIZED_USERS"].split(",")
}

Troubleshooting the image

It is often useful to troubleshoot the image while on the development VM. Azure Web App On Linux uses the /home mount point to have a common area for containers. So to properly debug the image you must map a folder on your VM to /home inside the container. The following are a few samples command lines to mimic what happens on Azure:

  • debug the python script, start the container and execute it docker run –name c3 –rm gollum:latest /usr/bin/python /usr/bin/init_container.py
  • mount a local directory (/mnt/home) to the container docker run –name c3 –rm -v /mnt/home:/home -e “WEBSITE_ROLE_INSTANCE_ID=test” gollum:latest /usr/bin/python /usr/bin/init_container.py
  • mount a local directoy and specify the customization storage account docker run –name c3 –rm -d -v /mnt/home:/home -e “WEBSITE_ROLE_INSTANCE_ID=test” -e “GOLLUMCONF=https://gollum.blob.core.windows.net/config” -e “GOLLUMCONF_KEY=YOURKEYHERE” -e “GOLLUMCUSTOM=wiki” gollum:latest /usr/bin/python /usr/bin/init_container.py
    also for logging, using

Reference and useful links

Other Gollum Docker implementations

Advertisements

, , ,

  1. #Azure and #MSOMS notable updates week 36/17 | Quae Nocent Docent

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: