Dec 28, 2018

Continuous Deployment

When I created first post I hat to generated site (using hugo) and I had to use ftp to upload files to programiraj.ba site.

Do I want to do this every time I make or change content ? No one sane would like to do that. And rule of thumb is when You have to do something more than three times automate it.

What I want is that every time i push changes to master to build site with hugo and deploy it to programiraj.ba

This is called continuous deployment (CD). If this repository was on bitbucket this could be done using bitbucket pipelines or if I had Jenkins instance installed on AWS. Let us now use CircleCI (and leave bitbucket pipelines for some other tasks in future).

Every CI/CD tool has some kind of pipeline which is usually described with some yaml, json file or by some scripting language that describes process of integration and deployment. Those include some jobs, commands, executors, steps, workflows etc…

Bottom line is that You have to know how to tell CI/CD system what You want to do.

Let’s to describe process in pseudo pipeline steps:

  1. When I push new content to github on master branch checkout new code

  2. Build site with hugo

  3. Publish new site on programiraj.ba (upload via ftp)

For step 1. we have to options pooling or notification (usually via web hooks). Pooling is process where CircleCi (or other CI/CD tool like Jenkins) checks are there any changes on github for every x minutes (or any time period You set), second option is when there is push on master github notifies CircleCi.

Step 2. and 3. are all done on CircleCI.

We could also add more steps to notify by email one or more persons that there is change on site but let’s focus now only on this 3 steps.

CirceCI works like this:

  1. Create a directory called .circleci in the root directory of your local GitHub or Bitbucket code repository.

  2. Add a config.yml file in the .circleci directory.

We must learn how to write config.yml to do steps 2. and 3.

For this site I use this script

version: 2.1
jobs:
  build-publish-site:
    docker:
    - image: abazovic/hugo:1.0.0
    steps:
    - checkout
    - run:
          name: "Pull Submodules"
          command: |
            git submodule init
            git submodule update --remote
    - run:
        name: "Run Hugo for blog"
        command: cd blog && HUGO_ENV=production hugo -v
    - run:
        name: "Run Hugo for homepage"
        command: cd home && HUGO_ENV=production hugo -v
    - run:
        name: "Deploy website"
        command: deploy/deploy.sh
workflows:
  version: 2
  workflow:
    jobs:
    - build-publish-site

You can find this script on https://github.com/MirzaAbazovic/programiraj.ba and also all code that I use to deploy https://programiraj.ba and https://blog.programiraj.ba

In plain English this script has one workflow that has one job build-publish-site. Job is running on docker image abazovic/hugo:1.0.0 and has steps to: 1. checkout code checkout

  1. Pull Submodules Because I use two themes that I added as git submodules.

  2. Run Hugo for blog build static site for blog.

  3. “Run Hugo for homepage build static site for home page.

  4. Deploy website Deploy it (upload files via lftp) using script deploy.sh

What was tricky:

  1. Make custom docker image with installed software: hugo and lftp (will describe this in some other post)

  2. Had some problems with git submodules, after checkout step I had to explicitly call git submodule init and git submodule update –remote

  3. I also had to write script to upload folder with sub folders generated by hugo to remote folder by ftp protocol. Script uses lftp

Script for upload

#!/bin/bash
echo "Upload all files from blog/public to blog.programiraj.ba using lftp"
lftp -e "set ftp:ssl-allow no; mirror -R -P 5  ~/project/blog/public /blog.programiraj.ba; quit" -u $FTP_USERNAME,$FTP_PASSWORD ftp://programiraj.ba
echo "Upload all files from home/public to programiraj.ba using lftp"
lftp -e "set ftp:ssl-allow no; mirror -R -P 5 ~/project/home/public /www; quit" -u $FTP_USERNAME,$FTP_PASSWORD ftp://programiraj.ba

Environment variables $FTP_USERNAME $FTP_PASSWORD are set project on CircleCI.

And that’s it every time I change or add new post I just push it on github and it gets publised on my site.

If You have some questions or suggestions I will gladly answer to You.

comments powered by Disqus