CI/CD Environment for A Smaller Project

Advantages of Continuous Integration (CI) and Continuous Delivery (CD) are obvious even for small projects with few contributors and are easily achievable with help of  free cloud tools – like for instance with mighty combo of Github plus Travis. But what if we want to achieve similarly convenient  environment inside of our private network, available only to our internal teams. Luckily open source is here again to help us with another great tool – GitLab –  GitLab  is a similar platform to GitHub, but the code is open source and we can easily install it in our environment. In this article I’ll summarize my experiences and guidelines how to build convenient environment for a small project  with automatic testing and deployment.

What we what to achieve

Our goals are:

  • Central code repository for our projects with friendly Web interface – similar to GitHub, but hosted by ourselves
  • Developers can use their existing credentials to log into this central repository
  • For projects we can setup CI, where for each push to central repository code is built and tested automatically
  • If build  and tests are successful web application  (we are talking about web applications) in deployed automatically for current branch in the  projects repository, so we can have numerous “feature” versions, where developers, QA and operations can test new features.
  • All this have to be rather simple and be able to run in couple of VMs – as our project is also rather small and we do not need complex architectures line Kubernetes for this project.

Setup

1st VM (Debian, Ubuntu, CentOS …)

  • Size it appropriately 2 cores, 4GB mem 16GB free disk space should be enough for smaller projects.
  • Install here Gitlab – Omnibus package
  • For authentication Gitlab can integrate with your corporate LDAP or through OmniAuth service  enable any popular SSO methods, including SAMLv2 (which I’m using)
  • Create project(s) and push code to them

2nd VM (Debian, Ubuntu, CentOS …)

  • Size it appropriately – as you’ll run there multiple containers you’ll need more resources – say 4 cores, 16GB memory, 80 GB fee disk space (but it depends very much on your projects).
  • Install here Gitlab Runner, Docker CE, eventually Docker Compose
  • Register two Gitlab executors (yes you can register several executors with one runner):
    • Docker executor – will be used for running builds and  tests –  be sure to give it  a tag –  docker
    • Shell executor – will run instances of our application in Docker container – again give it distinguishing tag – shell
  • If you want to share easily links to running application instances install nginx, uwsgi and this small application of mine
  • If you want to use local docker images for test tasks edit docker runner pull policy to pull_policy = "if-not-present"  in file /etc/gitlab-runner/config.toml and restart runner

How to use in a project

Here is the example how to use previous setup in a project ( my project is Python based so no special build tasks were needed, just to test and deploy).   In order to use CI/CD in Gitlab two preconditions are required:

  1. Gitlab must have available runners that can be used for your project (in our case we registered runners for all projects)
  2. .gitlab-ci.yml script in root of your project directory

Here is sample script for our python project:

image: my-base-image-for-testing
variables:
MY_PREFIX: my_app
test:
tags:
- docker
script:
- pip install -r requirements.txt
- pip install nose
- nosetests
deploy_in_docker:
stage: "deploy"
tags:
- shell
script:
- docker build -t ${MY_PREFIX}-${CI_BUILD_REF_NAME} --build-arg BRANCH_NAME=${CI_BUILD_REF_NAME} .
- docker rm -f ${MY_PREFIX}-${CI_BUILD_REF_NAME} || true
- docker run -d -P --name ${MY_PREFIX}-${CI_BUILD_REF_NAME} ${MY_PREFIX}-${CI_BUILD_REF_NAME}
image: my-base-image-for-testing variables: MY_PREFIX: my_app test: tags: - docker script: - pip install -r requirements.txt - pip install nose - nosetests deploy_in_docker: stage: "deploy" tags: - shell script: - docker build -t ${MY_PREFIX}-${CI_BUILD_REF_NAME} --build-arg BRANCH_NAME=${CI_BUILD_REF_NAME} . - docker rm -f ${MY_PREFIX}-${CI_BUILD_REF_NAME} || true - docker run -d -P --name ${MY_PREFIX}-${CI_BUILD_REF_NAME} ${MY_PREFIX}-${CI_BUILD_REF_NAME}
image: my-base-image-for-testing

variables:
  MY_PREFIX: my_app

test:
  tags:
  - docker
  script:
  - pip install -r requirements.txt
  - pip install nose
  - nosetests

deploy_in_docker:
  stage: "deploy"
  tags:
    - shell
  script:
    - docker build -t ${MY_PREFIX}-${CI_BUILD_REF_NAME} --build-arg BRANCH_NAME=${CI_BUILD_REF_NAME} .
    - docker rm -f ${MY_PREFIX}-${CI_BUILD_REF_NAME} || true
    - docker run -d -P --name ${MY_PREFIX}-${CI_BUILD_REF_NAME} ${MY_PREFIX}-${CI_BUILD_REF_NAME}

 

Now with this setup  each push to central repository will run test and deploy tasks and if they are successful we will have an instance of our application running in container and out team can quickly check new features in that branch.

Leave a Reply

Your email address will not be published. Required fields are marked *