Gitlab CI and Home Assistant

To make your Home Assistance more reliable, it is a better idea to do all the development on a separate machine than on your actual server. Having a backup of all configuration available online can also spare you from some headaches. That is why it is a good idea to start doing some Continuous Integration (CI) on your project: Do all development on a local machine and when you are satisfied with your changes, commit it to git and it will automatically deploy it to the server.
Using Gitlab, you can push your changes and it will automatically check it against faults and deploy it over SSH.
I based my installation on two posts:
https://about.gitlab.com/blog/2018/08/02/using-the-gitlab-ci-slash-cd-for-smart-home-configuration-management/
https://webworxshop.com/continuous-integration-deployment-for-home-assistant-with-gitlab-ci/

This article presumes you have a (private) repository on gitlab where all your yaml files are located. Also there is a server (with openSSH installed) and Home assistant running in Docker.

Starting a local runner in Docker

Testing the software happens in a runner. It is possible to use some public shared runners, but they are limited to 2000 min a month. So, it is possible to install a runner on your own server. Because Home Assistant is already running in Docker, it is easy to create a runner container in Docker also.

Step 1: Activate the runner

This step should only be done one time. It activates your server as a runner. Run the following docker command: docker run --rm -t -i -v $(pwd)/config:/etc/gitlab-runner gitlab/gitlab-runner register.
There will be 6 questions:

Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.com

Please enter the gitlab-ci token for this runner:
DdyF_LIkZZ_lMFV8A-mr

Please enter the gitlab-ci description for this runner:
test-ci

Please enter the gitlab-ci tags for this runner (comma separated):
hass

Please enter the executor: ssh, virtualbox, docker+machine, docker-ssh+machine, custom, parallels, shell, docker, docker-ssh, kubernetes:
docker

Please enter the default Docker image (e.g. ruby:2.6):
alpine:latest

  1. The first question is the link to the URL where the GIT repo is located. If you run Gitlab locally, fill in that address.
  2. The CI token can be found in the project. Click “Settings-> CI/CD” and expand runners. The token will be found under "Set up a specific Runner manually".
  3. The description of the runner is a name that will appear in that runners page once active.
  4. The tag is a name that will be used in the .gitlab-ci.yml. In this case, name it "hass".
  5. The executor is docker.
  6. The default image name is"alpine:latest".

This script should end with the message the runner is successfully registered.

Step 2: Create the runner itself.

The runner itself is a container that needs to run all the time. This container will be contacted by Gitlab when a commit is done. The docker-compose file is found here. Just start it and the job is done.

version: ‘3.3’
services:
gitlab-runner:
container_name: gitlab-runner
restart: always
volumes:
– ‘./config:/etc/gitlab-runner’
– ‘/var/run/docker.sock:/var/run/docker.sock’
image: ‘gitlab/gitlab-runner:latest’

Creating the .gitlab-ci.yml

First, start by creating a new file in your repository: .gitlab-ci.yml. This file will be red by Gitlab every time a commit is done. The content of this file should be as followed:

stages:
  - preflight
  - homeassistant
  - deploy

variables:
  GIT_SUBMODULE_STRATEGY: recursive

# Generic preflight template
.preflight: &preflight
  stage: preflight
  tags:
    - hass

# Generic Home Assistant template
.ha: &ha
  stage: homeassistant
  variables:
    PYTHONPATH: "/usr/src/app:$PYTHONPATH"
  before_script:
    - python -m homeassistant --version
    - mv fake_secrets.yaml secrets.yaml
  script:
    - |
      echo $CI_PROJECT_DIR && python -m homeassistant \
        --config $CI_PROJECT_DIR \
        --script check_config \
        --info all
  tags:
    - hass

# Preflight jobs
shellcheck:
  <<: *preflight
  image:
    name: koalaman/shellcheck-alpine:stable
    entrypoint: [""]
  before_script:
    - ln -s $CI_PROJECT_DIR /config
    - shellcheck --version
    - apk --no-cache add grep
  script:
    - |
      for file in $(grep -IRl "#\!\(/usr/bin/env \|/bin/\)" --exclude-dir ".git" .); do
        if ! shellcheck -x $file; then
          export FAILED=1
        else
          echo "$file OK"
        fi
      done
      if [ "${FAILED}" = "1" ]; then
        exit 1
      fi

yamllint:
  <<: *preflight
  image: sdesbure/yamllint
  before_script:
    - yamllint --version
    - mv fake_secrets.yaml secrets.yaml
    - rm known_devices.yaml
  script:
    - yamllint .
  allow_failure: true

jsonlint:
  <<: *preflight
  image: sahsu/docker-jsonlint
  before_script:
    - jsonlint --version || true
  script:
    - |
      for file in $(find . -type f -name "*.json" -path "mysensors.json" -prune); do
        if ! jsonlint -q $file; then
          export FAILED=1
        else
          echo "$file OK"
        fi
      done
      if [ "${FAILED}" = "1" ]; then
        exit 1
      fi

markdownlint:
  <<: *preflight
  image:
    name: ruby:alpine
    entrypoint: [""]
  before_script:
    - apk --no-cache add git
    - gem install mdl
    - mdl --version
  script:
    - mdl --style all --warnings --git-recurse .

# Home Assistant test jobs
ha-latest:
  <<: *ha
  image:
    name: homeassistant/home-assistant:latest
    entrypoint: [""]

ha-rc:
  <<: *ha
  image:
    name: homeassistant/home-assistant:rc
    entrypoint: [""]
  allow_failure: true

ha-dev:
  <<: *ha
  image:
    name: homeassistant/home-assistant:dev
    entrypoint: [""]
  allow_failure: true

deploy:
  stage: deploy
  image:
    name: alpine:latest
    entrypoint: [""]
  environment:
    name: home-assistant
  before_script:
    - apk --no-cache add openssh-client
    - echo "$DEPLOYMENT_SSH_KEY" > id_rsa
    - chmod 600 id_rsa
  script:
    - ssh -i id_rsa -o "StrictHostKeyChecking=no" $DEPLOYMENT_SSH_LOGIN "cd /home/sibrecht/homeassistant && git pull && git fetch --all && git reset --hard origin/master && docker restart homeassistant"
  after_script:
    - rm id_rsa
  only:
    refs:
      - master
  tags:
    - hass

There is not much that needs to be changed in the file. Only the path where the configuration files are located on the server needs to be changed. (I assume they are not located at /home/sibrecht/homeassistant)

What does the file do:
– check for syntax and coding style guides. (shellcheck, yamllint, markdownlint, jsonlint). These are just checks if the files are available and if they are formatted correctly.
– check if Home Assistant would startup with the current configuration files. This is done for the latest stable version (latest), the latest release candidate (rc) and the version they are currently working on (dev).
– Deploy the current branch to the release server over SSH.

In every step, you can see the tag is "hass". This is the link to your runner that has the same tag.

This file needs to be committed and the pipeline will start running. The first times, it would be a miracle it passes. Some files will not have the right layout, but after some tweaking and changing, it will pass. The deploying to the server will be set up next.

Deploy to the server

Getting SSH keys

Navigate to a new folder called "Test". It is not important where it is, because it can be deleted afterwards.
Run: ssh-keygen -t rsa -C "hass-deploy" -b 4096. The script will ask where to save the ssh key pairs. Save it in this folder with the name "test" by entering ./test. Two files will be created: test and test.pub.
Copy the content of the public key "test.pub" into ~/.ssh/id_rsa.pub. Create the file if it does not exist.

Next two variables need to be created in Gitlab: DEPLOYMENT_SSH_KEY and DEPLOYMENT_SSH_LOGIN. Those two variables are needed in the .gitlab-cy.yml file. Creating these variables is done in the project “Settings -> CI/CD” when expanding variables.
In the DEPLOYMENT_SSH_KEY, paste the content of test, generated from the server. Add also the headers "-----BEGIN OPENSSH PRIVATE KEY---" and "---END OPENSSH PRIVATE KEY---".
In the DEPLOYMENT_SSH_LOGIN, enter your login for ssh. In my case it was "sibrecht@192.168.1.100". It is possible to use a local address because the ssh login happens from the runner that is running in the same network.

I had some issues when logging into the ssh server. I constantly had the error message: "permission denied (publickey,password)". I had to generate the keys again and then it worked.

Start the pipeline again, and you should see the commit is pulled and the container is restarted. Enjoy.

Leave a Reply

Your email address will not be published. Required fields are marked *