Odagrun Intro


Preface of the Project


This project grew out of curiosity of the GitLab-api for CI, the many setups and maintenance of my infrastructure, that grew out of proportions.

On writing this, my private GitLab-CE instance has processed 10K pipelines with +50K jobs, all build on my dedicated build server.

I started with a shell runner, which quickly became apparent that that did not work for multiple setups, so next step was LXC instances each with a shell runner, for multiple architecture cross compiling of a c-code project. But this also became problematic, how to maintain, update libraries and more over, keep the GitLab-runners up to date.

I choose shell runners as my project could be build,tested and published in 40..60 seconds (3 stages for 6 different OS / Architecters), something that was absolute not possible on e.g. public GitLab-runners that would need worst cast 60 seconds to spin a VM, and then pulling an image, fresh as the docker daemon is freshly start up, and on top of that pull the git repository….., for each stage.

I did not see docker as an immediate solution, pulling and starting an image, more over that it has to run privileged since I wanted to build my specific build images to build my projects.

Finaly I stumpled on openshift, oc-cluster up and set the auth linked to the GitLab account. But now how to build my images, setting the cluster privileged of course, but I didn’t like it.

Since I was already working on deployctl for a token-less deployment of website, custom packages (RPM/Deb) and finaly to realese and publish software, I started experimenting with the OAPI to spin of containers on openshift, but it wasn’t till I found a way to build new custom images from a RPM repository (kinda like the Fedora debootstrap) that things started to fall in place.

The idea odagrun started the grow, especially with almost no real changes in the GitLab-runner performance since GitLab was or is pursuing the PaaS dream.

My main issue’s were speed, since I’m just to lazy to setup debug/build environments, I rely on development and debug on my Mac, push frequently and see if test still pass, git tag, and hop, a new release is published.

So the initial key requirments were:

  1. full non-root operation, so it can run on an okd cluster.
  2. easy image building, without draining to much resources.
  3. Low maintenance and easy setup.
  4. copy images from dev to production
  5. label support for tracability.

Meanwhile, more and more I get confirmations that building and/or communicating with a registry should not be done as root nor priviledged:

  • kaniko, although there still seems to be an issue for non-root user
  • skopeo, only for copying images and inspecting, but has OCI support!
  • …etc

Concept of odagrun


Concept of odagrun

dispatcher
  1. Use the internal registry as storage for WORK_SPACES and GIT_CACHE
  2. Create an executor image on the fly,
    • take the requested image,
    • add the git-cache layer,
    • add the WORK_SPACES layer(s)
    • and the executor layer
  3. transfer image layers to the ImageStream (local registry on Openshift)
  4. push new manifest
  5. create a pod config and/or wait untill suficiant resources!
  6. start the executor POD with the job to execute.
  7. follow-up and delete POD on Error or Success!
executer
  1. read job
  2. checkout Git if needed (cached)
  3. parallel download artifacts
  4. run script
  5. integrated commands use the dispatcher token for registry ops.
  6. feedback trace and status to Gitlab-CI

Disclaimer


The project odagrun, the Author and Gioxa Ltd. is in no way affiliated with GitLab nor with Openshift, okd

Credits


  • The project would not have been possible with the support of my Family.
  • The many opensource projects and c-libraries used in this project:
    • libcjson
    • libcurl
    • libarchive
    • libzip
    • libbzip2
    • libcares
    • libgit2.. and some more
  • And perhaps yourself:
    • submit a issue GitLab
    • mergerequest at GitLab
    • Share this page
    • you could buy me a coffee buy me a coffee for some extra inspiration,
    • donate paypal to keep the cogs turning.

Limitations of this version.


the following features have not been implemented yet:

  1. services
  2. cache, due to superior WORK_SPACES performance
  3. Failure reasons (time)
  4. async trace feedback (time)
  5. only one executer per registered runner for a runner pod, work around, scale up odagrun pods or register more runners.
  6. webterminal support.

Example projects using odagrun:


see projects under gioxa on GitLab

todo make a list with descriptions for public projects on GitLab.com

MIT License


MIT License

Copyright (c) 2018 Danny Goossen, Gioxa Ltd.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.