QA Testing Backports

QA Testing Backports

Reasoning

Currently we do not have a set of servers/environments that can easily install older versions of GitLab that extend beyond our current patching policy. This means backports have no place to be installed. Therefore, we rely on QA pipelines to determine if the proposed package is safe to be released. Note that we are not testing the results of a built package with this procedure, but instead testing builds that occur from our pipelines.

Procedure

Build the QA Image and Execute

  1. Prior to merging the preparation branch.
  2. Depending on the changes picked in the preparation branch, the package-and-qa job can be manual or start automatically.
    • If it’s manual, you can just start it. It will trigger a downstream pipeline on the matching stable branch of omnibus-gitlab for the proposed release.
    • See Extra Information if package-and-qa is unavailable.
    • This job takes a few hours.
  3. Wait till, minimally the job Trigger:gitlab-docker has completed prior to moving on.
  4. Notify Quality of this pipeline such that they can monitor for any issues.
  5. This pipeline will take a few hours.
  6. If all is well, QA should sign off on the release issue associated with the backport indicating that they approve that Release Managers can proceed to release the build.

Extra Information

  • If the preparation MR is merged prior to running this pipeline, we can still perform the above process. If new changes need to be picked, they should be done in a new branch and QA run prior to merge to avoid repeating this lengthy process multiple times.
  • If a new branch needs to be created for any reason, avoid the use of the character / in the branch name as not all scripts properly manage this and will fail to create a Docker image with a usable tag.
  • The variable RELEASE can be confirmed by navigating to the triggered pipeline and looking for the job Trigger:gitlab-docker. Browse the logs and then view the raw logs (the data we want is in stderr). Scroll to the bottom and we’ll see a message from docker that indicates it pushed an image. The tag should match the sha of the commit of our branch in the gitlab-org/gitlab project.
  • Update tests should use the published docker image with the backport version as the previous to update from. If this is not found, then the update tests fall back to the latest, which may cause the test to fail.

Reference