Gitlab and continuous integration

This guide is for setting up a CI environment using gitlab, jenkins and sonarqube.

Databases will be managed by a postgres container (official how to use an external database with gitlab).

• Copy docker-compose.yml and postgres_init_sonar.sql to the server (e.g. in /srv/ci/)
• chmod 644 postgres_init_sonar.sql
• Edit docker-compose.yml
  • Edit gitlab external_url: put a http url here (anyway normally your reverse proxy should redirect automatically to https), otherwise your gitlab may not work
  • Edit gitlab_shell_ssh_port (default value may be fine, actually)
  • Edit traefik labels and network name (or disable them if you don't use traefik)
  • Set up stronger passwords for databases
  • If you want to use docker inside jenkins, head to the "Launch docker commands in jenkins" section
  • docker-compose config to verify config.
• docker-compose up -d
• gitlab takes a long time to start, check its status with docker ps and wait for when its health is "healthy"

Initial gitlab configuration

• Connect to gitlab container through your browser
• Choose an admin password
• Connect as root using the password you chose
• Go to admin area(spanner icon), click on the cogwheel and choose settings to set up gitlab
• Create jenkins user:
  • Go to admin area and create a new jenkins user. You can choose whether to create a regular user (you will need to add it to every project in which you want a CI connection) or an admin user (will have access to every repository).
  • On the jenkins user page click on "Impersonation Tokens"
  • Create a new token with api access and copy it (you will need it for jenkins setup).

Create gitlab users

If you disabled signup, users need to be created by an admin. Gitlab doesn't allow you to set an initial password on the user creation page, because it wants to send an email with a reset link to the user. If the user doesn't have an email account, or you don't have a mail server, after you created the user you need to click on "edit" and you will be able to set an initial password.

Initial jenkins configuration

• On the server, execute docker-compose logs jenkins and find the initial admin password
• Connect to jenkins container through your browser
• Follow the wizard, choosing which plugins to install
  • Make sure that git and gitlab plugins are installed
• Change default security parameters (too lax)
  • Go to "Manage Jenkins -> Configure Global Security"
  • Disable "TCP port for JNLP agents" if not used
  • "Security Realm": do not allow sign up
  • We will later modify realm and authorization by installing roles.
• (optional) Install additional plugins by going in "Manage Jenkins -> Manage plugins"
  • Disk usage plugin (show the disk utilisation for every project)
  • Embeddable build status

Connect jenkins to gitlab

• Connect to jenkins as admin
• Add a new Gitlab API credential:
  • Go to "credentials"
  • Click on "Add credentials" (or find a way to add credentials)
    • Type: "Gitlab API token
    • API token: The impersonation token from gitlab copied earlier
    • ID and description: whatever you want
• Configure Gitlab plugin:
  • Go to "Manage Jenkins -> Configure System", scroll to "gitlab connections"
  • Add a new connection (or setup the default one):
    • Name: whatever you want
    • Host url: http://gitlab/
    • Credentials: The GitlabAPI credential you just created
    • Test connection
    • Save
• Add a new SSH key (for checking out repositories):
  • Connect to jenkins container with docker-compose exec jenkins bash on the server
  • Create ssh key: ssh-keygen, press enter until the key is generated.
  • cat /var/jenkins_home/.ssh/id_rsa.pub and copy it
  • On the web interface, select "Credentials ->Add credential -> SSH username with Private key", select "private key from the Jenkins master ~/.ssh", fill ID and description however you want (username doesn't actually matter, put git)
  • In gitlab, impersonate Jenkins user
  • Open settings, go to SSH keys
  • Add public SSH key you copied earlier

Enforce jenkins security with roles

By installing roles we will disable anonymous access and make a difference between admins (allowed to do anything) and users (allowed to manage jobs but not the global jenkins configuration).

• Go to "Manage Jenkins -> Manage plugins"
  • Install "Role-based Authorization Strategy"
• Go to "Manage Jenkins -> configure global Security" to configure access and permissions:
  • "Security Realm": jenkins' own database, do not allow sign up
  • "Autorization": Role-based strategy
  • Save
• Go to "Manage Jenkins -> Manage and assign roles" (refresh the page if you don't see it)
  • "Manage Roles"
    • role admin with every permission granted should already exist
    • Create user role, grant permission to Overall/Read and everything that is Job
    • Create anonymous role, grant permission to Job/ViewStatus
    • Save
  • "Assign roles"
    • Add user Anonymous to role anonymous
    • admin user should already be assigned to admin role
    • Save

Create a jenkins user

• Go to "Manage Jenkins -> Manage Users"
• Create user
• Set up roles (if you use roles authorization strategy):
  • Go to "Manage Jenkins -> Manage and assign roles -> Assign Roles"
  • Add the newly-created user to the user group
  • Save

Link a gitlab project to a jenkins job

You must have a gitlab project set up.

This needs to be done for every project, it can be done by any user that can configure the jenkins job.

• Connect to jenkins web interface.
• Create a new job of the type you want.
• Gitlab connection: should already be the connection you created when setting up jenkins
• "Source code management"
  • Select git
  • "Repository URL": git@gitlab:{path to your project e.g user/project-name}
  • "Credentials": use the ssh credentials that you created on jenkins installation.
• "Build triggers"
  • Select "Build when a change is pushed to GitLab" Copy the end of the URL (project/something).
  • Select triggers for which you want to listen
  • "Advanced -> Secret token -> Generate". Copy the token.
  • Eventually set up other advanced options
• "Build environment" and "Build steps": set up according to your project setup.
• "Post-build actions": select gitlab actions that you want (e.g. "Add note with build status on merge requests" and "Publish build status to commit")
• Save
• Go to gitlab the project page
• "Settings -> Integrations" to set up Jenkins webhook
  • URL: http://jenkins:8080/{rest of URL in jenkins job configuration(project/something)}
  • Secret token: token you generated in jenkins job configuration
  • Select triggers you want (they don't have to match jenkins triggers), you can select everything and it will be jenkins's role to filter only the ones you selected in the job configuration.
  • Add webhook
  • Test webhook, jenkins should launch a build. warning: in gitlab 9.5 this button may not work since gitlab changed its implementation and now it contains an error. You should try by manually pushing to the repo.

Initial Sonar configuration

• Connect to sonar container through your browser
• Log in as admin/admin
• Change admin password
  • Click on "Administrator" account name -> My account
  • "Security" tab
  • Set up a new password
• Go to "Administration"
• "General" settings:
  • "CI server": address of your jenkins server
  • "SCM server": address of your gitlab server
  • "Server base URL": address of your sonar server
• "Security" settings:
  • "Force user authentication": true
• Click on "Security -> Users" menu (not tab)
  • Click on "update tokens" button (or icon) next to "Administrator".
  • Generate a jenkins token, copy it
• Connect to jenkins web interface
• Install sonar plugin
  • "Manage Jenkins -> Manage Plugins"
  • Install "SonarQube Scanner for Jenkins"
• Configure sonar server
  • "Manage Jenkins -> configure System -> Sonarqube servers"
  • "Add SonarQube"
    • Name: whatever you want
    • URL: http://sonar:9000
    • Authentication token: the one you copied earlier
  • Save
• Configure sonar scanner
  • "Manage Jenkins -> Global Tool Configuration -> SonarQube Scanner"
  • Add scanner
    • Name: whatever you want
    • Keep "install automatically" and "Install from Maven Central"
  • Save

Add sonar to a jenkins job

You must have a jenkins job set up.

• Open job configuration
• "Build -> Add build step -> Execute SonarQube Scanner"
• Set up Analysis properties:

sonar.projectKey = {whatever, will be the project ID (seen in URLs, etc)}
sonar.projectName = {whatever}
sonar.projectVersion = {whatever}
sonar.sources = /var/jenkins_home/workspace/$JOB_NAME/{the source folder of your project structure}

Install sonar plugin

New plugins can allow to analyze more languages.

• Open sonar web interface and connect as admin
• "Administration -> System menu -> Update Center"
• Search plugins you want in the "Available" tab
• Install, server will be automatically restarted

Launch docker commands in jenkins

One you got you jenkins setup, you may want to be able to launch docker commands in jenkins jobs. We will do it by replacing the jenkins image with one containing the docker client, and giving access to the host's docker daemon to jenkins container.

Security warning: since jenkins will have access to docker, everyone with job configuration access can potentially mount your entire host filesystem inside a container and modify it. This can be a big security concern. But for a simple personal install it will be probably fine.

The image switch will not affect your data if you already have a docker install: since data should be in a jenkins_home volume, it is not affected by the image switch.

• Copy Dockerfile to the server, next to the docker-compose.yml
• Edit docker-compose.yml
  • Comment the jenkins image name
  • Uncomment the build commands and set arguments
    • Uncomment the new image name
    • docker_gid: the gid of the docker group on the host (in quotes). To find it run getent group docker | cut -d: -f3 on the host
    • docker_deb_url: go to https://download.docker.com/linux/debian/dists/, choose debian version running in the jenkins container (should be stretch, but you can verify by running cat /etc/*-release in the jenkins container), browse to /pool/stable/, choose the architecture of your container (run uname -a in the container if you are not sure), choose the version matching your host version (docker --version on the host). The address of the link is the value to set.
  • Uncomment the /var/run/docker.sock volume mount
• Stop the old jenkins container if it's running: docker-compose stop jenkins (it's better, but you could do it without stopping if you care about having the server always up: docker-compose up will do stop and restart the container automatically for you)
• Build the new image: docker-compose build jenkins
• Run the new container docker-compose up -d jenkins

Update the image

The image will not be automatically rebuilt. To update it launch docker-compose up -d --build jenkins.

Note on mounting volumes

If in your jenkins build you run a command that needs to mount a folder inside a container you may run into a problem. Suppose you want to mount the src folder of your project: jenkins will run something like docker run -v /var/jenkins_home/workspace/project_name/src/:/source_folder. but the docker command is executed on the host, which doesn't have a /var/jenkins_home directory!

To overcome that, instead of using a named jenkins_home volume mounted inside the jenkins container at /var/jenkins_home/, you need to create a /var/jenkins_home folder on the host and mount that instead.

• Copy the content of the current jenkins_home to the host (busybox cp command is too simple, we are forced to use debian): docker run --rm -v ci_jenkins_home:/src/:ro -v /var/jenkins_home/:/dst/ debian:jessie cp -R --no-target-directory --preserve /src/ /dst
• Modify docker-compose.yml
  • Instead of mounting jenkins_home in jenkins container, mount /var/jenkins_home
  • Delete/comment the declaration of jenkins_home volume
• docker-compose stop jenkins && docker-compose rm jenkins
• docker-compose up -d jenkins