Setup Jenkins - Docker way
Jenkins for Build, Test and Deployment Automation
While we are adopting the Microservices architecture, it also demands to have an efficient CI/CD tools like jenkins. Along the cloud-native application developement and deployment jenkins can also be run cloud-native.
Since all processes, including software build, test and deployment, are performed every two or four weeks, this is an ideal playground for automation tools like Jenkins: After the developer commits a code change to the repository, Jenkins will detect this change and will trigger the build and test process. So Let's setup Jenkins as a docker container. Step-by-step.
- VM or EC2 Instance or a Standalone on-premisis machin
- Docker 1.12.1
- Jenkins 2.32.2
- Job DSL Plugin 1.58
- Ubuntu or an Liniux Machine
- Free RAM for the a VM/Machine >~ 4 GB.
- Docker Host is available.
- Tested with 3 vCPU (2 vCPU might work as well).
If you are using an host already has docker installed, you can skip this step. Make sure that your host has enough memory.
We will run Jenkins in a Docker container in order to allow for maximum interoperability. This way, we always can use the latest Jenkins version without the need to control the java version used.
Installing Docker on Windows and Mac can be a real challenge, but possible: here we will see an efficient way by using linux machine.
Prerequisites of this step:
- I recommend to have direct access to the Internet: via Firewall, but without HTTP proxy.
- Administration rights on you computer.
This extra download step is optional, since the Docker image will be downloaded automatically in step 3, if it is not already found on the system:
(dockerhost)$ sudo docker pull jenkins
Using default tag: latest
latest: Pulling from library/jenkins
Digest: sha256:8820149b54bfc5d05146b82150b5fdab583eef3e0499fb4ed630f77647a42942
Status: Image is up to date for jenkins:latest
The version of the downloaded Jenkins image can be checked with following command:
(dockerhost)$ sudo docker run -it --rm jenkins --version
2.19.3
We are using version 2.9.13 currently. If you want to make sure that you use the exact same version as I have used in this blog, you can use the imagename
jenkins:2.19.3 in all docker commands instead of jenkins only.Note: The content of the jenkins image can be reviewed on this link. There, we find that the image has an entrypoint/bin/tini -- /usr/local/bin/jenkins.sh, which we could override with the--entrypoint bashoption, if we wanted to start a bash shell in the jenkins image. However, in Step 3, we will keep the entrypoint for now.
In this step, we will run Jenkins interactively (with
-it switch instead of -d switch) to better see, what is happening. But first, we check that the port we will use is free:(dockerhost)$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0ec82b4ca2fd google/cadvisor:latest "/usr/bin/cadvisor -l" 2 days ago Up 2 days 0.0.0.0:8080->8080/tcp cadvisor
...
Since we see that one of the standard ports of Jenkins (8080, 50000) is already occupied and I do not want to confuse the readers of this blog post by mapping the port to another host port, I just stop the cadvisor container for this „hello world“:
(dockerhost)$ sudo docker stop cadvisor
cadvisor
Jenkins will be in need of a persistent storage. For that, we create a new folder on the Docker host:
(dockerhost)$ mkdir jenkins_home; cd jenkins_home
Note: The content of the jenkins image can be reviewed on this link. There, we find that the image has an entrypoint/bin/tini -- /usr/local/bin/jenkins.sh, which we could override with the--entrypoint bashoption, if we wanted to start a bash shell in the jenkins image.
We start the Jenkins container with the jenkins_home Docker host volume mapped to
/var/jenkins_home:(dockerhost)$ sudo docker run -it --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
Running from: /usr/share/jenkins/jenkins.war
webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Nov 30, 2016 6:12:14 PM Main deleteWinstoneTempContents
WARNING: Failed to delete the temporary Winstone file /tmp/winstone/jenkins.war
Nov 30, 2016 6:12:14 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Logging initialized @347ms
Nov 30, 2016 6:12:14 PM winstone.Logger logInternal
INFO: Beginning extraction from war file
Nov 30, 2016 6:12:14 PM org.eclipse.jetty.util.log.JavaUtilLog warn
WARNING: Empty contextPath
Nov 30, 2016 6:12:14 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: jetty-9.2.z-SNAPSHOT
Nov 30, 2016 6:12:16 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
Nov 30, 2016 6:12:17 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Started [email protected]{/,file:/var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
Nov 30, 2016 6:12:17 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Started [email protected]{HTTP/1.1}{0.0.0.0:8080}
Nov 30, 2016 6:12:17 PM org.eclipse.jetty.util.log.JavaUtilLog info
INFO: Started @3258ms
Nov 30, 2016 6:12:17 PM winstone.Logger logInternal
INFO: Winstone Servlet Engine v2.0 running: controlPort=disabled
Nov 30, 2016 6:12:17 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started initialization
Nov 30, 2016 6:12:17 PM jenkins.InitReactorRunner$1 onAttained
INFO: Listed all plugins
Nov 30, 2016 6:12:19 PM jenkins.InitReactorRunner$1 onAttained
INFO: Prepared all plugins
Nov 30, 2016 6:12:19 PM jenkins.InitReactorRunner$1 onAttained
INFO: Started all plugins
Nov 30, 2016 6:12:19 PM jenkins.InitReactorRunner$1 onAttained
INFO: Augmented all extensions
Nov 30, 2016 6:12:20 PM jenkins.InitReactorRunner$1 onAttained
INFO: Loaded all jobs
Nov 30, 2016 6:12:20 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Started Download metadata
Nov 30, 2016 6:12:20 PM hudson.model.AsyncPeriodicWork$1 run
INFO: Finished Download metadata. 97 ms
Nov 30, 2016 6:12:20 PM org.jenkinsci.main.modules.sshd.SSHD start
INFO: Started SSHD at port 44955
Nov 30, 2016 6:12:21 PM jenkins.util.groovy.GroovyHookScript execute
INFO: Executing /var/jenkins_home/init.groovy.d/tcp-slave-agent-port.groovy
Nov 30, 2016 6:12:22 PM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Nov 30, 2016 6:12:22 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.[email protected]453fc3cf: display name [Root WebApplicationContext]; startup date [Wed Nov 30 18:12:22 UTC 2016]; root of context hierarchy
Nov 30, 2016 6:12:22 PM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.[email protected]453fc3cf]: org.s[email protected]79a53f4b
Nov 30, 2016 6:12:22 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.s[email protected]79a53f4b: defining beans [authenticationManager]; root of factory hierarchy
Nov 30, 2016 6:12:22 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.[email protected]7ea44b7: display name [Root WebApplicationContext]; startup date [Wed Nov 30 18:12:22 UTC 2016]; root of context hierarchy
Nov 30, 2016 6:12:22 PM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.[email protected]7ea44b7]: org.s[email protected]12544046
Nov 30, 2016 6:12:22 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.s[email protected]12544046: defining beans [filter,legacy]; root of factory hierarchy
Nov 30, 2016 6:12:22 PM jenkins.install.SetupWizard init
INFO:
*************************************************************
*************************************************************
*************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
0c4a8413a47943ac935a4902e3b8167e
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
Nov 30, 2016 6:12:27 PM hudson.model.UpdateSite updateData
INFO: Obtained the latest update center data file for UpdateSource default
Nov 30, 2016 6:12:27 PM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
--> setting agent port for jnlp
--> setting agent port for jnlp... done
Now we want to connect to the Jenkins portal. For that, open a browser and open the URL
<your_jenkins_host>:8080
In our case, Jenkins is running in a container and we have mapped the container-port 8080 to the local port 8080 of the Docker host. On the Docker host, we can open the URL.
localhost:8080
The Jenkins login screen will open:
2016-11-30-19_36_42-jenkins-jenkins
The admin password can be retrieved from the startup log, we have seen above (
0c4a8413a47943ac935a4902e3b8167e), or we can find it by typing(dockerhost: .../jenkins_home)$ cat secrets/initialAdminPassword
0c4a8413a47943ac935a4902e3b8167e
on the mapped jenkins_home folder on the Docker host.
Let us install the suggested plugins:
This may take a while to finish:
Then we reach a page, where we can create an Admin user:
Let us do so and save and finish.
Note: After this step, I have deleted the Jenkins container and started a new container attached to the same Jenkins Home directory. After that, all configuration and plugins were still available and we can delete containers after usage without loosing relevant information.
I have had a dinner break at this point. Maybe this is the reason I got following message when clicking the „Start using Jenkins“ button?
What ever. After clicking „retry“, we reach the login page:
In the nex, we will create our first Jenkins job. I plan to trigger the Maven and/or Gradle build of a Java executable file upon detection of a code change.
-> 
-> 
-> 
-> 
(with dash between job and dsl; wait for the filter to become active and do not press enter, otherwise you will get an error message)
-> 
We create a Job DSL Job like follows:
-> 
-> 
-> 
-> 
-> if you have got a Github account, fork this open source Java Hello World software (originally created by of LableOrg) that will allow you to see, what happens with your Jenkins job, if you check in changed code. Moreover the hello world software allows you to perform JUnit 4 tests, run PowerMockito Mock services, run JUnit 4 Integration tests and calculate the code coverage using the tool Cobertura.
-> insert:
job('Job-DSL-Hello-World-Job') {
scm {
git('git://github.com/<org>/java-maven-junit-helloworld')
}
triggers {
scm('H/15 * * * *')
}
steps {
maven('-e clean test')
}
}
-> 
Goto Jenkins -> Manage Jenkins -> Global Tool Configuration (available for Jenkins >2.0)
Scroll down to Maven -> 
-> choose Version (3.3.9 in my case)
-> Add a name („Maven 3.3.9“ in my case)
-> 
Since we have checked „Install automatically“ above, I expect that it will be installed automatically on first usage.
As described in this StackOverflow Q&A, we need to add the Git username and email address, since Jenkins tries to tag and commit on the Git repo, which requires those configuration items to be set. For that, we perform:
-> 
->
-> 
-> scroll down to „Git plugin“
->
-> 
-> 
-> 
-> 
-> 
-> 
This is showing a build failure, since I had not performed Step 5 and 6 before. In your case, it should be showing a success (in blue). If you are experiencing problems here, check out the Appendices below.
-> 
-> scroll down to Source Code Management
-> Scroll down to Build Triggers
-> Scroll down to Build
-> verify that „Maven 3.3.9“ is chosen as defined in Step 5
-> enter „-e clean test“ as Maven Goal
->
See, what happens by clicking on:
-> Build History
-> #nnn
->
If everything went fine, we will see many downloads and a „BUILD SUCCESS“:
In a new installation of Jenkins, Git does not seem to work out of the box. You can see this by choosing the Jenkins project Job-DSL-Hello-World-Job on the dashboard, then click „build now“, if the build was not already automatically triggered. Then:
-> Build History
-> Last Build (link works only, if Jenkins is running on localhost:8080 and you have chosen the same job name)
->
There, we will see:
Caused by: hudson.plugins.git.GitException: Command "git tag -a -f -m Jenkins Build #1 jenkins-Job-DSL-Hello-World-Job-1" returned status code 128:
stdout:
stderr:
*** Please tell me who you are.
Run
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident name (for <[email protected](none)>) not allowed
As described in this StackOverflow Q&A: we can resolve this issue by either suppressing the git tagging, or (I think this is better) by adding your username and email address to git:
->
->
->
-> scroll down to „Git plugin“
->
Step 2: Re-run „Build Now“ on the Project
To test the new configuration, we go to
Now, we should see a BUILD SUCCESS like follows:
-> Build History
-> #nnn
If everything went fine, we will a „BUILD SUCCESS“:
When running a Maven Goal, the following error may appear on the Console log:
FATAL: command execution failed
java.io.IOException: Cannot run program "mvn" (in directory "/var/jenkins_home/workspace/Job-DSL-Hello-World-Job"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at hudson.Proc$LocalProc.(Proc.java:245)
at hudson.Proc$LocalProc.(Proc.java:214)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:846)
at hudson.Launcher$ProcStarter.start(Launcher.java:384)
at hudson.Launcher$ProcStarter.join(Launcher.java:395)
at hudson.tasks.Maven.perform(Maven.java:367)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1728)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 15 more
Build step 'Invoke top-level Maven targets' marked build as failure
Finished: FAILURE
Resolution:
Perform Step 5
and
For Test, you can test a manual: choose the correct Maven version, when configuring a Maven build step like in Step 7:
and verify that does not throw the Maven error anymore.
For our case, we need to correct the Job DSL like follows:
In the Script, we had defined the step:
steps {
maven('-e clean test')
}
However, we need to define the Maven Installation like follows:
steps {
maven {
mavenInstallation("Maven 3.3.9")
goals('-e clean test')
}
}
Here, the mavenInstallation needs to specify the exact same name, as the one we have chosen in Step 5 above.
After correction, we will receive the correct Maven goal
-> JobDSL ->
Now, we can check the Maven configuration:
->
->
->
After scrolling down, we will see the correct Maven Version:
DONE
Updating Jenkins (in my case: from 2.32.1 to 2.32.2) was as simple as following the steps below
Note: you might want to make a backup of your jenkins_home though. Just in case…
(dockerhost)$ cd <path_to_jenkins_home> # in my case: cd /user/jenkins_home/
(dockerhost)$ docker pull jenkins # to update the jenkins image
(dockerhost)$ docker rm jenkins # to make shure the container named jenkins is removed
(dockerhost:jenkins_home)$ sudo docker run -d --rm --name jenkins -p8080:8080 -p50000:50000 -v`pwd`:/var/jenkins_home jenkins
However, after that, some data was unreadable:
I have clicked
to resolve the issue (hopefully…). At least, after that, the warning was gone.
The reference for the Job DSL syntax can be found on the Job DSL Plugin API pages. As an example, the syntax of Maven within a Freestyle project can be found on this page found via the path
> freeStyleJob > steps > maven:
maven {
- // Allows direct manipulation of the generated XML.
- // Specifies the goals to execute including other command line options.
- // Skip injecting build variables as properties into the Maven process.
- // Set to use isolated local Maven repositories.
- // Specifies the Maven installation for executing this step.
- // Specifies the JVM options needed when launching Maven as an external process.
- // Adds properties for the Maven build.
- // Adds a property for the Maven build.
- // Specifies the managed global Maven settings to be used.
- // Specifies the managed Maven settings to be used.
- // Specifies the path to the root POM.
}
A Maven example can be found on the same page:
job('example') {
steps {
maven('verify')
maven('clean verify', 'module-a/pom.xml')
maven {
goals('clean')
goals('verify')
mavenOpts('-Xms256m')
mavenOpts('-Xmx512m')
localRepository(LocalRepositoryLocation.LOCAL_TO_WORKSPACE)
properties(skipTests: true)
mavenInstallation('Maven 3.1.1')
providedSettings('central-mirror')
}
}
}
In this blog post, we have learned how to
- 1.Start and initialize Jenkins via Docker
- 2.Prepare the usage of Git and Maven
- 3.Install the Job DSL Plugin
- 4.Define a Jenkins Job via Groovy script
- 5.Create a Jenkins Job by a push of the „Build now“ button
- 6.Review and run the automatically created Jenkins job
We have seen that the usage of the Job DSL is no rocket science. The only topic, we had to take care, is, that Git and Maven need to be prepared for first usage on a Jenkins server.
Last modified 4mo ago