Learn how to scale your Continuous Integration and Continuous Delivery environment using containers. The Kubernetes project provides a container orchestration solution that greatly simplifies app deployments in large clusters and you can use Jenkins and Kubernetes together to run jobs on-demand.
Building and testing is a great use case for containers, both due to the dynamic and isolation aspects, but it increases complexity when scaling to multiple nodes and clusters.
Jenkins is an example of an application that can take advantage of Kubernetes technology to run Continuous Integration and Continuous Delivery workloads. Jenkins and Kubernetes can be integrated to transparently use on demand containers to run build agents and jobs, and isolate job execution. It also supports CI/CD-as-code using Jenkins Pipelines and automated deployments to Kubernetes clusters. The presentation will allow a better understanding of how to use Jenkins on Kubernetes for container based, totally dynamic, large scale CI and CD.
2. ABOUT ME
Engineer @ CloudBees, Scaling Jenkins
Author of Jenkins Kubernetes plugin
Contributor to Jenkins and Maven official Docker images
Long time OSS contributor at Apache Maven, Eclipse,
Puppet,…
4. WHEN ONE MACHINE IS NO LONGER
ENOUGH
Running containers across multiple hosts
Multiple environments: public cloud, private cloud, VMs or
bare metal
HA and fault tolerance
5. How would you design your infrastructure if
you couldn't login? Ever.
Kelsey Hightower
8. KUBERNETES
Based on Google Borg
Run in local machine, virtual, cloud
Google provides Google Container Engine (GKE)
Other services run by stackpoint.io, CoreOS Tectonic,
Azure,...
Minikube for local testing
16. We can run both Jenkins masters and agents in Kubernetes
17. INFINITE SCALE!
Jenkins Kubernetes Plugin
Dynamic Jenkins agents, running as Pods
Multi-container support
One Jenkins agent image, others custom
Pipeline support for both agent Pod definition and
execution
Persistent workspace
18. ON DEMAND JENKINS AGENTS
podTemplate(label: 'mypod') {
node('mypod') {
sh 'Hello world!'
}
}
23. podTemplate(label: 'maven-selenium', containers: [
containerTemplate(name:'maven-firefox',image:'maven:3.3.9-jdk-8-alp
ttyEnabled: true, command: 'cat'),
containerTemplate(name:'maven-chrome',image:'maven:3.3.9-jdk-8-alpi
ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'selenium-hub', image: 'selenium/hub:3.4.0'
// because containers run in the same network space, we need to
// make sure there are no port conflicts
// we also need to adapt the selenium images because they were
// designed to work with the --link option
containerTemplate(name: 'selenium-chrome',
image: 'selenium/node-chrome:3.4.0', envVars: [
containerEnvVar(key: 'HUB_PORT_4444_TCP_ADDR', value: 'localhost'
containerEnvVar(key: 'HUB_PORT_4444_TCP_PORT', value: '4444'),
containerEnvVar(key: 'DISPLAY', value: ':99.0'),
containerEnvVar(key: 'SE_OPTS', value: '-port 5556'),
]),
containerTemplate(name: 'selenium-firefox',
image: 'selenium/node-firefox:3.4.0', envVars: [
containerEnvVar(key: 'HUB_PORT_4444_TCP_ADDR', value: 'localhost'
containerEnvVar(key: 'HUB_PORT_4444_TCP_PORT', value: '4444'),
containerEnvVar(key: 'DISPLAY', value: ':98.0'),
containerEnvVar(key: 'SE_OPTS', value: '-port 5557'),
])
28. MEMORY LIMITS
Scheduler needs to account for container memory
requirements and host available memory
Prevent containers for using more memory than allowed
Memory constraints translate to Docker --memory
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#how-
pods-with-resource-limits-are-run
29. WHAT DO YOU THINK HAPPENS WHEN?
Your container goes over memory quota?
31. NEW JVM SUPPORT FOR CONTAINERS
JDK 8u131+ and JDK 9
$ docker run -m 1GB openjdk:8u131 java
-XX:+UnlockExperimentalVMOptions
-XX:+UseCGroupMemoryLimitForHeap
-XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 228.00M
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM
Running a JVM in a Container Without Getting Killed
https://blog.csanchez.org/2017/05/31/running-a-jvm-in-a-container-without-getting-killed
32. NEW JVM SUPPORT FOR CONTAINERS
$ docker run -m 1GB openjdk:8u131 java
-XX:+UnlockExperimentalVMOptions
-XX:+UseCGroupMemoryLimitForHeap
-XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 910.50M
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM
Running a JVM in a Container Without Getting Killed
https://blog.csanchez.org/2017/05/31/running-a-jvm-in-a-container-without-getting-killed
33. CPU LIMITS
Scheduler needs to account for container CPU requirements
and host available CPUs
CPU requests translates into Docker --cpu-shares
CPU limits translates into Docker --cpu-quota
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#how-
pods-with-resource-limits-are-run
34. WHAT DO YOU THINK HAPPENS WHEN?
Your container tries to access more than one CPU
Your container goes over CPU limits