Lessons learned from Jenkins are often learned too late. A lesson you don’t want to learn too late is that: Jenkins doesn’t scale.

…at least, not easily. A quick search online will provide plenty of results saying that you can use Jenkins at a “mega scale.” And sure, with the right amount of plugins, Kubernetes, and sorts of other customizations, you can get it to work. However, doing so would come at a monumental cost in manhours and almost never actually be worth it. 

If you’re new to CI/CD software or Jenkins, it’s not immediately obvious that Jenkins isn’t meant for scaling. But after a few months of use and a few business-critical projects that are dependent on Jenkins, you’ll quickly discover that it wasn’t built for scaling.

Teams looking to scale with Jenkins all encounter the same issues sooner or later:

• Painfully slow servers (No high availability or load balancing)
• Wasting time doing manual processes on an automation tool
• Things going wrong and no backups are available 
• The “Jenkins Expert” becomes a bottleneck

And these are just the tip of the iceberg. Mature organizations with a responsibility to customers and stakeholders often can’t afford the time and mistakes that come with discovering that Jenkins isn’t a long-term solution. 

Consider this your heads up – Jenkins doesn’t scale. 

4 Problems With Jenkins and Why it Doesn’t Scale Well

1. Not all plugins are supported by Jenkinsfile

Jenkins is hard to back up without heavy manual intervention

Because plugins are created by the Jenkins community, they won’t always have what you may consider “basic features,” and not all plugins are supported by Jenkinsfile (which is exported by Jenkins configuration). To back up Jenkins, you have to manually write down a list of plugins and manually install them, and then configure those plugins manually. Compare this to other tools that allow scheduled, automated backups.

This means your team will be wasting lots of time doing the manual process on an automation tool.

2. Different departments need different instances

Doing “Jenkins as Code” becomes much harder than similar CI/CD tools

In a similar way, doing “Jenkins as Code” requires extra, manual effort at scale, because different departments will spawn different, and often multiple, Jenkins instances. Because of the difficulty of back-ups, “Jenkins as Code” means backing up multiple instances in a difficult way every time.

Simply put, it costs a large investment of time to stay organized when doing “Jenkins as Code.”

3. Jenkins “database” is just XML files on a disk

XML format makes it relatively expensive to read and nearly impossible to index

Another thing that slows you down is Jenkins’s database, which is basically just XML. This was originally designed with interoperability in mind, not performance. As a result, the verbose and dynamic nature of the XML format makes it relatively expensive to read and more so to index.

This means that you need a Jenkins expert on hand at all times when scaling. This expertise costs money and a significant portion of their time that could be spent on other projects.

4. Jenkins can only have a single controller

Jenkins controller can’t run in high-availability mode

And perhaps most importantly: scaled organizations rely on constant uptime. This requires high availability and load balancing, which Jenkins cannot do with just a single controller. You can “duct tape” solutions together as a standby failover, but (as with any duct-taped solution) these often have performance issues.

Mature, scaled businesses can’t afford the risks of downtime and lost work time due to Jenkins outages.

How to Scale?

Use a tool that scales

There’s a long list of criteria for a scalable CI/CD tool. My personal favorite is BuildMaster. BuildMaster is free,  easy to start using, and even integrates with Jenkins.

For a full rundown of how BuildMaster integrates with Jenkins, you can read my full article here or refer to our official documentation. Or, download BuildMaster’s free forever version and start scaling right away.