Does Jenkins Scale? Nope! Here's 4 Reasons Why.
by Iris Chubb, on Jul 5, 2021 7:45:00 PM
Lessons learned from Jenkins are often learned too late. A lesson you don’t want to learn too late is that: Jenkins doesn’t scale.
...at least, not easily. A quick search online will provide plenty of results saying that you can use Jenkins at "mega scale." And sure, with the right amount of plugins, Kubernetes, and sorts of other customizations, you can get it to work. However, doing so would come at a monumental cost in manhours and almost never actually be worth it.
If you’re new to CI/CD software or Jenkins, it’s not immediately obvious that Jenkins isn’t meant for scaling. But after a few months of use and a few business-critical projects that are dependent on Jenkins, you’ll quickly discover that it wasn’t built for scaling.
Teams looking to scale with Jenkins all encounter the same issues sooner or later:
- Painfully slow servers (No high-availability or load balancing)
- Wasting time doing manual processes on an automation tool
- Things going wrong and no backups are available
- The "Jenkins Expert" becomes a bottleneck
And these are just the tip of the iceberg. Mature organizations with responsibility to customers and stakeholders often can’t afford the time and mistakes that come with discovering that Jenkins isn’t a long-term solution.
Consider this your heads up – Jenkins doesn’t scale.
4 Problems With Jenkins and Why it Doesn’t Scale Well
1. Not all plugins are supported by Jenkinsfile
Jenkins is hard to back up without heavy manual intervention
Because plugins are created by the Jenkins community, they won't always have what you may consider "basic features," and not all plugins are supported by Jenkinsfile (which is exported Jenkins configuration). To back up Jenkins, you have to manually write down a list of plugins and manually install them, and then configure those plugins manually. Compare this to other tools that allow scheduled, automated backups.
This means your team will be wasting lots of time doing manual process on an automation tool.
2. Different departments need different instances
Doing "Jenkins as Code" becomes much harder than similar CI/CD tools
In a similar way, doing "Jenkins as Code" requires extra, manual effort at scale, because different departments will spawn different, and often multiple, Jenkins instances. Because of the difficulty of back-ups, “Jenkins as Code” means backing up multiple instances in a difficult way every time.
Simply put, it costs a large investment of time to stay organized when doing “Jenkins as Code.”
3. Jenkins "database" is just XML files on a disk
XML format makes it relatively expensive to read and nearly impossible to index
Another thing that slows you down is Jenkins's database, which is basically just XML. This was originally designed with interoperability in mind, not performance. As a result the verbose and dynamic nature of the xml format makes it relatively expensive to read and more so to index.
This means that you need a Jenkins expert on hand at all times when scaling. This expertise costs money and a significant portion of their time that could be spent on other projects.
4. Jenkins can only have a single controller
Jenkins controller can't run in high-availability mode
And perhaps most importantly: scaled organizations rely on constant up-time. This requires high-availability and load balancing, which Jenkins cannot do with just a single controller. You can "duct tape" solutions together as a standby failover, but (as with any duct-taped solution) these often have performance issues.
Mature, scaled businesses can't afford risks of downtime and lost work time due to Jenkins outages.
How to Scale?
Use a tool that scales
For a full rundown of how BuildMaster integrates with Jenkins you can read my full article here or refer to our official documentation. Or, download BuildMaster's free forever version and start scaling right away.