Avoid the Disaster of Self-Service Jenkins
by Iris Chubb, on Jun 2, 2021 3:44:00 AM
Jenkins is a powerful and versatile CI tool that helps automate the parts of software development related to building, testing, and deploying, facilitating continuous integration and continuous delivery. Self-service is at the center of Jenkins. Any team member is able to improve and modify Jenkins as they see fit with little to no oversight or permissions needed. This self-service is achieved through Jenkins impressive collection of over 1800+ plugins
However, empowering your team to get work done the way they want while balancing compliance, visibility, and stability comes with risk that can quickly turn your Jenkins instance into a disaster.
The unfortunate reality is the self-service Jenkins supports is almost always it’s demise. If left unmonitored, your Jenkins instance will become more complex, more difficult to keep functioning, and much more unstable.
The Gap Between "Self-Service" and "Required Expertise"
Teams that use Jenkins as it was originally intended (as a project-specific, expert-managed CI server) don't experience these problems at first. The Jenkins expert(s) sets up Jenkins quickly and provide basic self-service options for everyone.
However, things quickly turn chaotic since Jenkins encourages users to expand it’s capabilities through their large selection of plugins. And that's where the problem starts. Most teams end up with Jenkins configurations much more complex than the average dev can handle.
This turns your Jenkins experts into a bottleneck.
To bypass the bottleneck, other team members will try to maintain the server themselves or will just create a separate Jenkins install someplace else…and this process repeats itself over and over… This creates complications and chaos. Jenkins is constantly being added to and Jenkins projects are distributed in a non-systematic way.
All of this ends up being very expensive. Your Jenkins people have created more of a blocker than Jenkins itself. Sounds a lot like pre-DevOps, manual processes, doesn't it?
|Jenkins instances are complex and often do more than what they were intended to do: Basic CI||You need a Jenkins expert for even the most basic maintenance||Experts become bottlenecks, slowing new projects, builds, releases, etc.|
|Tightly-secured instances create expertise bottlenecks||Self-service seekers will create their own instances or find other alternatives||Builds/projects are not systematically distributed and are chaotic, increasing risk and slowing things down|
|Loosely-secured instances become chaotic and unstable over time||Non-expert users modify configuration, install/update plugins for their project||Unrelated builds/projects stop working, causing lost productivity and frustration across teams|
Self-service can be too much of a good thing
Jenkins is a by-design "democratic," self-empowering tool, allowing anyone to create projects, configure agents, install plugins, etc. But like anything, too much of any good thing can go bad. Too much self-service can cause problems: speed, vulnerability, and longevity.
Because anyone can install any plugin, odds are that you won’t know who installed a plugin, why they installed it, how often it's being used, or whether it can be removed without breaking anything; Jenkins doesn't indicate any of this. In fact, some plugins get automatically added because another plugin requires that plugin.
If a less-thorough colleague doesn't seek answers to ALL these questions from the people who installed them and modifies or deletes a plugin, your stuff can break. And if all these plugins are on your controller, you end up with a sluggish Jenkins that may result in more failed builds.
Because anyone can install any plugin—which are not of equal quality—it's too easy to introduce vulnerabilities or even malware into your Jenkins installations. To avoid this, perhaps you'll require your developers to get approval and advice on installing plugins from Jenkins expert... which makes the Jenkins expert a bottleneck again.
And when your information relies on human memory and manual intervention for updates, your Jenkins pipelines become "legacy" much faster than competing tools' pipelines.
|Having a ton of plugins and complications in your Jenkins (especially in the controller/master)||The Jenkins controller slows way down AND becomes difficult to navigate for non-experts||Bottlenecks form, defeating the purpose of faster-paced DevOps tools|
|Anyone can install any plugin, which don't require automatic testing||Vulnerabilities can be easily introduced||Increased risks of malware, downtime, and other nasties|
|Lack of visibility means people hold Jenkins knowledge, rather than Jenkins itself||Information is behind human "firewalls"||Slows self-service (and thus work) down, since you're dependent on people rather than just the tool|
Chances are if you’re reading this article you’re already familiar with the problems of self-service in Jenkins and are looking for solutions. But I want to stress: Don’t dump Jenkins! Replacing it entirely would cost too much, and your team already knows Jenkins.
Instead, combine it with a dedicated tool like BuildMaster that will fix your Jenkins problems while giving you better CI/CD overall. Using a tool that integrates and works well with Jenkins is going to be your best bet of breaking out of Jenkins chaos.
Here’s a quick review of how self-service through BuildMaster is done right.
Self-Service Done Right
1. Restriction: Environment Targeting
Management can restrict deployment to target a pre-specified environment. This control can be applied in a few ways:
- Permissions: users must have permission to deploy to that environment in order to deploy to that stage
- Role Selection Filtering: if you also target by role, then the server must be in that environment to be included
- Runtime Server Protection: a server must be in the target environment, or a runtime error will occur
- Visualization: this helps you see which builds have been deployed to which environments
2. Restriction: Server Targeting
Server targeting restriction can be applied in multiple ways:
- No default server context; this means you will need to explicitly specify which servers to deploy to in the deployment plan itself
- All servers in target environment; this will run the deployment plan against all servers in the targeted environment
- Specific servers; the plan will be run against the list of servers specified
- Specific server roles; the plan will be run against servers with the specified roles and that are associated with the targeted environment
- Server Pool; the plan will run against one server from a specified role that will act as a pool. This will effectively put an Acquire-Server operation at the beginning of the plan.
3. Restriction: Release/Job Templates
- Management can define key/value pairs on pipelines and/or stages. These behave just like configuration variables, in that you can use these variables within deployment plans that are executed through the plan.
- However, unlike configuration variables, pipeline variables, cannot be multi-scoped, and cannot be modified through the variables API.
4. Restriction: Manual Pipeline Stage Approvals
- To prevent a build with any failing unit tests from being deployed to a pipeline stage, management can add a "Unit Test" automated approval to the target stage. This approval will validate that tests recorded in the latest execution to the previous pipeline stage either all passed or none failed, where inconclusive tests are dependent upon the approval's configuration.
Jenkins is a self-service-oriented tool that your team should continue to use. However, it's so important to introduce restrictions that balance self-service with stability, visibility, and compliance. The easiest way to do this is to use a tool like BuildMaster.
All BuildMaster features are designed as much for the casual user as for the power user. BuildMaster was designed to integrate well with nearly every other DevOps tool you might be using. It’s mature enough for experts but accessible enough to enable self-service for non-experts.
For a full rundown of integrating BuildMaster and Jenkins, check out our Docs page or download our Free Forever version and try it out for yourself.