Combing through hard drives, emails, USBs and file folders looking for a script is annoying, it’s wasteful, but most importantly, it’s not necessary.

It’s common for a company’s scripts to be spread across different platforms, changed without documentation, and not in source control. It shouldn’t come as a surprise that this isn’t the best practice. Why do so many organizations do this with their scripts? Usually, because it’s the default. But there is a better way.

Leveraging version and source control for your scripts enables an automated Software Development Life Cycle (SDLC), which (as I’ve said before for Python & Powershell) is essential to your team’s success.

This article explains source control (and how it’s different than version control), when/how to use it (hint, it just takes a bit of organizing), and how it actually helps your team. 

No Control vs. Version Control vs. Source Control

Phase 1: No Control

This is the default and is unfortunately where many teams are. This is where everyone starts before they learn a better way.

“No control” in practice? Picture files scattered across different hard drives, emails, USB sticks, etc. It’s also copies of scripts set to the side just in case something goes wrong with the changes you’re about to apply. While that’s theoretically a good idea, without any type of control, this leads to completely unhelpful, arbitrary file names that are eventually confusing even to the original author: MySCript_final, myScript_final2, myscript_reallyfinal(1), etc.

The benefit to this method is there is no training or time cost. Learning new things takes time. Version/Source Control is no exception, so a lot of teams, thinking the expense of time isn’t worth it, will just keep playing Russian Roulette with the ‘no control’ method.

A major drawback of this method is that changes made to scripts can’t be undone and are difficult to fix. Generally, if you have no version or source control, scripts aren’t centrally located. This means that sharing scripts relies on your team being perfect communicators, which humans are notoriously NOT. Another major con is the lack of changelogs, which means no accountability or bread crumb trail to find out why something was changed.

Phase 2: Version Control

Version control is used to roll back changes if things have gone wrong and audits both when and who made changes. This is done by using a tool like SharePoint, OneDrive, or DropBox that stores previous versions of files. Version control is automatic and happens every time the file is saved.

The main issue with adopting only version control is that the tools usually used for it aren’t designed for scripts. It’s easy to roll back changes but, rollbacks are all-or-nothing (unlike with source control). Another drawback is that multiple people aren’t able to work on a script at the same time without risking overwriting each other. 

Imagine you spend four hours making changes to a script but while you’re working the scope changes slightly without anyone telling you. Since version control is automatic and all-or-nothing, if you’re only using version control, the rollback options are either:

1. Rollback all of the work, throwing away those four hours of work.
2. Spend another couple of hours combing through the code and editing the lines that need to be changed

Both options suck. But it is better than having no control in place, which is why I still say everyone should at least have version control. Using Version Control means you don’t have to rewrite deleted or modified work. Another major benefit (that I see managers excited about) is the ability to see and audit who made changes. However, these audits only show who and when a script was changed. It’s not easy to compare 2 versions of a file unless you do it side-by-side on your screen making approval processes time-consuming. 

Version control makes it easy to audit and roll back to a previous version when undesired changes or mistakes are made.

Phase 3: Source Control

Source control is the gold standard of organization because it’s a way to manage changes to a script multiple people are working on.

A very common Source Control workflow is Gitflow. The basic steps are:

1. Create a “branch,” which is an exact copy of the main version, but which is segregated from the main code
2. Make edits in the branch
3. Commit changes, which kicks off several approval stages

If there are no conflicts, the in-branch changes merge into the main

The most common form of Gitflow source control is through Git-based source code repositories like GitLab, Bit Bucket, etc.

There are lots of benefits to source control:

• You can work on the same thing at the same time as your teammates without messing up the main branch of code, overwriting each other’s work, or creating conflicting changes.
• Syncing to your repository shows changes made by other team members that can then be introduced to the code you’re working on.
• It’s easy to pick and choose SPECIFIC changes you want to accept or revert.
• Team members can add important contextual information when committing code.

Source control essentially removes a large blocker that your team is currently facing.

Source control also provides an extra step of quality-control and security for code. Merging changes are done by request only and must be approved before they’re applied to the source code. Someone looking at your GitHub that thinks “Oh this is junk we don’t need this anymore” can’t just bin it, and the maverick on your team who never tests his stinkin’ code can’t just merge it in and crash everything.

I’m repeating myself, but the biggest and only drawback of Source Control is the time and money that it takes to learn and adopt it. But it’s SO worth it.

Which to Choose

If you simply want a backup and the option to roll back, version control might be the smartest way to go since the cost of adoption is low.

If you’re currently on ‘no control,’ a good place to start for version control is a service like SharePoint that tracks what changes are made and also lets you know when someone else is working on something. These services also offer a checkout system to lock a file while someone is working on it. They also typically allow you to create folders and subfolders that can be searched.

A great example of a tool with both versions to start and optional source control is Otter. The scripts you upload in Otter are automatically versioned, and you can easily compare previous versions to see what, who, and when changes were made.

But you can transition into source control very easily. The built-in “raft” can be replaced with a Git raft to edit scripts both in Otter (edits are automatically committed to your Git repo) and Git directly (Otter will read from Git). For more advanced users, you can even use branches with different rafts! Oh by the way, all of these features are available with the free version of Otter!

Remember that source control is the gold standard for a reason. For anything more than the basics, source control is almost certainly worth the adoption cost and should be a goal to work towards on your team.

Don’t JUST Use Source Control

Without organization, you’re not really using version/source control, you’re just wasting time. Version/Source control are only as great as they are organized.

Before jumping in eyes-closed, head-first, create an organizational method that makes sense for your team. For example, we have hosted our documentation on GitHub. These docs have mainly been divided by product then subdivided further based on tasks or technology that the product interacts with. Then we follow the logic described before: When a page needs to be updated, content creators request to edit the doc page. After editing the page, content creators create a pull request and one of the engineers then makes sure the changes are OK and technically accurate before committing changes to the docs page.

But here’s the problem: you’re not organizing docs. You’re organizing scripts. So, what’s your first step? Start by identifying the problem that your team is trying to solve. Some examples might be:

• Distribution of common repositories.
• Record changes and have the option to roll back.
• Simple back-up.
• Control what changes are made to the source code.

How to Organize

The many ways to organize your version or source control can be mixed and matched together based on the organizational method that makes the most sense for your team. 

Option 1: Organize by Project

Breaking down projects into categories like new products, maintenance, etc.

Why it’s good: This is a great system for team cooperation and streamlining workflow. Whoever is working on a project can have all the scripts they need, guaranteed.

What to do: Create multiple folders for each module or set of scripts under the same project. Splitting these into their own folders means they can be built and versioned separately from one another.

Possible issues: The problem with project organization is it’s difficult to organize at a very granular level. Should this go in project A or project B? This is a multi-purpose module, should it go in subfolder X or Z? And since projects are typically temporary, when your team archives a project, they will effectively be archiving these scripts with it.

Option 2: Organize by Department & Team

Breaking down folders by department name (Sales, Accounting, Marketing, etc.)

Why it’s good: Projects can cross teams, but organizational departments and teams are vertical by nature. This means you can make sure certain extremely sensitive or important scripts would be inaccessible to any below a certain part of the organizational chart (e.g., maybe only managers in the Finance department can get into X folder).

What to do: Create folders first by department and then by role in that department. Then use credentials (or even Active Directory if possible) to block certain people’s access to different folders.

Possible issues: The main disadvantage of organizing by department/team is that you are creating information silos. Not all the scripts in “Finance” can/will ever be accessed by other members of your team. They will never be able to use/improve/learn from those scripts.

Option 3: Organize by Technology

Create folders based on the technology the scripts interact with.

A simple example of this would be separating your Azure scripts from your local mail service scripts.

Why it’s good: This form of organization really helps level up the knowledge of more junior team members and if you’ve read any of our PowerShell articles or Python articles you know that’s a good thing.

What to do: Create folders based on the various technologies being used on your team (like Azure). Sometimes it’s really straightforward. For example, if you need to do Azure storage maintenance, then you would look in the Azure folder and the Storage subfolder. It’s a specific script to work with a specific technology and uses specific cmdlets for that technology.

Possible issues: Other times it’s not straightforward. For example, if you need to file the “QuickBooks maintenance” script. Should that go in the “QuickBooks” technology folder? But wait, QuickBooks is an accounting tool, so maybe it belongs in the “Accounting” folder? Or maybe this is part of the QuickBooks Migration Project, so should it go there?

Option 4: Miscellaneous Scripts

Organize miscellaneous scripts that don’t belong to one specific project, department, or technology, into a “general” or “miscellaneous” folder.

Why it’s good: You can allow anyone to push commits to this folder since many people will just use it to create random things to experiment with.

What to do: These miscellaneous scripts can be organized by the product or general technology that they interact with, such as Azure, on-prem servers, etc.

How to Choose

There’s no “correct” way that will fit everyone. Your best bet is to pick and choose a combination of these four options and start organizing your scripts. See what works and what doesn’t for your team. Just like documents on your work computer, files get moved around and folders change. Expect and accept that that’s going to happen here too.

How to Organize in Source Control

One final piece of advice: Think of Git repositories like top-level folders. A single top-level folder is just as disastrous as 1,000 granular top files. (Too much of a good thing, as the saying goes…) Generally, many organizations have found that having one Git repository per module is the just-right level of granularity. Trust your instincts and your teammates; you’ll quickly find out what number of folders and what organization method works for you and your team. As you find what the number looks like, continue moving around files and folders and before you know it you’ll be in the correct habit of creating repositories. 

Become a Script Master through Control and Organization

You will never have a perfect organizational method for your scripts since people will be working on them all the time. That’s fine, and something you just need to accept. All you can do is try to create a system that enables cooperative work while providing security to both the scripts and the organization

Using version and/or source control, team members can work in tandem. Source control removes blockers and lets you get work done faster. Having the safety net of a rollback system, a general repository to experiment, and removing blockers lays the groundwork for your team to become script masters.

Otter is a straightforward (and free) way to manage your scripts. Not only does it prevent undocumented script executions and changes, but it also stores your scripts and tracks who, when, and what changes are made to them. Download Otter and take your first step toward mastering scripts!