Building Something from Nothing

It’s pretty cool to build something that people grow to rely on. It’s also a little terrifying for someone like me who has a habit of second guessing decisions and who tends to be risk adverse.

A few years back, the developer who writes our installers made changes to enable them to run in headless mode. To facilitate testing, I wrote a small PowerShell script that ran through the basic installation steps that our engineers and QA followed manually. My goal was to give this script to our QA department so they could download the build from our build system and with a few parameters, run this script against their testing environments to perform an upgrade.

We demoed it to our R&D organization. We showed how it could be tied into the build system so that when the build was completed, it could kick off a “deployment” job that would upgrade environments. It was pretty straightforward: Download the build binaries into a folder called “artifacts” in a sibling directory to the said script and then execute the script with a few parameters.

For over a year, no one really bit. My colleague and I just used it for installer testing, mostly. That’s not completely true. There were a few QA teams that decided to start using it, but if I had to guess, the automated upgrade script was being used for approximately one in twenty test environments.

At some point, the organization began to build out a performance testing team. They had requirements around automating the deployment of environments with more than one server. They were trying to replicate how our application was run in production environments, which meant that there might be dozens of VMs running different portions of our product.

Well, perhaps unfortunately for them, I was the only person really fooling with ways to automate the deployment of the product. I took their requirements and changed my script so that you could give it all the same parameters as before but also give it a “run mode” flag that’d tell it what kind of work it’d be doing. For example, perhaps you wanted to stop or start all of the application services, or just update the database, or maybe just install a subset of the products functionality againt the target VM. I (quite kludgily, if that’s a word) added that functionality to the existing script all while sticking to my original goal: Have a single PowerShell script that a user could run from anywhere with just a few parameters.

No one bit and the requester exited the company shortly thereafter.

Around the time I was asked to extend the functionality of my script, other teams began researching how to build out full automation for our SaaS environments. I sat with them and went over what I had devised. Obviously, what I had would not scale but I at least had sorted out the process to upgrade an environment and scripted it out. They began building a large, scalable automation system. I later found out that for several months my script was tucked inside their automation system doing some of the heavy lifting until they had re-written everything.

One day out of the blue, my manager sent me a link to a repository the performance team owned that had taken the script I had written and began developing additional tooling to coordinate their use case of large scale deployments. At the core, it was calling my script to do all of the installation work. I was both elated and annoyed. It felt like my work, which had largely been ignored or dismissed as being insufficient was being used and built on for bigger and better things. Additionally, the only direct proof that this was my code was a single comment left in the script that had my name and email and some commit messages vaguely referring to my repository.

In short, it felt like a group of engineers grabbed my work, tweaked a couple things, and ran with it without giving credit where credit was due. Still, swallowed my pride quick. The other side of the coin was this: I had built something that was seen as worthy of being reused. I had never done that before. Never mind that I was never hired as a developer. I came onto the build team through a string of transfers / promotions and built the script in my spare time. It wasn’t robust. It wasn’t easily extendable. It was not built with any solid patterns. My goal was to write a single, portable tool that people could use to upgrade their single environments with and it had grown into something much, MUCH larger.

Now days, it’s used to upgrade several dozen testing environments including several large multi VM product environments each day, some several times a day. Though I’ve never measured it, I’m positive It has by now given hundreds and hundreds of hours back to the engineering team and rescued them from a boring, thankless, and sometimes tedious task.

I learned a lot from building my automation tool. There’s a lot I wish I had done differently, but that’s okay. It’s in maintenance mode these days, it mostly does everything that it’s ever been asked to do, and it continues to be relied on two years after I demoed it initially. People keep waxing poetic about a replacement but it hasn’t happened yet.

EDIT: I recently came across another instance of this code being used without attribution (which is fine, but goes against the spirit of code sharing) for some tools being shipped to a customer. 😐

 

Leave a Reply

Your email address will not be published. Required fields are marked *