The only thing more tedious than having to do things manually for lack of automation is automating tedious things.
Author: cb
ARC 1.0 is Done
Today, I finished the first version of ARC. There are a few minor changes to come, but the tagged 1.0 commit is fully functional. You can see the repository on GitHub here.
Minor Updates
- Working on cleaning up KillAllBindings a little. I’ve made some (probably) final changes that I need to test out. Once that’s done, I’m not certain what else I’ll be doing to/with it unless a bug comes up or there’s a need for functionality to be extended.
- I’m continuing progress to extending the testing quality of development builds internally while trying to address build speed concerns. Recently I optimized our build process and got build time cut by about 30%. We’re looking at other ways to speed this up all while performing more testing during the build process (which may offset the optimizations, but I’m cool with that).
- I’ve started working on a tool that will make source code collection for backing up source easier. It’s called ARC (Archive Released Code) and more information will come as I get closer to completion.
Happy New Year
Stay safe and have fun.
Deploying Builds Automagically
The main application we ship is pretty big and complicated. Currently, the only way to install it is through the installer. The installation is, from what I’ve been told, pretty complicated. Installing the application by updating binaries and restarting IIS is not an option.
Recently, a teammate configured the installer to run headless by passing a XML to it with all of the properties needed to install and we’ve been dogfooding it within R&D to help ease the burden of installing builds. I wrote a PowerShell script to connect to the server we want to upgrade and performs the installation. It’s designed to run connect to a server using a remote PowerShell session, copy the artifacts over, backup the database, update the database, then update the application.
Now that we’ve gotten this far, I’m trying to implement a set of tests to run post install and pass/fail the build based on these additional criteria (Did it install? Did it pass all the tests?). I’m trying to have the running, at least nightly, by the end of the year.
Process Enhancements
For a while now, we’ve required code reviews for all changes before they could be merged into a team’s trunk fork. This helped catch some obvious stuff and helped increase code quality. This stuff seems obvious but when it’s not part of the culture, it seems like a big feat.
The next step was to require the code to build before it could be merged. We implemented this about a month ago and so far it’s been a pretty smooth transition. Bamboo picks up the branches (annoyingly, it will build a newly created branch even if there are no commits but that’s another matter), build all of the solutions, and report the status to Stash. If the build fails, the pull request can’t be merged. Progress.
Next, we need to implement unit tests. One team has gotten that working in their fork so it’s just a matter of time to get the changes merged.
This is all text book stuff and it’s been neglected for a while, but it’s good to see us begin to enhance our process. It means more work for me, but it also means that there are less firedrills because less issues will slip through the cracks and into customers hands. I came from QA and I know what it’s like to be in the room with a pissed off manager, pissed off support agent, and a pissed off customer and it feels like it’s all your fault because you missed something.
Extended Downtime
This blog is hasn’t been updated since the beginning of the year and has been down for a long time. I decided to stand this server back up tonight and begin documenting things again.
The previous post mentioned six things that I needed to do by the end of January. Well, we accomplished all of them except for the last one (CI).
This has proven to be a daunting task that we’re still tackling but we’ve managed to get some things in place to help increase the quality of builds and cut down on the number of failed builds.
Over the next few weeks, I’ll document what we’ve done and where we’re headed.
The Next Month
In the next month, I have to:
- Prepare documentation to assist 200 developers in their transition from TFS to Git.
- Provide training for developers (do a video?).
- Solidify branching strategy and document branching strategy for developers and for release management team.
- Get builds working with Bamboo (This is already done, actually.)
- Migrate source code from TFS to Stash.
- Demo a rudimentary CI build, including unit tests, deployment to a test environment, execution of functional tests, and test results.
I’m kinda freaking out. I’m going back to work on Monday and I know the next month is going to be insanity…
Learning a New Language
I’ve been going back and forth for awhile on learning either C# or Java. They’re pretty similar syntactically and there are plenty of opportunities with either language. I’ve chosen C# for two reasons.
- The recent announcement that C# will soon be cross-platform.
- I was asked if I’d be interested in working on some InstallShield tasks for our installers.
As for #1, I’m not holding my breath, but I hope the support and the community will come. As for #2, I can’t say no to getting relevant, transferable experience.
So, in order to start learning C#, I’ve decided to re-write the Python application KillAllBindings that I shared (and then unshared) last week. I wrote this on my personal time on my own personal copy of Visual Studio. It’s done(-ish) and it works. I’ve tested it and I’m reasonably certain that it’d work for others. It’s not very well abstracted nor does it have any error handling, but it should do the job. I will likely use this application as base for continuing to learn C# in my personal time. Hopefully, I can add additional features (if needed) and get good error handling implemented.
I’ve put it up on GitHub for the world to see. Don’t judge too hard, It’s my first foray into C#.
Branching Strategy
We’re spending a lot of time discussing our migration from TFS to Git at work. We need this transition to go smoothly. I need this transition to go smoothly. There are a lot of doubters. Yesterday, I was talking to one of our senior DBAs about this and he said “I don’t care, I’m just waiting for it to crumble.” After I told him that I was the one driving this transition, he just sorta wandered off without saying anything else.
This is such a dramatic departure for all of our .NET developers that are used to Visual Studio being a one-stop-shop and I’m worried about backlash when we start making them use another tool (SourceTree since we use Stash).
Anyways, I wanted to go over our current TFS branching strategy and our proposed branching strategy in Git. Or, at a minimum, try to start documenting it.
Main----o---\-----------------o--o Dev Branches \----o----o----o/ There's a dev branch per team.
The current issue is that Main is used to integrate all of the teams code. As you might imagine, Main breaks. We don’t have a “golden” copy. Additionally, “branching” in TFS is more like a forking operation in Git. A branch is a unique copy of a particular project from some changeset. The major difference is that when you branch in TFS, you’re ONLY getting that one branch whereas in Git, you get the entire repository.
This is what we’re proposing in Git.
Master----o------------------ Integration\----------------- | | Fork of "Main" repo per team. | Master - - - - - - - - - - - Teams will not use this branch. Integration\---------------- Feature Branch\----o---o---o-^ Pull Request to merge into team integration branch.
At the end of the sprint, the dev lead will merge their teams integration branch into the “Main” integration branch. All of the other dev teams will need to either rebase their branches when another team commits their code to the Main integration branch or deal with resolving countless merge conflicts.
So, why not branches for these development teams? Well, we discussed this, but the assumption was that we could help keep everyone’s experience as clean as possible so they aren’t sifting through hundreds of feature branches.
What about releases? Hot fixing?
Release-X.y /------- We plan to branch and tag this. Potentially branch off of this for HFs?? Master----o------------------o Integration\-----------------^ | | Fork of "Main" repo per team. | Master - - - - - - - - - - - Integration\-----------------o Feature Branch\----o---o---o-^
Enter fork syncing. Any branch made on the repository that is forked off of will get this branch. In the event of a HF, they will make their fix in their Release-X.y branch and then submit a pull request to have this code merged into the Main release-X.y branch.
Why the hell is release management not merging all this code and resolving conflicts? We’ll, to be frank, there are a couple reasons.
- We don’t know the code base.
- The assumption that any developer should be able to resolve a conflict simply isn’t true.
- The code base is massive and touches so many technologies that no one person can know how it all works (hence #1)
The only time release management will merge code is when we merge into the Master branch or Merge code into a release branch for HFing.
The big benefit to moving from TFS to Git is the ability to enforce code reviews by taking advantage of pull requests. Additionally, we have other development teams that work on mobile apps and Java apps that are already using Git (previously GitHub, now Stash) so it’s be nice to get everyone using the same technology.
Finally, this branching strategy is still in it’s infancy and we’re still toying with it to make sure that it breaks the KISS rule and to ensure that it’s sustainable for everyone. Also, forgive me if I’m using any terms incorrectly, there’s a lot of TFS lingo beat into my head.