Upgrading Atlassian Applications via Ansible

Over the next few weeks, I’ll be doing write-ups on how I upgrade some of our developer tools. Specifically, I’ll focus on the Atlassian applications I manage, but the process for our other tools (like Artifactory, SonarQube, etc.) is largely the same and most of the Ansible Roles are common across all of the applications.

For now, the playbooks and roles will not be available but I’ll post relevant snippets where I feel it’s needed.

Finally, we use RHEL/CentOS internally (for hosting our internal tools, we have other OSes for doing builds) so our playbooks are oriented towards a single OS.

 

This Old Blog

I originally created this blog as a way to document my career-work. I have a real passion for what I do and I truly enjoy sitting down and looking for refinements in an existing system or process. Still, my personal passion is not my career. I draw a clear line between my personal interests and the work I do to pay the bills. I know that doesn’t work for all people. We’re told to follow our passions and I do but just not through my 9-5 type work.

Still, I haven’t written about my work in a long time, so here’s my attempt to lay out some of the projects I’ll be working on and documenting next year.

  1. Automate the deployment of all our internal applications. Currently our internal tools are upgraded manually and approximately every six months. I’d like to be able to do that on a regular basis and have it be touchless. I’ve already laid the groundwork, demoed it to my team, and I’m over 50% complete.
  2. Create a SSP for common requests we currently handle. There’s a lot of work that’s simple and repetitive for us (like creating repos, build configurations, user provisioning) that we’d be better off not dealing with. I propose a self service portal for teams to use to get these things done faster and without us being directly involved with each request.
  3. Automate all of our release procedures. We do some work, like release branching andcertain packaging tasks, manually today. I’ve already automated a lot of this but it needs to be rolled out fully so some of our tasks become virtually touchless.
  4. Completely automate provisioning of our build infrastructure and expose the provisioning scripts to engineering so they can review, propose changes, or even make those changes themselves.

So yeah, I think the theme of 2018 is automation and standardization. It’s gonna be a good year, just gonna send it.

 

Who’s Actually Using Bitbucket?

I recently had to deal with an Bitbucket user issue where people were being added to a JIRA group automatically that granted r/w access to Bitbucket. Nearly half of the people who were in this group had never even logged into Bitbucket, but they counted against our license seats. We were running dangerously close to running out of seats and had only hours to spare. Rather than focus on the problems that lead up to this situation, I’d rather focus on the core question “Who’s actually using Bitbucket?” and how we regained nearly 1000 seats.

Problem #1: How many people are actually using Bitbucket?

In order to cast a wide enough net to find everyone, our metric was “Who’s ever logged into Bitbucket?” To do that, we queried the database (MySQL) and exported the results to CSV. Below is the query and what the resulting output looked like.

select cua.attribute_value, cu.user_name from cwd_user_attribute as cua
inner join cwd_user as cu on cu.id = cua.user_id
where cua.attribute_name = "lastAuthenticationTimestamp";

The resulting CSV looked something like this:

attribute_value,user_name
1497642118994,user_a
1497642069440,user_b
1497642117652,user_c

Problem #2: How do we put these people in a new permissions group?

To solve that, we turned to JIRAs REST APIs. First we created a new group in JIRA, then we wrote a little bit of code to loop through the CSV and add those folks to the group. I need to preface that this snippet was written very hurriedly to solve the problem (I have an ego to protect). This is written in Python 3 and uses the Requests module.

import requests
import csv

url = "https://<JIRA-URL>/rest/api/2/group/user?groupname=group-name"
admin_user = "admin"
admin_pass = "password"

headers = {'content-type': 'application/json'}
post_data = """{"name": "%s"}"""

def make_json(user):
 return(post_data % user)

with open('active_users.csv', newline='') as csv_file:
 reader = csv.reader(csv_file)
 for row in reader:
 user = row[1]
 response = requests.post(url, auth=(admin_user, admin_pass), headers=headers, data=make_json(user))
 if response.status_code == 200 or response.status_code == 201:
 print('{} added to new group'.format(user))
 else:
 print('Failed to add {} to new group'.format(user))
 print('Status code: {}'.format(response.status_code))
 print('API Response:\n\n' + response.text)

There were a few errors, but they seemed to all just be local Bitbucket users. No big deal.

Problem #3: How do we update all of our Bitbucket projects with this new group?

Unfortunately, Atlassian does not provide a way to bulk edit project or repository configurations. This is terrible oversight in my opinion, but is out of scope for this post. Still, though…

To solve this problem, we turned to Script Runner for Bitbucket. We performed the work in this order to prevent anyone from running into errors.

  1. Add the new group to all of the projects.
  2. Add the new group to global permissions.
  3. Remove the old group from global permissions.
  4. Remove the old group from all of the projects.

We used the provided code snippets from Adaptavist. See docs here.

 

 

Atlassian

Atlassian must be one of the most evil brilliant companies ever. They sell what, on the surface, appear to be amazing pieces of software, but underneath the surface there’s an ulterior motive. The reality is that you only get 75% of the functionality you need and are forced to buy add-ons offered by Atlassian or 3rd party software vendors to accomplish the remaining 25%. I buy the app, deploy it, realize I need it to do something else, and am forced to return to the market to purchase bolt ons. The vendors get paid, Atlassian gets their cut (and doesn’t have to develop anything if a 3rd party vendor has them covered), the admin is happy, and the users get some piece of functionality they really wanted. It’s only much later that I realize how light my coffers are and how slow my JIRA server has become.

They do platforms extremely well.

For the record, I like Atlassian. I’m mostly impressed with their software, what its capable of, and the community that’s popped up around it.