The Importance Of Clear Error Logs

Once again, I’m reminded of just how crucial good error logs are. Last night our production MySQL instance went down, taking with it several missions-critical systems, namely JIRA. Of course, we didn’t know that at the time because JIRAs error logs never indicated a problem connecting to the DB, nor did at least one other affected application. Luckily, Stash had clearly put in the logs that the DB connection had timed out and we were able to use this information to go talk to DB guys to get this resolved.

I know that sometimes things fail in odd and unexpected ways and that this might not be a problem that could have been accounted for, but the takeaway is simply: make sure your logging is stout because it’s the difference between getting the system back up in a jiff or scratching your head.

This was the error that came up (in MySQL after we determined where the issue was).

141026 12:19:17 InnoDB: ERROR: the age of the last checkpoint is 9446264,
InnoDB: which exceeds the log group capacity 9433498.
InnoDB: If you are using big BLOB or TEXT rows, you must set the
InnoDB: combined size of log files at least 10 times bigger than the
InnoDB: largest such row.

It looks like we’ll need to some performance tuning to make sure we can handle large data blogs and large transactions.

 

JIRA REST APIs

JIRA has a collection of REST APIs that can be used for doing some tasks programmatically. I haven’t been using them for long but I’ve found three that are particularly useful.

I’m referencing to the 6.2.4 API doc which is found here. I used Python with Requests and the built-in JSON module to help parse through the responses.

reindex

If you want to re-index on a schedule, you can simply call this API and background indexing will start.

http://<jira_instance_url>/rest/api/2/reindex

If you get 200 or 202 back, then it means that indexing has been successfully started. If you 409 back, that means indexing has already started. The returned JSON will contain the % complete and the type of indexing.

project

If you want a list of all projects in your JIRA instance, you can call this API and you’ll get back a JSON list of them. You get several pieces of useful information back including the project URL, the key, and the friendly name.

http://<jira_instance_url>/rest/api/2/project

components

If you want to see a list of all the components for a particular project, you use the components API to fetch those.

http://<jira_instance_url>/rest/api/2/project/<project_key>/components

This may seem silly, but there doesn’t appear to be an easy way to produce a project chart from JIRA that would show this data. I recently used APIs to produce a Python script that dumps out each project and list of it’s components. For people who want to review this sort of thing, it makes their lives a lot easier. Below is an example of what it looks like, per project.

Project: My Cool Project
Key: MCP

UI/UX
Login
Core Functionality
Database
Security
APIs
======================================================

My only complaint with their APIs currently is that I cannot add versions via API and (as of 6.2.4) adding versions to multiple projects is tedious and error-prone because it’s so manual.