Author: cb

by cb on Ansible, Automation, Windows

Ansible and Text Encoding and Line Endings and Git and Windows and Frustration

I ran into an issue the other day with Ansible while provisoning a Windows machine. After installing InstallShield 2015 SAB, Ansible copies a small license file configuration file into the installation directory.

[FlexNet Publisher Server]
Server=Port@ServerName

That’s the configuration. Should be easy right?

Not in this case. I was confounded by errors in while testing the installation. If I opened up the file using Notepad and saved it, InstallShield would start working.

I had tried all of the usual suspects once I realized there was something wrong with the file. Change the text encoding from UTF8 to Windows ANSI. No change. Change the line endings from LF to CRLF. No change.

So what was going on?

As it turns out, my text editor (Atom) was adding an extra LF to end of the file. Why would it do that? Well, this part of the POSIX standard.

See: https://gcc.gnu.org/legacy-ml/gcc/2003-11/msg01568.html

Before saving.
After saving.

To get around this issue, I uploaded this file to Artifactory and I treat the configuration file as an artifact, like the InstallShield installer, and just download and copy it to the installation directory.

This is a reminder to be conscious of how Git and your editor treat files when you’re provisioning Windows machines. Occasionally, you will still run into maddening little issues like this. I don’t want to admit how much time this thing left me stumped.

 
by cb on Ansible, Automation, Cloud, Process Improvement

Re-Architecting This Website VIII

“Done. For now…”

All of my goals for the first re-architecture of this website are now complete. This evening I fixed the backup script, added a role that installed EFF’s certbot, and updated the README to reflect the current status.

You can see what’s new here: https://github.com/seaburr/WordPressOnDigitalOcean

Have a good week. In the coming weeks, there will not be much activity as I’ll be changing gears to Kubernetes.

 
by cb on Ansible, Automation, Cloud, Process Improvement

Re-Architecting This Website VII

“Baby steps.”

I’ve added a single change today. There’s new role that will install and configure the DigitalOcean monitoring agent.

In the coming days, I’ve got a few more items to wrap up. Once those (minor) missing pieces are in place, I will call this project done and move on to something else, like re-re-architecting this website using Kubernetes.

You can see what’s new here: https://github.com/seaburr/WordPressOnDigitalOcean

Have a good weekend.

 
by cb on Ansible, Automation, Cloud

Re-Architecting This Website VI

I can’t believe it’s been over two weeks since I touched this project. So far, things have been good. I took a look at AWS with the intent of redoing this project on Lightsail. I assumed it would be cheaper. Unfortunately, it won’t be. That doesn’t mean it’s not worth the effort, but it certainly changes the time table to migrate because it will actually cost MORE per month than hosting on DigitalOcean. Instead of focusing on migrating, I’ve decided to focus on general work that needs to be done which translates to any other cloud vendor.

Here’s what’s new:

  • Extended centos-base role to install and configure fail2ban for SSH and Apache. It also now includes two packages (htop and screen) that were previously missing.
  • New role called create-swap that will create and configure a swapfile.
  • Reconfigured install-apache to include mod_security and a more robust 80 -> 443 redirect.

You can see what’s new here: https://github.com/seaburr/WordPressOnDigitalOcean

Have a good weekend.

 
by cb on Ansible, Automation, Cloud

Re-Architecting This Website V

“Bootstrapped.”

Last night, I migrated this website onto its new infrastructure! The tooling I’ve been working for the last week has become feature-complete enough that it was used to create the database, droplet, and storage volume this site now uses.

There was some manual work (SSL configuration, import from previous website, updating DB connection string) however it was relatively minor and these things are likely to be the next features added.

Here’s where we’re at:

Database and WordPress directory have been moved off of the droplet.

I’m really excited about the progress made in a week.

 
by cb on Ansible, Automation, Cloud

Re-Architecting This Website IV

“Biting off more than I can chew.”

As of Sunday, I had gotten droplet creation, volume storage, and a good bit of the base OS setup work handled. The major missing piece was the creating the database. I am happy to say that work is now complete, but it’s not without bumps, warts, and bruises.

Creating the database server proved daunting because Ansible modules do not exist for DigitalOcean’s managed database server. I thought I’d just write a module or two, however that proved to be a much more daunting task than I first realized.

I’ve never written a module for Ansible, I’m no DBA, and the number of APIs exposed for database actions left me feeling deflated. Still, I soldiered through and created a Python script I can use to create, configure, or destroy a managed MySQL server. There are a ton of assumptions made to keep the scope of work reduced, but it’s sufficient for my needs at this point. It’s wrapped with argparse so I can run it from an Ansible command task then parse the JSON output.

Check it out here: https://github.com/seaburr/WordPressOnDigitalOcean/blob/master/roles/database-server/files/digital_ocean_database.py

As I said, I believe Ansible modules should exist for their managed database, as they do for AWS and GCP however, I think that’s a larger project than one individual can tackle on their first attempt writing modules. I’d be happy to collaborate on that if it ever comes up.

In related news, I’ve also extended the install-wordpress role to create a wp-config.php from template, including fetching unique salts from wordpress.org, I’ve also fixed a few small issues with the Apache installation and added missing packages.

At this point, the only thing missing to go from nothing to a functioning WordPress site is adding database connection information into wp-config.php and handling SSL.

 
by cb on Ansible, Automation, Process Improvement

Re-Architecting This Website III

“Automate all the things!”

This weekend I’ve been busy laying much of the groundwork requires to be able to re-deploy this website. In part one, I provided a diagram that showed how the entire website, including the database, is deployed on a single droplet (VM). While this model has worked well for about three years, it’s time to move on. To that end, I’ve begun building automation to create and manage WordPress sites.

I’ve created a repo on Github to house the infrastructure code. It’s available here: https://github.com/seaburr/WordPressOnDigitalOcean

As you can see, I’ve decided to continue using DigitalOcean. They’ve proven reliable. Unfortunately, DigitalOcean lacks some things, like NFS, that would make the website even more scalable. Still, I don’t feel they’re necessary at this point.

Currently, I have the ability to create/destroy VMs, create/destroy/attach storage volumes, and a decent portion of the application installation is complete. It’s capable of going from nothing to the WordPress setup page over HTTP.

Finally, I know I could just move all of this to AWS LightSail but what’s the fun in that?? I learn best by doing and I want to have a deep understanding with all the moving parts. Once I’m done building this out with DO, I might do the same on the other major cloud vendors.

 
by cb on Cloud

Re-Architecting This Website II

We’re onto part two! Luckily, this is a simple change.

The VM that runs this website is on DigitalOcean. It’s so old (3 years) that it was only provisioned with 1CPU/512MB of memory. This has been sufficient but it’s a tight squeeze with the reverse proxy and MySQL shoehorned onto the same VM. Tonight when I ran OS updates, I noticed this website went down. After looking further, the Linux OOM killer had stepped in and killed MySQL to keep the machine up and running. After starting MySQL, I scaled the droplet to a larger size.

DigitalOcean allows you to scale your droplet but they cannot perform hot add, so the droplet needed to be powered off. Luckily, the $5/mo droplet size has increased to 1CPU/1GB so I was able to double this VMs memory at no cost.

Let’s call this our “get well plan” so the next time updates are run, this site doesn’t go down.

 
by cb on Cloud, Process Improvement

Re-Architecting This Website I

Let’s embark on a journey to re-architect this website to something that is more resilient. Currently, the entire website runs on a single DigitalOcean droplet.

Backups are handled through a Bash script that zips up the application directory, dumps the DB, and tars up the output.

This could be better. Let’s make this better. As the website gets re-architected, I’ll provide diagrams and links to the code & documentation leveraged.

 
by cb on VMware

Installing CentOS 8 on VMware Workstation 15.5

It appears that the automated install of CentOS 8 does not work correctly in the latest version of VMware Workstation Pro 15.5 (15.5.0 build-14665864 at time of writing).

I’m not surprised considering it’s not even an option in the drop down but it’s good to know that there’s a workaround for this.

During an install , you will be prompted by the following error:

0016457: Pane is dead

The following problem occurred on line 31 of the kickstart file:

Section %packages does not end with a %end

Luckily, the workaround is simple. When you create your VM, select “I will install the operating system later.” You can configure the hardware and boot the machine for the first time. Afterwards, attach a optical drive with the CentOS 8 ISO and reboot. You can then go through the regular installer process.

The issue is logged as a bug on CentOS Bug Tracker though I’m guessing this is actually an issue with Workstation.

https://bugs.centos.org/view.php?id=16457

Even more good news! CentOS 8 ships with a much, much more modern version of Git, 2.18.1 which released in September of 2018 (latest at time of writing is 2.23 which released mid August, 2019). You no longer have to download and build from source just to use HTTPS Git repos 🙂