As a data warehouse, a significant part of my job involves log analysis. Besides the standard root cause analysis, I need to verify database writes, diagnose user access issues, and look for under-used (and over-used) data sets. Additionally, my boss needs quarterly and yearly reports for client billing, and some of our clients need usage reports to identify data they might be paying for but not using (which we can then shut off to reduce costs). This has recently become a popular space for new solutions.

On the other side, as a sysadmin, I need to get other reports like how all the machine's resources (CPU, memory, disk, network) are being used, what processes are running on the machine and how those processes used resources over time. This is basic monitoring, and there are lots of solutions here, too. In the true Unix philosophy, there are command-line programs to query every one of these, which write out text that I can then parse.

In my previous post about ysql, I showed how to use the ysql utility to read/write YAML documents to SQL databases. Now, Yertl has a ygrok utility to parse plain text into YAML documents.

Continue reading ygrok - Parse plain text into data structures...

How many times have you needed to do this?

my @found_names = grep { /^[A-D]/ } @all_names;
my @topfive = @found_names[0..4];

Or worse, this.

my @topfive = ( grep { /^[A-D]/ } @all_names )[0..4];

Or this.

my @bottomfive = @names < 5 ? @names : @names[$#names-5..$#names];

Or this.

my @names
        = map { $_->[0] }
        sort { $a->[1] <=> $b->[1] }
        grep { $_->[1] > $now }
        map { [ $_->{name}, parse_date( $_->{birthday} ) ] }
        @all_users;
my @topfive = @names[0..4];

There's got to be a better way!

Now there is! Introducing: List::Slice!

Continue reading List::Slice - Slice operations for lists...

For what seems hours, you scan the board. The colors are sharp against the simple background. Some movement catches your eye, but it doesn't feel right, so you ignore it. Time stretches on.

There! The perfect move. Leaving the perfect next move. A quick flick. A match. The pieces fall into place. Another match. Another. Another. A special piece. Another special piece. It fires, triggering more. Chaos consumes.

The board is in ruins. Your carefully planned next move is lost in the destruction. You're back to scanning the board to try to find where you belong in this new world.

Is this a game, or is it your development strategy?

Software development is chaos. Either you work to managing chaos, consuming it, or it works on consuming you. There are too many possibilities, too much input, to brute-force your way to completion (how much software do you know of that can be considered complete?).

In the face of these possibilities, a rigid development plan will fail. Vague goals are better. Goals written in terms of a problem are best. Problems don't change, once you find their roots.

I didn't know this post was going to be about Agile, but there it is.

Exact is for computers. We are not computers. We are human. We are chaos.

Static site generators are popular these days. For small sites, the ability to quickly author content using simple tools is key. The ability to use lower-cost (even free) hosting, often without any dynamic capabilities, is good for trying to maintain a budget. For larger sites, the ability to serve content quickly and cheaply is beneficial, and since most pages are read far more often than they are written, generating a full web page to store on the filesystem can improve performance (and lower costs).

For me, I like the convenience of using Github Pages to host project-oriented websites. The project itself is already on Github, so why not keep the website closely tied to it so it doesn't get out-of-date? For an organization like the Chicago Perl Mongers, Github can even host custom domains, allowing easy collaboration on websites.

It's through the Chicago.PM website that I was introduced to Octopress, a blogging engine built on Jekyll. It's through using Octopress that I decided to write my own static site generator, Statocles.

Continue reading Announcing Statocles...

I got a text at 8:00am:

"Hey, can you jump on a conference call?"

Groggy and disoriented, I blearily type the conference line and enter my passcode, followed by the pound or hash sign. At the tone, I would be the 6th person to enter the conference. Tone.

"The app is down, and trading has stopped."

Continue reading Mojolicious Triumphs Over Legacy Code...