fec2din, a simple script to clean up ec2-describe-instances output

After looking at this block of output one too many times, I decided enough was enough, and set out on fixing it:

$ ec2din
RESERVATION r-57192e11 103757199432 production
INSTANCE i-89ba1299 ami-a7f539ce ec2-50-99-41-60.compute-1.amazonaws.com ip-10-99-241-197.ec2.internal running production 0 t1.micro 2011-11-09T07:16:45+0000 us-east-1c aki-805ea7e9 monitoring-disabled 50.99.41.60 10.99.241.197 ebs paravirtual xen sg-c129fe91 default
BLOCKDEVICE /dev/sda1 vol-938be057 2011-11-09T07:17:07.000Z
BLOCKDEVICE /dev/sdj vol-c47fb1a9 2011-11-09T07:36:06.000Z
BLOCKDEVICE /dev/sdk vol-604da915 2011-11-09T17:18:37.000Z
...

Using a simple bash & awk script, I created a wrapper for ec2-describe-instances (ec2din) which re-formats the output in a more human-readable output...

AWS SES : Solving a problem with another problem

As a huge fan of Amazon Web Services, it pains me to say this: We've got a black sheep in the flock. As of September 2011, Simple Email Service (SES, originally released January 2011) is still in beta, so these critiques may not be relevant when the service is made official. That said, Amazon has had many AWS services in beta for years; I don't have high hopes for SES leaving beta anytime soon.

"wc -l" too slow? Use "qwc" for quick line counts

Line counts: a staple of command-line administration

wc -l parse_urls.py? "BAM! 426"

wc -l /var/log/httpd/access_log? "No problem 18203"

wc -l 10gb.csv? "Thinking, thinking, hammering your drive, spiking your load, thinking, EXACTLY 4,892,454"

Merging email messages in Kerio Connect Mail Server

As an administrator of Kerio Connect (Mail Server, KMS, whatever you'd like to call it), I have found myself needing to move e-mail messages around, often in excess of 50,000. Performing a move this large using an MUA like Apple Mail, Thunderbird, or even Microsoft Outlook is time-consuming and error-prone, resulting in thousands of duplicated messages in the destination directory. Kerio utilizes a very simple, Maildir-like backend, which makes moving to a new, empty mail folder a breeze! However, if you need to MERGE messages (a common administration task), you're in for a world of hurt, unless you follow this guide.

Don't hate the tape! LTO vs. hard drive backup

A lot of old administration tools suck by today's standards, but tape-based backups don't. Often, they are more cost-effective, durable, and even flexible than hard drive, optical, or cloud-based backups.

Just 5 years ago, I probably would have lumped tapes in with the archaic, over-priced, under-featured technologies sold to "enterprises" who cling to outdated technology like a grandparent to their 1972 Lincoln Continental; the sort of technology I looked forward to ripping out of an existing architecture and replacing with something less expensive and more flexible

Tags:

Sphinx Speeds Sizable String Searches, Substituting SQL (p3 - Programming)

Previous sphinx entries: Part: #1, #2

After you've completed the Overview and Creating Index section of this instructional series, you will have a sphinx index that is command-line searchable. Queries sent to this index via sphinx's search program will accept a string to search as an argument and return a list of document ID's matching the search term.

Sphinx Speeds Sizable String Searches, Substituting SQL (p2 - Creating Index)

In my last post, I gave Sphinx a brief overview. Let's actually install sphinx, setup a simple sphinx index, and run a command-line query against it.

Installation

This tutorial covers a Linux/*nix installation, although only the installation should be notably different.

Download sphinx

Sphinx Speeds Sizable String Searches, Substituting SQL (p1 - Overview)

Sphinx has compelled me to finally get around to setting up my own blog. I couldn't tell enough people I already know just how awesome it is... And many of the people I do know are tired of hearing me talk about it. I'm branching out, and you're next

Sphinx is awesome, and I'll get right to the point.

Subscribe to Scott Bigelow $ ./blog RSS