Aggmedia blog

Let the junior person speak first

Monday, 22nd March 2010 - 3:46pm (EST, GMT+1100) by Richard Bennett

"When opinions are asked, the tradition in the military is to let the junior person (rank in this case) speak first. This approach gets ideas on the table without having the senior set the direction at the outset." (Tom Stephen)

Switching from email and Bugzilla to Tender

Saturday, 20th March 2010 - 11:09pm (EST, GMT+1100) by Richard Bennett

The release of MacCallGrind 2.0 last month marked a return for us to developing commercial software products after a break of about 15 years, and in that time a lot has changed in what is now called the micro ISV market.

Back in the 80s and 90s, the only way to sell a software product was through a publisher, who would have the distributor and retailer connections, and would take 60-80% of the revenue, much like the record companies have until recently. Don't get me wrong, I've worked with some really good and really nice publishers over the years, who certainly earned their cut. But these days small developers like ourselves can now take control of the entire development, marketing and sales cycle, which means we can put much of the revenue back into development, instead of the pockets of the man.

The great thing about publishers though, was that they filtered all the support requests, and only sent on the ones they'd been able to reproduce and verify, meaning it was usually a snap to find the bug, fix it, and send back an update or patch. But now, we have to do all that ourselves. And so we're taking the jump into Tender, a fairly new web based support tool.

We're still finding our feet, so please bear with us, and if there is anything critical that you need support/help for and you're not getting a fast enough reply, then please continue to use our phone number if necessary.

MacCallGrind 2.0: pricing and trialling explained

Saturday, 6th March 2010 - 12:29am (EST, GMT+1100) by Richard Bennett

You may have noticed that we finally released MacCallGrind 2.0 at the end of February 2010, our Mac callgrind profiling tool. This is our first commercial product under the Aggmedia banner, and personally my first product in almost 15 years. We're pretty stoked to finally get it out.

MacCallGrind 2.0 has had a long 18 month development cycle, due to it being primarily a side project for our company. What started as a freeware side project at version 1.0, has now turned into a fully funded product, with internal development resources allocated to it. We think this is a good thing for MacCallGrind users, and good for all the wishlist features we're hoping to add in the future.

However we have received a few comments about how we approached the 2.0 release, particularly the 1MB open file limit in the trial version, and the US$149 price tag. I'd like to address each of these, try to explain why we chose to do it, and what we're doing about it.

We feel the price comes down to the value gained (lots), the potential market (very small), and the cost of development (a moderate amount), along with consideration of the rest of the profiling tool market. How much value would you gain from using MacCallGrind to tune your software? A couple of hundred page views per minute? An additional 500ms or more off your average page request time? These are high value metrics, and we'd suggest are a bit of a bargain for $149, if your company depends on a high performance web site. We originally considered pricing it at around the $500 mark, which would be more in line with professional development tools for the level of performance software we're targetting, and would better cover the development costs we've put into the product. But we thought the jump from free might be a concern for some people, so we specifically chose a lower price.

The clear majority of users of MacCallGrind are professional software engineers, and due to the nature of code profiling, are a senior software engineer in particular. They either work on a consulting/contract basis for other companies, or work full time for a development company. In the case of a consultant, $149 would be about an hour or two of contracted work time, and for a development company, $149 is fairly trivial for the performance gains being realised.

So we don't feel the price is expensive, on the contrary, we think its pretty cheap. Thus the concern by some people is probably because the previous version was free. Version 1.0 was freeware, and it remains freeware. We've decided to no longer distribute it, but if you have a copy, we're more than happy for you to keep using it for free. There's no obligation for you to upgrade to 2.0 if you don't want to.

The remaining group of users is those working on open source projects. But again, the knowledge to tune performance is that of a professional software engineer, and in most cases they're already doing performance tuning in their paid work. And there's always the free 1.0 version if they already have it anyway.

We had planned to release a "Not For Profit" version of MacCallGrind 2.0, which would be free for use for open source projects, but in the end we decided against having to maintain an additional ongoing development branch. Instead, if you are an open source project which relies on the performance of your code, then please contact us and (at our discretion), we'll give you a free 2.0 license for your open source project. We had always planned on doing this, we just hadn't had a chance to announce it yet.

And finally, the original release of MacCallGrind 1.0 asked users to post a message in our blog to say they're using it. If you did this, or we have spoken with you directly at any time since the original release of MacCallGrind, then you will be receiving a free 2.0 license over the next few weeks. If it doesn't arrive, then please let us know, and we'll make sure it happens.

In summary, we think the price is fairly low for what MacCallGrind 2.0 gives engineering professionals; we have a free upgrade path for open source projects and users of 1.0; and if you still have an issue with the price, then there's always the free options for the Mac like KCacheGrind (under Fink), and the open source Webgrind project. Having said that, we are listening to feedback, and we are looking again at what the price for MacCallGrind should be.

Which brings us to the 1MB trial file size limit. You can download MacCallGrind from our web site, and all its functionality works. The only restriction is that it will only open files up to 1MB in size. Buy a license, and obviously that restriction goes away. We've been active in the performance tuning market for over 20 years, and unfortunately there are unscrupulous engineering consultants who will use trial software tools for their consulting projects, so they don't have to buy the tool. A time limited restriction on MacCallGrind would allow engineers to do their performance tuning with their trial, and then not have to buy the product. This is just the nature of the irregularity of performance tuning unfortunately, and is why we instead switched to using a never ending trial but with a file restriction, so engineers can test it with test code or moderately sized production scripts.

Callgrind files can be quite small, even for long code paths, and while we still think 1MB is enough to trial MacCallGrind's features, we have listened, and have decided to up the limit to 3MB. It will however take us a few weeks to make this happen, so please check back with us later.

If any of this is unclear, or you have any concerns or questions about MacCallGrind, then please contact us. We think MacCallGrind is a good solid product, at a good price point, and we have some great features coming in following releases. But we are listening to your feedback, and are always open to suggestions about how we can make MacCallGrind an even better product.

Delete line, duplicate line, key bindings in Xcode and Mac OS X text system

Monday, 14th December 2009 - 1:05pm (EST, GMT+1100) by Richard Bennett

Firstly you've probably read my other concerns about delete line and duplicate line, in that I can't believe any modern IDE or text editor wouldn't include these as standard. Luckily however the Mac OS X text system does include modifiable key bindings. There's a couple of posts on the web about how to do it, but they're either too vague to be useful, or they implement the commands clumsily.

Here's a step by step of how I do key bindings for delete line and duplicate line in Xcode:

1. Create the directory ~/Library/KeyBindings if it doesn't already exist
2. Create a file called PBKeyBindings.dict
3. Paste the following into the file and save it:
   

   {
       "~d" = (
           "selectLine:",
           "copy:",
           "moveToBeginningOfLine:",
           "paste:",
       );
       "~D" = (
           "selectLine:",
           "copy:",
           "moveToBeginningOfLine:",
           "paste:",
       );
       "~e" = (
           "selectLine:",
           "cut:"
       );
       "~E" = (
           "selectLine:",
           "cut:"
       );
   }

4. Launch Xcode, select the Preferences menu item and select the Key Bindings tab. Change the Key Binding Sets drop down list to "Xcode Default". Key Bindings in PBKeyBindings.dict won't work if you have a custom set selected
5. Quit and relaunch Xcode.

This gives you option-e and option-E to delete the current line, and option-d and option-D to duplicate the current line. If you want control instead of option, then change the tilde "~" in the key binding dict to caret "^", and then quit and relaunch Xcode.

If you add these definitions to ~/Library/KeyBindings/DefaultKeyBinding.dict as well, then they should work in other IDEs and editors as well.

Count lines recursively in a Linux directory, ignoring subversion/svn

Wednesday, 9th December 2009 - 5:44pm (EST, GMT+1100) by Richard Bennett

Hacky count of lines recursively in a directory.

grep -r "" local | grep -v .svn | wc -l

MySQL on Amazon EC2

Tuesday, 3rd November 2009 - 12:19am (EST, GMT+1100) by Richard Bennett

Ars Technica is reporting that Amazon is now offering MySQL support for their EC2 cloud service.

Mac PHP editors -- the holy grail

Friday, 30th October 2009 - 9:01am (EST, GMT+1100) by Richard Bennett

How difficult is it to make a PHP editor for the Mac? Up to now I've suffered from what should really be a thriving market for good IDEs. The closest I've found to perfection, is Zend's ZDE 5.5, the standalone IDE they made before moving to Eclipse. And when I say perfection, I really mean productive enough that the tool doesn't get in the way of the work.

Everything I need that's missing from various IDEs can usually be worked around somehow, but workarounds are like sore thumbs, they distract your attention and reduce your productivity.

Every six months I download the latest version of the most popular IDEs or editors on the Mac, and try the following tasks:

• Mount a project volume via FTP/SFTP and be able to easily see it in a hierarchical view and click to open each file. PHP's a web language, yet for some reason the ability to easily edit files on a server doesn't seem to be near the top of the requirements list for editor developers. Sure, I can use Finder's FTP mount, but that's a finder window that doesn't usually integrate with the editor. I could you MacFUSE, but its really slow, really unstable, and you still need to mount stuff using shell. Zend's ZDE 5.5 is the only one that supports FTP/SFTP to a usable degree. Don't even start me on ZDE 6 and Eclipse's braindead and unusable FTP support.
• Delete a line. I'm amazed at the number of editors that don't support this out of the box. Delete a line. Seems pretty core editing functionality to me. And when it comes to defining macros to do it, most editors make you jump through multiple hoops in order to build even the most basic macro.
• Duplicate a line. This is my second most used function aside from the basic arrows, enter, code completion and delete a line. Yet again, very few editors actually support this out of the box.
• Integrated PHP manual. When code completion pops up, I want to see a tool tip of the complete PHPDoc from either my own code if its one of my functions, or from the PHP manual if its a PHP function. We all know how inconsistent and badly named PHP functions are, and its a pain in the arse keeping the PHP manual web site open, or even worse, accidentally clicking on w3schools.

I currently use Zend ZDE 5.5 for all my PHP work. For everything else I use BBEdit, which suffers the same problems as most editors for PHP work. In fact when I emailed BB a few years ago to suggest that they needed an integrated FTP browser in the project window, they said "that's not our domain, just use the Finder FTP support or a third party FTP client". A year later they brought their FTP out, but its still not integrated with the project window.

For what its worth, TextMate, Coda and Espresso all have niggling issues that make them less productive than ZDE for a professional developer. It would be nice to have XCode support PHP, because while it suffers from similar problems, I have at least already personalised the editor.

What I really want though, is ZDE 5.5, but upgraded. Zend are arguably in the best position to know what a PHP developer wants from an IDE, and as far as I'm concerned, ZDE pretty much nails it. But then they went and moved to Eclipse/PDT, and while they keep adding features, it just doesn't do the basics that well, like FTP/SFTP and fast keyboard response, both of which are pretty bad in Eclipse. Basically it looks like what it is, a PHP IDE embedded inside a generic IDE platform.

I actually emailed Zend and suggested they open source ZDE 5.5, but all I got back was a form letter email for an Eclipse ZDE 6 training course, not even personally addressed. Not a particularly good way to keep your PHP developers happy, and certainly now one of my main reasons for not even entertaining the idea of switching to ZDE 6.

Why can't someone write a good PHP editor?

Directory and file size list as human readable display in Linux

Thursday, 22nd October 2009 - 4:21pm (EST, GMT+1100) by Richard Bennett

Pretty simple:

du | sort -nr | cut -f2- | xargs du -hs

Handling unintended duplicate rows/records from MySQL (or any database).

Monday, 19th October 2009 - 4:32pm (EST, GMT+1100) by Richard Bennett

Querying for duplicates

select columnWithDuplicateValue,count(*) as n from tableName group by columnWithDuplicateValue having n >1;

Querying for duplicates with additional criteria

select columnWithDuplicateValue,count(*) as n from tableName where additionalColumns = 'something' group by columnWithDuplicateValue having n >1;

Deleting duplicates

Copy the result table from the above, and create the following statement for each line.

delete from tableName where columnWithDuplicateValue = 'the value from the result table' limit 1;

Deleting duplicates with additional criteria

Copy the result table from the above, and create the following statement for each line.

delete from tableName where columnWithDuplicateValue = 'the value from the result table' and additionalColumns = 'something' limit 1;

Pregging query results

For MySQL result tables, copy/paste the result table into an editor and perform the following regular expression find/replaces, to create a list of delete statements.

Find = "\| (.*?) | \d* |" (without the double quotes)

Replace = "delete from tableName where columnWithDuplicateValue = '\1' limit 1;" (without the double quotes)

Querying duplicates with multiple distinguishing columns

select columnWithDuplicateValue,2ndColumnWithDuplicateValue,count(*) as n from tableName group by columnWithDuplicateValue,2ndColumnWithDuplicateValue having n > 1;

Deleting duplicates with multiple distinguishing columns

Copy the result table from the above, and create the following statement for each line.

delete from tableName where columnWithDuplicateValue = 'the value from the result table' and 2ndColumnWithDuplicateValue = 'the other value from the result table' limit 1;

Blogging

Sunday, 18th October 2009 - 1:40am (EST, GMT+1100) by Richard Bennett

One of the problems with having a lot of client work, is the lack of time left for self promotion and community interaction, namely blogging.

When blogging took off back in 2003, part of our daily work routine would be reading and writing blogs. This got the company I was working for out into the space we were working in, and brought knowledge of the bleeding edge back into the company. In retrospect, the writing probably did more for me personally than for the company, as the company eventually went under, and yet my posts are still out there on the web with my name on them. Not that I got too much out of it that I wouldn't have without the company's support, as I was already blogging before I joined them, and may well have had the same interests even if I hadn't.

The impact of reading blogs however was even more distorted. We were in the blogging and web content space, the bleeding edge at the time, and there was a mass of blog traffic about the technology behind blogging. At one point I was reading ~300 blogs, it was insane. And while some of these weren't blogging related, I was only reading them because I already had a daily routine for blog reading. As the technology of blogging and microcontent started to stabilise and I went on to other ventures, it became difficult to justify spending the time just reading, and I eventually went cold turkey. What I failed to see was how the technology I was working with had distorted my perceived need for, as No.5 would say "input!".

Since then, portals have begun to restore themselves to their rightful place in the world, and email has prevented yet another potential breakout. These days I read one portal site regularly to get my updates, and subscribe to RSS feeds of several mailing lists to get the rest. I rarely keep up with them, and I skim them about once a week. RSS has become the tool that it was designed to be, a pull syndication technology, nothing more, nothing less. And as we were all hoping all those years ago, we can now apply filters to make sure we only get the information we want, such as monitoring the web for mentions of our products. Most other stuff these days you can find via a portal's feed or a syndicated link blog.

Aside from time, the other reason why my blogging is so irregular, is client confidentiality agreements. There's a wealth of blogging and research material in client projects, yet in most cases it is best kept inside the organisation.

So more blogging? Sure. I'll sign up for a bit of that.

Colouring svn diff output

Monday, 11th May 2009 - 8:31pm (EST, GMT+1000) by Richard Bennett

svn diff doesn't colour diff output, and all the fixes on the web are dodgey hacks and wrapper code that don't really do what most of these authors really wanted in the first place: svn diff to be coloured.

So here's my solution:

Enter the following into a file called "svn-coloured-diff.php" in your home directory and chmod it to 755 (be careful of newlines and line wrapping caused by web site layout):

#!/usr/local/bin/php -q
<?
// option to temporarily turn colour off
$showColour = true;
// option to temporarily force colour into pipes as well as tty
$forceColour = false;
// grab the paths to diff
$fn2 = array_pop($argv);
$fn1 = array_pop($argv);
// remove our path
unset($argv[0]);
// we're left with the switches and their parms, which with labels (-L) could be broken
// across multiple array elements, so make a string, rip out the labels and leave the
// remaining switches.
$labels = array();
preg_replace('#-L (.*?)(?= -L|$)#se','$labels[] = "$1"',join(' ',$argv));
$switches = preg_replace('#-L (.*?)(?= -L|$)#s','',join(' ',$argv));
exec('diff '.$switches.' --label "'.$labels[0].'" --label "'.$labels[1].'" '.$fn1.' '.$fn2,$diffLines);
// configure text styling for screen only (not for piping or non-TTY)
if ((posix_isatty(STDOUT) || $forceColour) && $showColour) {
   $addedStyle = chr(27).'[1;42m';
   $removedStyle = chr(27).'[41m';
   $resetStyle = chr(27).'[0m';
} else {
   $addedStyle = $removedStyle = $resetStyle = '';
}
// display it
foreach ($diffLines as $line) {
   if (substr($line,0,1) == '+') {
       echo $addedStyle.$line.$resetStyle;
   } elseif (substr($line,0,1) == '-') {
       echo $removedStyle.$line.$resetStyle;
   } else {
       echo $line;
   }
   echo "\n";
}
?>

Next edit ~/.subversion/config and find the line:

# diff-cmd = diff_program (diff, gdiff, etc.)

Replace it with this:

diff-cmd = /home/your-user-name/svn-coloured-diff.php

There's two booleans at the top of the script to make it a little more flexible. $showColour allows you to turn off all the colour coding, in case you need the original diff output; and $forceColour forces it to always include ANSI colour coding, when normally it doesn't colour code if there's no screen (i.e. you're piping or something similar).