Musings on software engineering, the artisans and the craft.
Monday, 22nd March 2010 - 3:46pm (EST, GMT+1100) by Richard Bennett
"When opinions are asked, the tradition in the military is to let the junior person (rank in this case) speak first. This approach gets ideas on the table without having the senior set the direction at the outset." (Tom Stephen)
Saturday, 20th March 2010 - 11:09pm (EST, GMT+1100) by Richard Bennett
The release of MacCallGrind 2.0 last month marked a return for us to developing commercial software products after a break of about 15 years, and in that time a lot has changed in what is now called the micro ISV market.
Back in the 80s and 90s, the only way to sell a software product was through a publisher, who would have the distributor and retailer connections, and would take 60-80% of the revenue, much like the record companies have until recently. Don't get me wrong, I've worked with some really good and really nice publishers over the years, who certainly earned their cut. But these days small developers like ourselves can now take control of the entire development, marketing and sales cycle, which means we can put much of the revenue back into development, instead of the pockets of the man.
The great thing about publishers though, was that they filtered all the support requests, and only sent on the ones they'd been able to reproduce and verify, meaning it was usually a snap to find the bug, fix it, and send back an update or patch. But now, we have to do all that ourselves. And so we're taking the jump into Tender, a fairly new web based support tool.
We're still finding our feet, so please bear with us, and if there is anything critical that you need support/help for and you're not getting a fast enough reply, then please continue to use our phone number if necessary.
Saturday, 6th March 2010 - 12:29am (EST, GMT+1100) by Richard Bennett
You may have noticed that we finally released MacCallGrind 2.0 at the end of February 2010, our Mac callgrind profiling tool. This is our first commercial product under the Aggmedia banner, and personally my first product in almost 15 years. We're pretty stoked to finally get it out.
MacCallGrind 2.0 has had a long 18 month development cycle, due to it being primarily a side project for our company. What started as a freeware side project at version 1.0, has now turned into a fully funded product, with internal development resources allocated to it. We think this is a good thing for MacCallGrind users, and good for all the wishlist features we're hoping to add in the future.
However we have received a few comments about how we approached the 2.0 release, particularly the 1MB open file limit in the trial version, and the US$149 price tag. I'd like to address each of these, try to explain why we chose to do it, and what we're doing about it.
We feel the price comes down to the value gained (lots), the potential market (very small), and the cost of development (a moderate amount), along with consideration of the rest of the profiling tool market. How much value would you gain from using MacCallGrind to tune your software? A couple of hundred page views per minute? An additional 500ms or more off your average page request time? These are high value metrics, and we'd suggest are a bit of a bargain for $149, if your company depends on a high performance web site. We originally considered pricing it at around the $500 mark, which would be more in line with professional development tools for the level of performance software we're targetting, and would better cover the development costs we've put into the product. But we thought the jump from free might be a concern for some people, so we specifically chose a lower price.
The clear majority of users of MacCallGrind are professional software engineers, and due to the nature of code profiling, are a senior software engineer in particular. They either work on a consulting/contract basis for other companies, or work full time for a development company. In the case of a consultant, $149 would be about an hour or two of contracted work time, and for a development company, $149 is fairly trivial for the performance gains being realised.
So we don't feel the price is expensive, on the contrary, we think its pretty cheap. Thus the concern by some people is probably because the previous version was free. Version 1.0 was freeware, and it remains freeware. We've decided to no longer distribute it, but if you have a copy, we're more than happy for you to keep using it for free. There's no obligation for you to upgrade to 2.0 if you don't want to.
The remaining group of users is those working on open source projects. But again, the knowledge to tune performance is that of a professional software engineer, and in most cases they're already doing performance tuning in their paid work. And there's always the free 1.0 version if they already have it anyway.
We had planned to release a "Not For Profit" version of MacCallGrind 2.0, which would be free for use for open source projects, but in the end we decided against having to maintain an additional ongoing development branch. Instead, if you are an open source project which relies on the performance of your code, then please contact us and (at our discretion), we'll give you a free 2.0 license for your open source project. We had always planned on doing this, we just hadn't had a chance to announce it yet.
And finally, the original release of MacCallGrind 1.0 asked users to post a message in our blog to say they're using it. If you did this, or we have spoken with you directly at any time since the original release of MacCallGrind, then you will be receiving a free 2.0 license over the next few weeks. If it doesn't arrive, then please let us know, and we'll make sure it happens.
In summary, we think the price is fairly low for what MacCallGrind 2.0 gives engineering professionals; we have a free upgrade path for open source projects and users of 1.0; and if you still have an issue with the price, then there's always the free options for the Mac like KCacheGrind (under Fink), and the open source Webgrind project. Having said that, we are listening to feedback, and we are looking again at what the price for MacCallGrind should be.
Which brings us to the 1MB trial file size limit. You can download MacCallGrind from our web site, and all its functionality works. The only restriction is that it will only open files up to 1MB in size. Buy a license, and obviously that restriction goes away. We've been active in the performance tuning market for over 20 years, and unfortunately there are unscrupulous engineering consultants who will use trial software tools for their consulting projects, so they don't have to buy the tool. A time limited restriction on MacCallGrind would allow engineers to do their performance tuning with their trial, and then not have to buy the product. This is just the nature of the irregularity of performance tuning unfortunately, and is why we instead switched to using a never ending trial but with a file restriction, so engineers can test it with test code or moderately sized production scripts.
Callgrind files can be quite small, even for long code paths, and while we still think 1MB is enough to trial MacCallGrind's features, we have listened, and have decided to up the limit to 3MB. It will however take us a few weeks to make this happen, so please check back with us later.
If any of this is unclear, or you have any concerns or questions about MacCallGrind, then please contact us. We think MacCallGrind is a good solid product, at a good price point, and we have some great features coming in following releases. But we are listening to your feedback, and are always open to suggestions about how we can make MacCallGrind an even better product.
Monday, 14th December 2009 - 1:05pm (EST, GMT+1100) by Richard Bennett
Firstly you've probably read my other concerns about delete line and duplicate line, in that I can't believe any modern IDE or text editor wouldn't include these as standard. Luckily however the Mac OS X text system does include modifiable key bindings. There's a couple of posts on the web about how to do it, but they're either too vague to be useful, or they implement the commands clumsily.
Here's a step by step of how I do key bindings for delete line and duplicate line in Xcode:
This gives you option-e and option-E to delete the current line, and option-d and option-D to duplicate the current line. If you want control instead of option, then change the tilde "~" in the key binding dict to caret "^", and then quit and relaunch Xcode.
If you add these definitions to ~/Library/KeyBindings/DefaultKeyBinding.dict as well, then they should work in other IDEs and editors as well.
Wednesday, 9th December 2009 - 5:44pm (EST, GMT+1100) by Richard Bennett
Hacky count of lines recursively in a directory.
Friday, 30th October 2009 - 9:01am (EST, GMT+1100) by Richard Bennett
How difficult is it to make a PHP editor for the Mac? Up to now I've suffered from what should really be a thriving market for good IDEs. The closest I've found to perfection, is Zend's ZDE 5.5, the standalone IDE they made before moving to Eclipse. And when I say perfection, I really mean productive enough that the tool doesn't get in the way of the work.
Everything I need that's missing from various IDEs can usually be worked around somehow, but workarounds are like sore thumbs, they distract your attention and reduce your productivity.
Every six months I download the latest version of the most popular IDEs or editors on the Mac, and try the following tasks:
I currently use Zend ZDE 5.5 for all my PHP work. For everything else I use BBEdit, which suffers the same problems as most editors for PHP work. In fact when I emailed BB a few years ago to suggest that they needed an integrated FTP browser in the project window, they said "that's not our domain, just use the Finder FTP support or a third party FTP client". A year later they brought their FTP out, but its still not integrated with the project window.
For what its worth, TextMate, Coda and Espresso all have niggling issues that make them less productive than ZDE for a professional developer. It would be nice to have XCode support PHP, because while it suffers from similar problems, I have at least already personalised the editor.
What I really want though, is ZDE 5.5, but upgraded. Zend are arguably in the best position to know what a PHP developer wants from an IDE, and as far as I'm concerned, ZDE pretty much nails it. But then they went and moved to Eclipse/PDT, and while they keep adding features, it just doesn't do the basics that well, like FTP/SFTP and fast keyboard response, both of which are pretty bad in Eclipse. Basically it looks like what it is, a PHP IDE embedded inside a generic IDE platform.
I actually emailed Zend and suggested they open source ZDE 5.5, but all I got back was a form letter email for an Eclipse ZDE 6 training course, not even personally addressed. Not a particularly good way to keep your PHP developers happy, and certainly now one of my main reasons for not even entertaining the idea of switching to ZDE 6.
Why can't someone write a good PHP editor?
Thursday, 22nd October 2009 - 4:21pm (EST, GMT+1100) by Richard Bennett
Monday, 19th October 2009 - 4:32pm (EST, GMT+1100) by Richard Bennett
Querying for duplicates
Querying for duplicates with additional criteria
Copy the result table from the above, and create the following statement for each line.
Deleting duplicates with additional criteria
Copy the result table from the above, and create the following statement for each line.
Pregging query results
For MySQL result tables, copy/paste the result table into an editor and perform the following regular expression find/replaces, to create a list of delete statements.
Querying duplicates with multiple distinguishing columns
Deleting duplicates with multiple distinguishing columns
Copy the result table from the above, and create the following statement for each line.
Sunday, 18th October 2009 - 1:40am (EST, GMT+1100) by Richard Bennett
One of the problems with having a lot of client work, is the lack of time left for self promotion and community interaction, namely blogging.
When blogging took off back in 2003, part of our daily work routine would be reading and writing blogs. This got the company I was working for out into the space we were working in, and brought knowledge of the bleeding edge back into the company. In retrospect, the writing probably did more for me personally than for the company, as the company eventually went under, and yet my posts are still out there on the web with my name on them. Not that I got too much out of it that I wouldn't have without the company's support, as I was already blogging before I joined them, and may well have had the same interests even if I hadn't.
The impact of reading blogs however was even more distorted. We were in the blogging and web content space, the bleeding edge at the time, and there was a mass of blog traffic about the technology behind blogging. At one point I was reading ~300 blogs, it was insane. And while some of these weren't blogging related, I was only reading them because I already had a daily routine for blog reading. As the technology of blogging and microcontent started to stabilise and I went on to other ventures, it became difficult to justify spending the time just reading, and I eventually went cold turkey. What I failed to see was how the technology I was working with had distorted my perceived need for, as No.5 would say "input!".
Since then, portals have begun to restore themselves to their rightful place in the world, and email has prevented yet another potential breakout. These days I read one portal site regularly to get my updates, and subscribe to RSS feeds of several mailing lists to get the rest. I rarely keep up with them, and I skim them about once a week. RSS has become the tool that it was designed to be, a pull syndication technology, nothing more, nothing less. And as we were all hoping all those years ago, we can now apply filters to make sure we only get the information we want, such as monitoring the web for mentions of our products. Most other stuff these days you can find via a portal's feed or a syndicated link blog.
Aside from time, the other reason why my blogging is so irregular, is client confidentiality agreements. There's a wealth of blogging and research material in client projects, yet in most cases it is best kept inside the organisation.
So more blogging? Sure. I'll sign up for a bit of that.
Monday, 11th May 2009 - 8:31pm (EST, GMT+1000) by Richard Bennett
svn diff doesn't colour diff output, and all the fixes on the web are dodgey hacks and wrapper code that don't really do what most of these authors really wanted in the first place: svn diff to be coloured.
So here's my solution:
Enter the following into a file called "svn-coloured-diff.php" in your home directory and chmod it to 755 (be careful of newlines and line wrapping caused by web site layout):
Next edit ~/.subversion/config and find the line:
# diff-cmd = diff_program (diff, gdiff, etc.)
Replace it with this:
diff-cmd = /home/your-user-name/svn-coloured-diff.php
There's two booleans at the top of the script to make it a little more flexible. $showColour allows you to turn off all the colour coding, in case you need the original diff output; and $forceColour forces it to always include ANSI colour coding, when normally it doesn't colour code if there's no screen (i.e. you're piping or something similar).
“Everything you always wanted to know about color management but were afraid to ask”
"The [Yahoo!] Exceptional Performance team has identified a number of best practices for making web pages fast. The list includes 35 best practices divided into 7 categories."
Value Stream Maps help us bring about organizational improvement, progress in our processes and methods, and most importantly, better software. Value Stream Maps can help both identify and stop waste in an organization. By leveraging the lessons explained in this article, your organization can find greater efficiencies and deliver higher-quality products to your customers.
"The biggest roadblock is pushback from highly-trained experts who are offended by the idea. After 8 years of medical school, and in a regime that already demands vast amounts of paperwork, why should a doctor have to check off basic items on a list? Because we are fallible in the face of complexity, Gawande says, and because checklists work. Although he led research in this area he was skeptical about adopting checklists in his own operating rooms. But when he did, he made two critical discoveries. First, well-made checklists are easy to use. Second, they almost always caught errors."
"Here, a Kanban system is used in a traditional waterfall development model but with a flow. This project has separate and serial processes which they call "design", "development", "validation" etc., and the Kanban cards move between processes. Each card represents a requirement for change or addition to the system and is a handoff to the downstream process. Note that this is not a classic waterfall process, where all the requirements are "designed" at one time, "developed", and "validated" at another time, which would cause all the cards to move in a group. Instead, the cards move one by one, like the one-piece-flow of manufacturing. What's happening here is a stable "sustaining" phase in a product‘s lifecycle, managed in a waterfall state transition model with a flow. Here, you can clearly see the "flow of work" concept instead, different from the "iteration" concept of Agile. It looks more like Kanban in factories than Agile Kanban does and it can be a pull system by making a rule to allow only the downstream process to move the cards 8. I call this "Sustaining Kanban", and find it similar to David Anderson's "Kanban System for Sustaining Engineering", which I discuss in the later section."
"You break the work process in a series different activities (phases). You then set a limit of how many job units you will allow at any phase. A simple rule of thumb: you can have a few more work units as you have people doing that job, so that each has one thing to work on at a time, and a small cache of completed jobs. The people in a given phase will do their work on a job unit to completion, so that it is ready for the next phase of work.
"This is where the somewhat brilliant key idea behind KSDM comes to play. The completed job does NOT free the person for working on another job, until that job is pulled into the following phase and work is started there. If work is piling up at a particular phase, those people are NOT ALLOWED to work ahead. As Taiichi Ohno makes so clear, that working ahead is waste. Instead of working ahead, they can look around to see what is wrong."
Goals Gone Wild: The Systematic Side Effects of Over-Prescribing Goal Setting — HBS Working Knowledge
"Advocates of goal setting have had a substantial impact on research, management education, and management practice. In this article, we argue that the beneficial effects of goal setting have been overstated and that systematic harm caused by goal setting has been largely ignored. We identify specific side effects associated with goal setting, including a narrow focus that neglects non-goal areas, a rise in unethical behavior, distorted risk preferences, corrosion of organizational culture, and reduced intrinsic motivation."
"Agile is mostly driven driven by managment and consultants, seldom bottom up by developers. Some developes are suprised about agile and don’t know how to react. Lots of literature is written about agile, but very little with the developer in mind. What do you need to know about agile as a developer?"
"The PHP worldwide developer count will grow to as high as 5 million developers by 2013, up from 3 million in 2007 and 4 million in 2009."
This week Bob talks with Eric Ries about his Lean Startup concept and taking an entirely different approach to building your startup. Is 1 success and 9 failures the only way to build software startups? Eric doesn’t think so, and he convinced me. This is an approach to building your startup that I predict will get a lot of attention in 2010.
The Australian Federal Government is pushing forward with a plan to force Internet Service Providers [ISPs] to censor the Internet for all Australians. This plan will waste tens of millions of taxpayer dollars and will not make anyone safer.
"Results of a public trial have bolstered the Australian government's plan to implement both mandatory and optional ISP filtering in 2010. For the moment, though, violent video games are safe."
In case you misinterpret the sarcasm, we do not agree with the study, the outcome or the actual proposal.
"The Government 2.0 Taskforce is being formed against a backdrop of increased interest by governments worldwide in the potential uses of public sector information and online engagement."
"The Taskforce will advise Government on structural barriers that prevent, and policies to promote, greater information disclosure, digital innovation and online engagement including the division of responsibilities for, and overall coordination of, these issues within government.
"The Taskforce will work with the public, private, cultural and not for profit sectors to fund and develop seed projects that demonstrate the potential of proactive information disclosure and digital engagement for government."
The contribution of this research is the development of the ‘naked objects’ approach to designing business systems, and the demonstration that the adoption of this approach yields significant benefits both to the developed system and to the development process. Using the naked objects approach to designing a business system, the domain objects (such as Customer, Product and Order) are exposed explicitly, and automatically, to the user, such that all user actions consist of viewing objects, and invoking behaviours that are encapsulated in those objects.
In an OOUI, the user interacts explicitly with objects that represent entities in the domain that the application is concerned with. Many vector drawing applications, for example, have an OOUI - the objects being lines, circles and canvases. The user may explicitly select an object, alter its properties (such as size or colour), or invoke other actions upon it (such as to move, copy, or re-align it). If a business application has any OOUI, the user may be selecting and/or invoking actions on objects representing entities in the business domain such as customers, products or orders.
This chart lists browsers that receive A-grade support as defined by Graded Browser Support. All YUI projects, including YUI 2 and YUI 3, aim to provide full A-grade support.
An education organization in France has successfully sued a software vendor for failing to comply with the open source General Public License. The ruling is significant because it demonstrates that software recipients, and not just developers, can launch GPL enforcement cases in France.
These are patterns & practices produced performance testing How Tos. You get to see them here first, before their final destination on MSDN. These How Tos represent current recommended practices. Use the how tos to supplement product documentation. Using the How Tos below, you can drill deeper into performance testing related areas, such as user experience modeling.
Software developers don’t really like to make schedules. Usually, they try to get away without one. “It’ll be done when it’s done!” they say, expecting that such a brave, funny zinger will reduce their boss to a fit of giggles, and in the ensuing joviality, the schedule will be forgotten.
So far, this isn't much different from the kind of analysis any competent operations team would conduct for a site outage. The next step is this: you have to commit to make a proportional investment in corrective action at every level of the analysis.
SproutCore 1.0 is a huge milestone. It’s the point where you tell developers your code is good enough for them to base their business on it. You only get to do it once, so its worthwhile to take the time to get things right. Sometimes that means doing more surgery than you would like. In the end, though, it is worth the effort to have something you can really love to use.
Email Blacklist Check - See if your server is blacklisted.
Great parody of Agile vs. waterfall methodologies. Hitler goes crazy when he learns his team has not been following the tenets of the Agile Manifesto.
Often you will see in computer books and articles a pattern where a function is applied to some but not all of it's required arguments, resulting in a function of fewer arguments. [..] This is called "partial function application"
In computer science, currying, invented by Moses Schönfinkel and later re-invented by Haskell Curry, is the technique of transforming a function that takes multiple arguments (or more accurately an n-tuple as argument) in such a way that it can be called as a chain of functions each with a single argument.
Today there are two leading debug protocols: DBGp, an open source protocol and Zend Studio's protocol. Both protocols have been used in commercial products in the last few years and are good solutions for debugging PHP applications and PHP web servers.
With more than members located around the globe, the Agile Alliance is driven by the values and principles of the Manifesto for Agile Software Development.
We recommend agile approaches to software development because they deliver value to organizations and end users faster and with higher quality.