Being a Judge in the WordPress Plugin Competition « planetOzh

This year once again, Weblogtoolscollection.com organized their annual WordPress Plugin Competition, which has had this time a special flavor for me: I was not authorized to compete (being one of the winners of a former edition) but I did take a role in it, as I've been one of the judges.

Being a judge was quite a straightforward task: Mark from WLTC sent us a zip containing all the 43 plugins submitted to the competition, and asked in return for a grade from 0 to 100 for each plugin. Straightforward, but still quite a task, as it took me nearly a month to review all these plugins.

When the results are out, I'll publish a short review for each plugin, as I took notes and wrote down my thoughts through the whole process. I won't reveal the grades I gave (although Mark is free to do so), simply because I don't want to elaborate on why I rated 65 this plugin and 64 that one, but I'll post all the comments, critics, remarks, suggestions or ideas I've had.

Today and before the results are announced, I want to share the method I used to review each plugin, which is exactly the one I use when, as a user, I come across a plugin that I want to try. This way, if you've joined the competition, you'll already know what I may have liked or disliked ;)

First things first: a bit of packaging

The very first thing I review is… the download page. The plugin's page has to explain to me, clearly, what the plugin is about, what I'll be able to do with it. Like with anything in this world, if you want people to try something, you have to convince them, and you don't have long to do so.

The typical plugin page that works with me has the following, and very common, structure:

• A short description — the pitch, the sentence that has to make me sure that I'm going to be interested. If a screenshot is worth it, add one. The goal is to tell in no more than 15 or 20 seconds what all this is about.
• If applicable and if the short intro does not tell absolutely everything, add now a longer description, more details, features. A bullet list of features, if possible, is a winner in this section.
• A download link. Yeah, that sounds a bit stupid to state this, right? If only. On several download pages I had a hard time finding the download link, and on a few I simply did not find any, including viewing the HTML source of the page and looking for "download" or ".zip".
• A way to send feedback. It can be in the comments, or with a link to a support forum, but if I want to say thanks, to ask for help or to report a bug, I must easily find a way to do so.

Other non mandatory but always appreciated sections are:

• A changelog can be a nice touch, showing that the plugin is maintained, hopefully improved, maybe from users feedback.
• A FAQ. Save yourself and myself some valuable time and list the common questions users had or you think they'll have. This includes installation procedure if something unusual has to be done (like creating an account on a third party site for instance), removal, server requirements if different from those WP has (PHP5+ for example)
• Technical details, if applicable. If the plugin does something in a particularly witty way, or does something unusual and complex, explain please! This section is the place where you seduce geeks and coders, where you make them drool over your work before they've seen a bit of your code.

Overall, I think the plugin's page is definitely one of the most important component of the plugin. Unclear? There will be no users. Too long and too cumbersome to read? There will be no users. No download link? You get the point.

Second step: read the source

Before I even upload the plugin to my server, I check what lies under the hoods. I read the plugin source and try to follow the flow of events, to understand both what is done and how it is done. What I expect at this point is code that looks clean and that's readable. Otherwise, I usually run away.

First, I quickly skim the code and search for classic mistakes and errors. This may sound a bit perverted to look for errors first, but this is just the natural way to make sure something is not going to harm your setup. When you rent a car, first things you look for are an empty tank, flat tires or broken lights. If found, run away. If not, continue.

Once and if I have the feeling that everything is done nicely, I also read the source to be impressed by something, to fall in love with a code snippet I may steal later. Sources is where you learn things and become a better coder.

Note: the next day or so, I'll blog about those "classic mistakes" that litteraly infest plugins. Hint: subscribe or follow :)

Third and final step: actually try the plugin

If the plugin's page was good and if the plugin's code was nice, I eventually try the plugin. As a user, most of the time I simply don't go this far. As a competition judge, I always completed the process but honestly most of the time my opinion was done before activating the plugin.

What I want at this step is a plugin that has an understandable and easy to use interface. I have to know what to do with the interface just by looking at it, without needing to refer to the readme file or the plugin's page.

I've found that the best experiences were with plugins that have a true "WordPressian" look: you don't feel like you're on a custom plugin page but instead it's just as if it was something WP does out of the box. Ugly buttons, odd font sizes or weird spacing are distracting and make you expect something… unexpected. A solid interface is an interface you use but don't notice.

When trying the plugin I usually do two things. First, I use it as I would do it normally, as the average tech inclined user. Then, if possible, I act like the random, read dumb, user. For instance, if an input field needs an URL, I enter one without the prefixing 'http://' and see what happens. As a user, I don't go this far when testing, but as a judge I think it's important that a plugin has safeguards that will tell the user what went wrong instead of just outputting errors – or worse, just don't work without telling why.

Last step: write down, think, conclude

As a judge, the last step of each plugin review has been writing down my thoughts, my ideas or my remarks as they came, then give a grade to the plugin.

I hope that if and when you'll read the review of your plugin, you'll find at least something insightful, be it criticism, feature request or development idea :)