My good friend @barneyhanlon has been specifying the next release of the SVN version control system recently, with some help from our design team:

More of their contribution can be found on flickr

The latest version of Subversion is in alpha, and a list of new commands are in testing. I've added a list of some of those currently in testing so that you can familiarize yourself with them. Note that as the product is in alpha this will be a growing list as I believe there is a "wishlist" system going on.

svn repent
Used after another repository user has set svn blame. Indicates that the user acknowledges the fault and is very sorry.

svn resent
Used either after an svn blame function or to show disaffection with version control. The flag essentially shows that the use of Subversion in general is causing them distress.

svn relent
Used to either stop the above two commands or to stop arguments regarding changes to a committed file.

svn revolt
Turns the svn client into a simple FTP program, bypassing the version control system.

svn commie
Flag to show that the user has an ideology they wish to express. Allowed flags are -che, -stalin, -marx, -lenin, or -lolcats

svn tor -rent
Turns the subversion client into an anonymous Torrent client and hides the resultant .iso files in the repository. If used in conjunction with svn blame, then the IP address of another user is stored so the RIAA will seek legal action against the other user.

svn odd
Marks that changes made by another user look weird.

svn picard
Immediately makes a file live, sucking the entirety of the company bandwidth to do so, and bypasses the sysadmin.

svn ensign
Sends a request to the sysadmin for a tag and release, attaching a lolPicard to the email with the -caption attatched. The default caption is "wtf is this shit?"

svn fu man chu
The file is deleted, but the world shall here from it again.

Those who know me or are familiar with this blog will know I often bemoan (aka: rant about) the lack of good PHP software engineers, or the mass of "junior" ones.

But it might be worth thinking about why this lack arises. I said, in my recent "Getting started in PHP" post:

If you're not interested in becoming great, please don't start.

While probably a touch harsh, I feel it's a valid sentiment. But how do you become great anyway? How long does it take?

To be honest, I don't know. I've only been doing this for 10 years professionally.

So, am I slow, or does it really take over a decade?

Well, at the risk of a "piece of string" answer, it depends on two things:

- How much you need to learn
- How fast you can learn it

My feeling at the moment is that there's a great deal to learn, and it's not often well, or coherently, taught. Even finding out what you need to learn can be really tricky.

However, I'm going to try and throw together a partial list, to try and get some idea of just what you have to know to be good in this game - and what that means in terms of how long it takes a developer to "mature".

As per the previous post, you'll need to start off with some basic coding principles and grammar; statements, variables, and functions. Then, wrapping those into classes.

You'll need a coherent idea of how to properly manage your code library and file inclusions (ideally with a class autoloader), which tends to lead to a coherent naming strategy.

Then, I'd strongly recommend learning TDD (recommendations on good sources for learning it gratefully received!). Not only does this help you write stable code, in a way that's useful in commercial environments, it also helps you learn the importance of good code encapsulation, and of dependency management. It's often considered a fairly advanced topic, but my feeling is that it should be learned as early as possible. TDDing your early code will also greatly increase its lifespan.

If you want your code to stick around and evolve, you *must* learn version control. Personally my recommendation is for Subversion, but there are many people who prefer Git. (You're advised to google these terms liberally). Your code's no use if it's scattered all over the place and not backed up, and a repository will help keep your code together and safe.

At some point very early on, and as mentioned previously, you need to learn about security. However, you can reduce your exposure to risk by using good practices, and data access layers such as prepared statements, PDO and Zend_DB. But be aware that there's a lot more to it than that!

Once you start looking at this sort of library or abstraction it's worth taking more time to look at frameworks and application structure. The MVC model is one key design you should learn, but it's only one of a wide range of Design Patterns which form an advanced vocabulary of development.

Design Patterns form the advanced building blocks of developments, but you should also advance your development practices - your tools, effectively. The techniques you should learn are collectively known as "Pragmatic Programming", and are neatly encapsulated in the book "The Pragmatic Programmer", which you might as well buy and read now, because you'll be a better developer afterwards.

These formalised approaches and skills mark a transition in your approach to creating code and applications, from "mere" development to software engineering. A hobbyist can certainly be a developer, but engineers are skilled professionals and experts. Much as it takes 3-4 years at a University (in the UK at least, more elsewhere) to become an engineer, it can take that long, plus a few years of practice, to truly grasp the skills of software engineering. And, as things stand, I'm personally dubious that university courses exist that give you a broad and modern enough skillset.

Even once you're engineering your software, rather than just building it, there's more to learn. Much as hardware engineers attend conferences and read technical papers, software engineers need to keep studying at a postgraduate level. These topics are widespread and continually being developed and discovered, but include continuous integration, load testing, scalability. You'll also need experience in the specific tools of your particular domain; this can include advanced SQL such as stored procedures & triggers, the full capabilities of your framework, use of advanced javascript toolkits, ajax, DOM & xpath and so on, before you even get to third-party APIs for other sites.

And there are also auxiliary skills that don't directly relate to development; configuration of servers, estimation skills, Scrum development practices, code reviewing; you'll need many of these to work in a team.

So, there's plenty to learn and, as said before, you won't stop learning. But, given some idea of your syllabus, a few pointers to resources, and plenty of interest and support, you can certainly build a very strong skillset in a few years. One of the problems I've felt over the years while learning is that it's very hard to know what I should be studying, which costs you time and direction. Hopefully the above list will be of some use; for more, just keep reading, attend conferences, and consult your local user group.

So... you'd like to get in on the act and start coding PHP? Perhaps you'd like to add a few features to your homepage, or perhaps you're looking for a career shift? How do you get started in the Brave New World of PHP?

The main thing you'll need is a strong desire to learn and to improve. There are thousands of junior developers out there who frankly have poor skills and produce terrible code, and who are unlikely ever to get much better. By just displaying a reasonable amount of determination and drive, you can rise above them and become skilled and highly employable! If, on the other hand, you're not interested in becoming great, please don't start - the shallow end of the pool is full enough already.

There are three key points you'll need to bear in mind:

1) You're aiming at a moving target. The discipline of software engineering, and the PHP language, are both young and evolving.

2) Security is absolutely key to your code! Unlike 20 years (or so) ago, when I started coding in BBC Basic and the code never left the room, your site and the code on it will be out in the wild from day one, and will need to be resistant to numerous security attacks. The good news is that security basics aren't that difficult to learn.

3) You will never stop learning. And it'll (usually) be fun.

There are also three areas you'll need to study in:

Firstly, the grammar and rules of the language you want to learn, for which you can start with a basic book on PHP such as Learning PHP 5 or the slightly more advanced Programming PHP.

Secondly, the tools and techniques of software engineering. Much as you wouldn't build a house without some architectural understanding, or a car without some key ideas in mechanical engineering, your code will be immeasurably improved if you understand the best techniques to use. To a large extent, these techniques are language-independent, and will serve you well whether you end up coding in PHP, C, Java or ay other modern language. My own personal favourite book on this topic has to be The Pragmatic Programmer.

And thirdly, you'll need an understanding of the environment you're working in, both with regards to security (for which I invariably recommend Chris Shiflett's Essential PHP Security) and wider issues such as the platform you'll be running on, and the wider functionality and expectations of the web.

The good news is that it's not all book learning, and you won't be on your own. However, I've presented these books first because the community is also far keener to help someone who'll help themselves and, if you choose to splash out on professional training, it never hurts to get a head start to make the best of your investment.

So, who is out there to help you? Well, tens, possibly hundreds, of thousands of people, arranged into various types of online and real-world communities. If you're in London UK, where I am, you should join PHP London; sign up to the mailing lists and join us at the social meets. If you're elsewhere, take a look at PHP User Groups or MeetUp.com to find a local group.

Does that feel a bit too much like jumping in at the deep end? Don't worry about that. These communities are here to support you at all levels, and as per this recent piece by Chris Cornutt, you're unlikely to become a great developer on your own. We could save you a few years of confusion, and do wonders for your skills and earning power!

If you do want to go for professional coaching, the best place to go in the UK is probably iBuildings, who are Zend's partners for the UK and Netherlands. They can provide a range of training for various skill levels - I and my team took some a few months back and found it very useful. However I should probably disclaim here that I've since become friends to a few of the iBuildings staff, and I will (much as Paul will hate me for admitting this) point out that you can become a good developer without their help! For individuals in particular, the training can be quite an investment, and so self-study can be a good option.

So, where (and how) to self-study? Well, as you're reading this I presume you're already familiar with various blogs and online sources, but I'll point out that PHP London has a collected RSS feed (including this blog) at http://feeds.feedburner.com/PHPLondon, and you'll find many more blogs at Planet PHP. Another good resource is the hardcopy (and recently relaunched) PHP|Architect magazine, which is a must at its new price.

Personally, I find self-study most productive when I have a goal. One I'd particularly recommend would be the Zend PHP5 Certification. This is not a basic certification (for that, look at something like BrainBench), and may take you a year or two to learn enough to pass if you're new to programming, but that's precisely what makes it useful. If you can pass this, you probably know what you're doing. Note that while there is a study guide for the certification, it's not enough to refer to this book alone as it's mainly a syllabus - you'll need to study from other resources to pass.

Well, I figure that that should get you started! As ever, do feel free to comment or drop me a line with any ideas or queries.

Confession: I'm a complete nut for Prog Rock, so being home early on Friday evening and getting to listen to Planet Rock's Fish on Friday is very pleasant (even if the reason I'm at home is that I've just had minor surgery on a toe).

In fact, I'm enjoying many of the tracks enough that I want to look up what they are. This should be easy on a DAB radio, but unfortunately Planet Rock broke their "now playing" DAB Metadata a while back.

So, I thought I'd give their much vaunted new website a visit. Firefox handily remembers, as I type in 'planet' that it used to be at http://www.planetrock.co.uk/default.asp .

Try the link. Uncustomised IIS 404 page. Would a 301 from the old homepage to the new one have been so hard? Or a humorous custom 404? They're all the rage these days...

OK, maybe I'm expecting too much. I tend to find, when it comes to web design or development agencies, that the best thing to do is to set my expectations low, stamp on them a few times, and then throw them out.

So anyway... the site, once I get to it, boasts this "On Air" page:



Where would you expect to be able to click?

If you guessed any of these locations, you'd be completely wrong:



If you're cripplingly literal-minded, and guessed any of these locations, you'd be in luck.



So who on earth designed that? Leaving aside the revolting semantics of making (almost) all links have the text "Click Here", why can't you click on any of those 4 square boxes, particularly "How to listen"? Why can't you click on any titles?

Those "click here" links, by the way, look like <div class="click_here"><a href="rockblock.html"></a></div> - not text, nor even images, let alone images with title or alt text... In fact the nearest thing to alt text or titles is the text 'adholder' for those four boxes.

So, what were the design and usability objectives here? What accessibility standards were they working to?

Frankly, what were they playing at?

And why am I getting so excited about a dud design for a radio station's website?

Because I care about usability.

I care about professionalism.

I care about giving customers, and site users, a good deal and a good experience.

And because I spend far too much time in my day job cleaning up after charlatans who can't code, who know that they're dealing with non-technical clients (most of the time, I tend to give them a nasty shock) who don't know what to look for, ask for, or test.

These guys give my industry a bad name. They provide crap where they could provide quality; they choose "barely good enough" where they could delight. They hold back the development of the web.

I never did find a "now playing" banner. The nearest I got was this delightful stream player:



Doesn't really delight, does it? For some reason you *can* click the Alice image this time, although it then opens the page in the (non-resizable) popup window.

Fortunately I did manage to identify a few of the tracks. I picked up my iPhone, switched to Shazam, and pressed the button. It listened, and told me what the tune was. A usability dream - and I'll be putting 13th Star on my Christmas list!

The basic model of Zend MVC is the use of Modules, Controllers and Actions. Each URI is translated via a set of specified or default routes to call a specific action, which, in the default behaviour selects and populates a view which is then automatically rendered.

This works well for any behaviour which is URI-specific, but doesn't work so well for controller-wide or site-wide behaviour, or behaviour repeated in multiple (but not all) actions.

These are handled in plugins and helpers, but precisely how to use those can be confusing (by which I mean, it took me a while to work out). There are multiple event-triggered functions that can be implemented to run at various times, as well as those which are called according to route, or manually.

The complexity further increases because Plugins, Action helpers and even Controllers can fire on events. Plugins have six possible event triggers, and controllers and helpers can fire on preDispatch and postDispatch.

There's a useful visual guide to those timings at http://surlandia.com/2008/11/03/zend-framework-plugins-action-helpers-and-controllers-life-cycle-during-dispatch/

As for particular use cases:

Plugins are registered with the front controller in the bootstrap file, and fire at any of the 6 triggers for all routes / actions.

Controller events (pre/postDispatch) fire at specific times when their own controller is routed, alongside the appropriate Action functions which tend to perform the bulk of the work.

Action helpers designed as pluggable functionality that can be registered and called as required (usually by Action functions), but can also "prepare" and "clean up" after themselves by means of their triggered functions.

The role of this post is only to clarify (partly for my own benefit) the above options and differences; for more detail, Matthew Weier O'Phinney has some excellent articles at Zend DevZone:
http://devzone.zend.com/article/3372-Front-Controller-Plugins-in-Zend-Framework for Plugins and
http://devzone.zend.com/article/3350-Action-Helpers-in-Zend-Framework for Action Helpers.

Zend Framework's Zend_OpenId_Consumer makes accepting OpenID logins on your site pretty easy, but its companion, Zend_OpenId_Provider is a little harder to get on with.

There's a few reasons for this:

1 - As per the philosophy of Zend Framework, you're given key parts of the tool, but have to fill in a lot yourself.

2 - There's a lot more data to manage and store for a Provider than a Consumer, and the OpenID system deliberately puts the complexity on the Provider side.

3 - The documentation of Zend_OpenId_Provider is, frankly, not too hot. To quote:
"Building OpenID servers is less usual tasks then building OpenID-enabled sites, so this manual don't try to cover all Zend_OpenId_Provider features as it was done for Zend_OpenId_Consumer."

The documentation is, unfortunately, written in somewhat stilted english, and the examples given are over-simplified and not altogether clear.

Now, there's no point simply ranting about this, as someone will quite reasonably request more detail...

So, given that ZF is a community effort, and that I've managed to work out enough of the code to get a basic provider running, here's a quick guide on how it seems to work:

As per the flowchart at http://framework.zend.com/manual/en/zend.openid.html#zend.openid.introduction.how, it's a multi-step process:

First, the user requests a login to the consumer site by provide an OpenID URI.
The consuming server checks the page at that URI for a server delegation metatag, and then contacts the server this identifies (optionally following a further delegation chain).

The providing server must accept a direct HTTP request from the consuming server and call:
$server = new Zend_OpenId_Provider; // Instantiate server with default settings
echo $server->handle(); // By default, look in $_GET for openID data and respond to it.
This can be done as for any other page so long as you only ouput the return value.

Unfortunately, this default behaviour will simply return an OpenID CANCEL command to the consumer, because:
- The Provider will call self::hasUser($id), which in turn calls Zend_OpenId_Provider_Storage->hasUser($id)
- The default storage mechanism has no users until configured (hence the hack in the first example in the docs)
- Failing to locate the user causes Zend_OpenId_Provider to decline the request.

So, we need to provide a form of storage that will respond 'true' for an ID we should be able to serve.

The very simplest way to do this is to subclass Zend_OpenId_Provider_Storage_File and pass that in to the provider class's constructor (it accepts any object inheriting from Zend_OpenId_Provider_Storage as a 4th parameter, but uses the file variant if none is provided).

The only function you'll need to override at this time is Zend_OpenId_Provider_Storage::hasUser($id). My personal quick hack was:

public function hasUser($id){
return ($id==='http://myuser.openid.server.com/');
}

Obviously for production you'll want to do something a bit smarter, but for now it's enough to give you the idea.
You could also try to use somthing like the documentation hack to register an id in the file storage, but I'd rather make ZF work my way.

So, if the ID is known (even if it has no relation to the current provider site visitor), the Provider will associatte with the Consumer in order to be able to confirm the identity securely. This association is stored by Zend_OpenId_Provider_Storage::addAssociation(), but for now I'm letting the Zend_OpenId_Provider_Storage_File handle this.

Once it's associated, the Provider will (I believe) see if it's aware of the owner of that identity being logged into itself (via a prior call to Zend_OpenId_Provider::login()). If so, it will move on to confirming that the user trusts the consuming site (ie, is happy to confirm their identity to it).
If the user is not logged in to the provider (NOT the same as being logged into the providing SITE), the user's browser will be directed to the provider's login page, as specified as the first parameter in the provider's constructor. Note that if the login URI is not provided to the constructor, a default URI is used as per the documentation.

This login page should verify the user's identity by whatever means the providing site already uses, then call a redirect to the trust page (the second constructor parameter). Note however that Zend_OpenId_Provider is written to expect that you log into itself directly (by username and password) and is written in a way that makes it hard to write the Zend_OpenId_Provider_Storage::checkUser function that needs to validate the username and password. Unusually for Zend Framework code, the encapsulation here is too weak, and developers will often need to replace or sidestep this particular bit of functionality.

Once the trust page is reached, the provider needs to verify that the user is happy for the consumer to know their identity. There are various options here; if you use the full built-in abilities of the provider, you may have previously told it "This site is always OK" or "This site is never OK". It should use that data (as stored by the Storage, IIRC) at this point if available, otherwise it will display the Trust page.

Note - there's a *lot* going on under the hood of the provider, and without testing every route I'm not exactly sure of program flow! For this guide, I'm covering the basic operation, and the principles, but for trust storage you'll have to experiment for now. It seems likely that if the user is already logged in to the Provider, and has designated the consuming site as "Trust Always", then Zend_OpenId_Provider will not try to display either the login or trust pages.

Finally, whatever method you use to verify trust, if you the user agrees to it, you then need to call
$server->respondToConsumer($_GET);
which will be the Provider's last act, confirming your identity and returning you to the consuming site.


To summarise the code required (we'll assume an MVC environment; this code isn't tested as-is):



class OpenIdServerController extends Zend_Controller_Action {
function init()
{
$this->_provider = new Zend_OpenId_Provider(
'/openid/server/login', // login path
'/openid/server/trust', // trust path
null, // default user session manager
new My_OpenId_Provider_Storage() // our tweaked storage class
);
}

function serverAction() //This is called first, by the consuming server
{
echo $this->_provider->handle(); // handle 'GET' params by default
exit; // Don't render views, etc
}

function loginAction() // The user is directed to this page if they're not logged in to Zend_OpenId_Provider
{
if(/* we can verify this user */) {
//optionally call $this->_provider->login() to maintain user identity on providing server
Zend_OpenId::redirect("/openid/server/trust", $_GET);
} else {
// Display login form and post back to this page
}
}

function trustAction() // The user is sent to this page after being logged in
{
if(/* we have permission to trust this consumer */) {
$this->_provider->respondToConsumer($_GET);
} else {
// Display a form asking whether to trust this consumer; post back to this page
}
}
}

Refer to the ZF manual for suggested forms.

Our hacked My_OpenId_Provider_Storage just looks like:
class My_OpenId_Provider_Storage extends Zend_OpenId_Provider_Storage_File
{
public function hasUser($id){
return ($id==='http://myuser.openid.server.com/');
}
}


NOTE that this process basically expects the user to log in, or be logged in, and to re-verify trust each time. Using the "Provider Sessions" is beyond the scope of this guide.

One of my projects at the moment is creating a small web app that interfaces to the Spend iPhone Application by Adamcode. The app's a 59p bargain which simply lets you assign expenses to budgets while on the move, and so keep an eye on how fast it's flowing out. I'm using it to try and improve the amount of money I can pay into my holiday and savings funds.

The app itself is nicely designed; the key thing is that it's simple, usable and uncluttered. In which vein, a picture paints a thousand words, so here's the interface:



To assign an expense to a budget, you simply tap the budget and add an amount and description. The main interface then shows the amount you've spent or have left; budgets within range are green, those overspent are red. Budgets can refresh at various frequencies and optionally roll over.

This is great; it gives a great "at a glance" view, but not much more. It doesn't give me historical data, trends, charts, or summaries of how well I generally adhere to my budgets.

And, frankly, I really don't want it to. It'd clutter up the app, and would most likely be hard pressed to display enough controls or data to be useful within the confines of the iPhone interface. What Spend does, it does well.

What it doesn't do, I can do on my desktop.

The iPhone's data export abilities are incredibly limited, but Spend lets you drop CSV data to an email and send that to yourself. The core aim is that you can then copy that into a .csv file and load that into a spreadsheet. It's a good solution to the iPhone's limitations, but trying to manage sequential updates this way is bound to get trying; it's slow and too manual, and stitching together sequential and overlapping data and deletions is awkward.

So, what's a web software engineer to do? Well, write a website obviously. When the only tool you have is a hammer...

Seriously, though, the web solution's a good one. Sites can easily accept data input by email, and PHP's great for the data management. Add SVG charts and we have all we need.

So, the plan (version 1) is:

- Mail data to a robot address
- Create user accounts for any new source email addresses.
- Notify users by email.
- Stick data in a database
- Analyse data in various formats
- Spend less money
- Profit!

Since it's almost as easy to write a web app for everyone as for one person (if you code securely and sanely), I figured I'd plan for wide usage. It's an interesting challenge and also a chance to do something useful.

The format of the incoming mail though provides some challenges:

- Only expenses are exported, not the budgets they're accrued against.
- There's a slight data bug with regards to commas in text not being properly escaped in CSV.
- Dates are exported according (as far as I can tell) to the individual iPhone's locale settings, and there's no way of telling whether 6/7/2008 is an American iPhone sending data from June 7th, or a UK one sending it from July 6th. Assume the wrong one and your data's a mess.

There are various approaches that can be used to deal with these:
- Manual budget entry into the web app (budgets change slowly, expenses are several-per-day)
- Potentially merge any fields displaying too many commas
- Code for UK dates first and worry later
- Hope for import periods of at least 2 weeks, which would guarantee at least one date that'd be invalid if days and months were reversed.
- Do other heuristics on import data to check order.

It's worth noting that I'm coding this test-driven, so I'll have strong regression tests, a modular design, and the ability to improve and optimise as I go. However, I'd still rather not working on code that guesses what it's doing.

So, to keep it simple, I'm handling UK dates for now, but I also got in touch with the developer of the app to raise the actual bugs, suggest Budget export and let him know my plans. Adam was polite and responsive, and took the suggestion on board even before I started coding.

This week I've got some time off work and have started coding. The first part needed was the input mail parser, which:

- Reads in the whole email as text.
- Identifies the sender.
- Identifies the time range covered.
- Collects up and parses the CSV rows

It doesn't try and store this data, or catch the incoming emails; instead it provides the parsing function that another module can use to do so. That encapsulation makes testing easier and also works well with the brief periods of time I have available to code.

It also lets me test my assumptions as I code, because every class and function I code can be run almost immediately.

Some of these early assumptions included:

- Users will send emails of various expenditures at various times.
- These emails will overlap in the time they cover.
- Users may correct or delete items they've already sent.

This means I need to be able to uniquely identify transactions that may change name, amount and budget. The logical solution is therefore to use the timestamp (per user) as the UID - after all, you can't create two transactions at the same moment.

Well, early development points out that you actually can. Timestamps are only granular to the minute, and if you go to three shops and then update Spend after lunch, the very slick interface means you can enter at least two of them within the same minute.

Oops. No UID. And since the email export's main purpose is human usability, and I'm only one user of many, it doesn't seem very sensible (or polite) to ask Adam to recode the export (although a generic "spreadsheet optimised" output may have its uses, I'd rather not need it if I can avoid it). That said, the iPhone's update mechanism does mean that users can generally get the latest version of the app pretty easily.

But does it matter if we don't have an exact, causal UID? Possibly not. If we get two synchronous transactions we can increment one by a second. The update mechanism means we get all transactions for a minute or none of them, so we can handle time scoping and updates even if we shuffle which means which - so long as we're not trying to attach unique data such as comments. For that, we'll need a guaranteed UID.

The mechanism for handling incoming emails is then as follows:

- Identify the time range a new import covers
- Identify the sender of a new import
- We need to store all transactions which don't already exist for this time/account pair
- Delete any between this start and end date that don't exist in this import
- Update any already in the DB

So far, that all works. The main problem so far is the lack of specificity in dates; for that, export metadata may be needed, or a transition to YYYY/MM/DD format, which is unambiguous.

The iPhone's location function return data in the form "59.12345,-0.122344". Unfortunately if you pass this data to the iPhone maps application, it drops you in the sea somewhere off Ghana.

The solution? Well, while we wait for Apple (or Google?) to fix their parser, there's a trivial one. Rewrite the location string in the format "59.12345N,0.12234W" and you'll be perfectly positioned.

That took all of 5 minutes to work out, so it's a great pity that the developers of apps such as Cardinal and Geohash didn't take the time for proper on-device testing.

Back when I was an RAF cadet, we had to study Aircraft Recognition, presumably on the premise that we'd one day have to know which ones to shoot down.

In Eve, it's often vital to recognise ship types and threats instantly from their names, so I'm working on a flashcards app. This can now be found at http://www.phase.org/Eve/showRandomShip/view/flashcard (or jog your memory at http://www.phase.org/Eve/showRandomShip/ first).

So far it's a click-to-view toy with no scoring - that will follow shortly.

Questions - should this include Rookie ships? Polaris? Jovian? There are 124 "special" ships in the DB which I could filter out. (NB: The Orca's already in there, but with the Rorqual's description).

For some time now I've been playing the MMOG "Eve Online"; a game of epic space battles and warring factions in the far future (and so on). Think "Massively Multiplayer Elite with Babylon 5-style graphics".

Eve is famous for various key ways it differs from most other MMOGs:

1) It's unsharded. All players (0.25 million at the last count) play within one server cluster, and any player can (subject to not getting blow up in transit) travel to, and play with or against, any other player.

2) It's extremely free-form. you can choose to mine, build, invent, explore, run missions, fight solo, run megacorps, carry freight - there's a vast in-game economy - and change roles as rapidly as you can train new skills.

3) The learning curve is incredible (although there are no shortage of newbie-safe areas); it can literally take years (of real time, not in-game time) to qualify to fly the largest ships, and months for some of fairly common intermediate ones. The skill tree is colossal.

In other words, it's a highly complex game that occasionally causes you to reach for a spreadsheet (or dedicated app) to work out what to buy, how to equip a ship, or what to study. Fortunately, there are very good tools - and, even better (for my purposes), there's a good supply of data through an API and periodic DB exports.

The DB and API (and a certain degree of transparency in sharing key formulae) allow the use of skill planning apps such as EveMon and ship fitting tools such as the Eve Fitting Tool. They also allow very simple tools as Skill Monitors, and more complex tools that let you search and analyse all your in-game assets.

At the moment, there are some very good tools for Windows, a few less-great ones for the Mac, a decent range on the web, and no iPhone apps. So, I'm looking to fill that gap.

I'm not a Mac coder; I've never written a line of Objective C or cocoa; I'm a PHP developer. But that also means I've a decent object-oriented programmer, and I've got a reasonable C/C++ background. So, I'm initially experimenting with the (XML/REST) API in PHP, while I learn enough about it and Objective C to move to the desktop or the phone.

So far, I've got a crude skills monitor (was there a shortage?), and, more interestingly, a complete asset viewer that also lets you see complete fittings and cargo of all your ships. These tools are currently web-only (hosted at phase.org) and currently open to a very small closed alpha test. However, they will grow, so if you're interested in access or have suggestions, comment here. Priority for alpha/beta access will go to members of my Corporation (Ominous) and Alliance (Ethereal Dawn).

(For what it's worth, the other thing I'm looking to write for the iPhone will be a decent LiveJournal client).

to see if Zend Studio crashes again.

I used to be fairly happy with Zend Studio, but these days it seems to be featuring as one of the banes of my working life. As a developer it should be the program I spend most of my time using (and so needs to Just Work), and indeed on my own system at home (smallish codebase, local files & webserver) it does. Usually.

In the office, however, we have a much bigger codebase, shared mounts for sandboxes (as per the ongoing series on this blog), and an occasionally slow or patchy network. This gives Zend Studio (both 5.5.x and 6.0.x) a myriad of ways and reasons to fall over or just seize up. To be honest, I'm not sure Java's a great language for an app of this complexity, but for whatever reason it will frequently just stop dead, fail to open files, or just ignore all input.

So, to work out how to get around this and actually be able to edit my files (with code completion, debugging, code lookup and all those things which make vim Not An Option), I've been trying to debug my way around the problem.

Firstly, I've been trying to work out if it really is the network that's the issue. So, remove that from the equation: read the files from a samba mount with no network latency; in other words, from a local samba server.

As I've recently received a higher-spec machine at work, with parallels supplied, my first step was to create a small internal ubuntu server with a samba share to see what happened.

What happened was that I discovered that ubutu 8.10 server doesn't run under parallels, although it installs fine.

So, I switched to the Ubuntu JeOS distro instead, which does run. A few apt-get's later and I have a local samba share. I check out our codebase, point a new project at it in Studio 5.5.1, open a file deep in the class tree, and do some other work for a while.

Result: ZDE seizes up and can't do code clickthroughs or open files. Great. So it's not the network. (I'm working on the basis that it's not my new intel 8-core mac, either). Maybe the virtual machine's too slow? Check out to local drive, perform same test, same fail. Ok, so ZDE 5.5 needs to be dragged out into a quiet alley and shot. I've suspected that for a while.

So, chuck out 5.5; try 6.0.1 (which I've tended to avoid as the interface, menu and naming is completely confusing, and it still falls down). And it turns out that I can load, edit and clickthrough a local project if I use a local folder as a project (acheived, uninituitively enough, by starting a new PHP project, and overriding first the file location and then the verification dialogue (and finding that in the manual was fun)).

[As an impromptu aside; ZDE 6 is modal. Very modal. It loves nothing more than to throw up an "information" dialogue that prevents you from editing, saving, or quitting. Except possibly for jumping to the foreground when you're working in another app and it wants attention.]

So, yeah... I can edit locally. Unfortunately, my dev server is my local box (yes, I could set it up that way for myself, but the live server is Linux not OSX, and I'm not configuring local servers for the entire team) and I need to save my files to the dev box. The good news here is that ZDE has a myriad of Remote Systems support; the bad news is that they're utterly baffling. FTP only is easy enough to understand, but then there's 'Linux' (which, last time I looked, was not a network protocol), 'Windows' and Unix (ditto), Local (which didn't sound very remote), SSH only (which turns out to mean SFTP), and Telnet Only (Experimental) which just sounds like bad news.

'Linux', btw, turns out to mean some weird perl/java hybrid server that you have to install on a linux server. Personally, I'd rather steer clear...

Having decided that SFTP looked the best bet, I thought I'd give that a go. Fairly easy (despite the interface's best efforts to confuse matters); set up a server, browse, select a folder and right-click to Create Remote Project. Wait while it refreshes, wait a bit longer while the progress bad gets stuck near the end, and wait while runs out of Java Heap space and crashes with an "Insufficient Memory. This advises you to refer to the "Running Eclipse" section of the readme.txt file; a quick glance in there shows that Zend have removed that section, and indeed most of the rest of the file. Might be nice if they updated their code to reflect this?

A quick google later and I find this page, which tells me to burrow into the depths of ZendStudio.app and edit 'eclipse.ini'.

Of course, that wouldn't do for Zend; they had to rename the file ZendStudio.ini. So, edit that instead.

Setting max memory up from 256M to 1024M crashes ZendStudio at startup. Setting it to 512M or 768M means it gets stuck even longer at the end of the "Create Remote Project" progress bar before crashing.

So, ok, abandon SFTP for now - let's try setting it up as a local PHP project on the Samba share. Set name, set directory, hit "finish", and it hangs - fortunately, only briefly. We can now open and edit files... but as yet, no code clickthroughs; the "processing" bar's spinning at the bottom of Eclipse and it's "building PHP Project" and "building workspace". Experience suggests it may do this forever, so I'm going for a cup of tea.


As suspected, it's got stuck at 4% in the depths of the Zend Framework.


Now what to try?

Erm, force-quit Zend Studio, restart it, and watch it all miraculously work, apparently? Ok, that's a complete anti-climax; maybe the extra memory fixed it. I'll update if it fails again.

OK, so this site goes 'bang' occasionally. Not ideal for a site that purports to advise on coding and project management techniques, so what's wrong?

Well, part of the issue is that I wouldn't recommend that anyone else code a site in the way that this one's actually being maintained. This code's the nearest I have to a playground, and so sometimes it falls off the swings. A lot of the code's unit tested (and those tests work), but there's still some weird race conditions and sporadic failures.

Why?

Because there are bits of this codebase that date back about 4 years. When I started, PHP4 was young (and I was, well, younger) and I knew nothing of object code, so the first 'PhaseCode' release was purely procedural. No, you can't see that codebase, but if you drop back into the archives you can have a good laugh at my youthful naivety in some of the design notes. These days you can still laugh, but I'm losing that particular excuse.

Then at some point (I can't work it out from the archive offhand) I started wrapping the code up in classes. These classes got slung together in bunches of files that were loosely grouped by "module", and ended up as something of a require() maze.

So, I decided to put away such childish foolishness (I have a feeling this was about the start of last year) and move to a class structure I could __autoload()! Wow!

Except that I did it not in the Zend format of Some/Class/Here.php, but in an older pattern of phasetwo_className_Class.php in a flat structure, which was fairly common around then - I think Smarty still uses it. This was a step forward, but it was a rather different forward from the one I'm now heading in.

Now, I'm using Zend's Some/Class/Here.php (since about this time last year, I think), which is based on a fairly shameful kludge (or, depending on viewpoint, a quick and pragmatic solution to manage a codebase for which no-one else is paying, and of which no-one else has to suffer the pain of maintenance - my pro codebases are stunning ;). That hack is as follows:

• Classes starting with phasetwo_ are autoload()ed from lib/phasetwo_someClass_Name_Class.php
• Classes starting with PhaseTwo_ are autoload()ed from lib/PhaseTwo/SomeClass/Name.php

Yeah, I can hear you wincing from here. Now you know my shame; my autoloader switches by case. The real fun comes as I move files from one location to the other; as I do so I have to change the case of every mention of that class in the rest of the code; sometimes bits escape, but the code works if the correct case was mentioned first, because while my autoloader is case sensitive, PHP's class naming isn't. (Yes, I heard that wince too). This provides no end of fun opportunities for hard-to-find bugs and race conditions, some of which make no sense even after I fix them. Failures inside callbacks (including the ancient template parser I still use) are even harder to find as the PHP interpreter tends to fail without either error or exception.

But hey, it's all fun ;) And now you know why I keep blowing my site up - because I'm not allowed to do it to anyone else's.

(But, seriously folks - don't try this at home ;)

phase.org is now running (in part at least) on the Zend Framework, v 1.5, with a lightweight front controller.

As far as I know, this makes absolutely no difference to the end user, but I figured it was worth mentioning.

So, I've fixed the RSS feed on phase.org, taking the chance to play with some PHP DOM code. The feed's actually built up as DOM operations - probably overkill for RSS, but an interesting exercise, and it should mean I can't somehow create a badly-formed feed. Although, as the post itself is encoded as CDATA, I'm not quite sure what'd happen if I tried to include the CDATA escape sequence. No, I'm not gonna try it now, it's late.

Why fix the feed now? Because Nigel James has come up with the great idea of aggregating the blog feeds from PHP-London members, and he was going to include mine 'til I broke the feed. Plus, my imagination has just been fired up by recently attending FOWA, *and* I've just got a new job, *and* I've got two weeks off, *and* I've been promising for far too long that I'd actually start blogging again. I've also been meaning to put some social and political content on here too (much as many other tech bloggers do) and Ming Campbell's resignation tonight provides as good a start point as any. Oh, and the revelation that people think I might actually be saying stuff worth following is useful too...

As you may have noticed from the flickr link in the previous post, I'm a fairly keen amateur photographer. I'm also a member of the London Photographic Meetup Group, and we're holding our first exhibition next week. Not only do I have three photos in the exhibition, I also wrote the exhibition website, incorporating all the new work for that site into the phasetwo codebase. So, there's a few changes around here, still mainly in the background, but all work that builds towards interesting future developments.

You may have noticed that phase.org is suddenly looking a bit bare. The reason for this is that I'm currently going through a code quality exercise, slowly porting the material from phaseCode into a newer framework. This will mean that I have a set of properly-encapsuated, autoloadable and properly tested code to which I can rapidly add new functionality.

So far I've put in just enough to display a journal (and a couple of other bits of back-end functionality which will be released later), and some of the features that were on phase.org will be ported over in time. However, this time I'm focussing less on "interesting features" than on stuff that I'll actually use - for systems where I have a perfectly good system available to me elsewhere, I may not actually bother with the porting.

For example, I've not put any new gallery software on this site. flickr does that for me fairly well, so what you'll see here are a few API tools (or the outputs thereof) that make flickr more useful for me. For example, I recently added a GPX-to-flickr app in the backend, so my photos can now be geotagged automatically if I have a suitable GPX file on my laptop.

Some of those can be seen on my flickr map.

Like many PHP developers, I'm pretty much self-taught, having started as a hobbyist. Commonly enough, I had a specific need - I wanted to write a cycling advocacy site, and PHP was an easy way to get started - after all, it was just a matter of adding a bit of code into a webpage, and I already knew HTML...

And that's one of PHP's main problems. It's too easy; anyone can write it, without knowing more than the first thing about code. People write low quality or insecure code in the language, so it gets called low quality and insecure itself.

It isn't. Previous versions had their problems, certainly, but PHP5.1 - while not perfect - is an extremely powerful, usable and versatile language. You can write extremely high quality applications and websites in it and manage code testing and documentation very powerfully and effectively.

The problem is that not enough people know how. And perhaps more critically, many of them don't know that they don't know. I don't know what the current state of software engineering teaching is like in universities, but in my day (excuse me if I sound old) we did some C that we knew would run in a trusted, single-machine, secure environment. This was around the time when someone had the bright idea of creating a "common gateway interface" that would let people put back-end applications on those new-fangled web servers! Revolutionary stuff indeed, and long before the days when people even thought of the security requirements of globally exposed, network-connected software.

Back in those junior days there was a youthful innocence - an extreme naivety in fact - as to the vulnerability of code. It wasn't widely realised that servers could be compromised via web apps, and so no-one coded in security. (On a slightly tangential note, the servers hadn't really got the concept of dropping privileges either, and we scared seven shades out of our sysadmin one day when we SSI'd /etc/passwd from a user account).

Unfortunately, many coders - particularly junior ones, although the flaw often seems to survive years of "experience" - still seem to live in those innocent times. When interviewing applicants a few months back I found far too many who didn't know what injection attacks, XSS or CSRF were, and couldn't spot them in example code. And it's precisely that lack of knowledge that's dangerous.

It's not just in security issues that knowledge is lacking, either. Design patterns are still a black art to the majority of coders, and version control and unit testing considered eccentric luxuries. Most revealingly, while everyone knows that OOP is good, few people seem to understand quite why they're using it, or just where its strengths lie.

It must be acknowledged that a lot of coders do seek to learn more, but it can be an uncannily difficult task. There's no shortage of instructional books and websites available on coding, writing websites, and so on, but their availability is massively skewed towards beginners. Even finding intermediate material can be challenging, and finding genuinely advanced material can seem like a needle-in-a-hayfield task. At least, until you find some experts, who can often be almost as well hidden.

This invisibility of good material means that a lot of mid-level coders don't really appreciate how much there is to be learned. Most of the books available tend to be references, and while there are tomes on "software engineering", this is too often considered to be the preserve of desktop applications, million-user sites, or "heavier" languages. It's not. PHP benefits from skilled software engineering and careful process as much as any language, and the techniques that make million-user sites more secure, and their codebases easier to extend, apply just as well to the company intranet or the hobbyist's project (which, these days, could very easily become the Next Big Thing).

So where to look? Schlossnagle's Advanced PHP programming is a fine example of the rare breed of advanced PHP books. But you don't by any stretch need language-specific books, or even, in many cases, particularly modern material. One of the best books on coding I've ever read - and I *wish* I could recall the title - was in its 25th anniversary edition when I read it about 4 years back. "No silver bullet" is 20 years old but at least as valid now as when it was written - and it cites sources going back 10 years. The Gang of Four's Design Patterns is 11 years old and still a primary source.

But you don't have to raid the archives to find good information. PHP bulletin boards and IRC channels may be strongly weighted towards beginners, but if you have a local PHP group (such as PHP-London) you'll find it well populated with skilled developers, many of them the authors of significant open source projects, technical books or articles in such magazines as PHP|architect (a great magazine, but don't bother with hardcopy if you're outside the US). As for advanced websites, Joel on Software is an excellent example.

The most critical aspect, however, is the attitude. We all pass though a phase when we believe that to keep up in this industry we need to continuously learn and study. Then, as with any activity - from photography to flying aircraft - there's a plateau (colourfully called the "death plateau" in the latter case) where we think we know enough. There's a risk of self-congratulation, and we lose momentum. And then - hopefully without having our site hacked or catching hill disease - we realise that there's still a lot more to learn.

And that's the key point of this piece. I've "taken a rest" a bit from studying coding in the past, and been ignorant of the use and even the existence of advanced knowledge. Hopefully in writing this I can push a few of you forward past this plateau, and save you a few hacked websites, a few failed job applications, and a whole load of legacy code headaches.

I'll write more later on various aspects of software development skills and processes, but for now I'll leave you with a small reading list, to add to those items linked above:

• The Pragmatic Programmer
• The Practice of Programming
• Php|architect's Guide to PHP Security

I don't usually cross-post blog articles from other journals to this one, but this one struck me as extremely useful; much like Chris' book I mentioned previously, this is something every PHP developer should read.

http://www.killersoft.com/randomstrings/2006/02/16/walk-the-walk-before-talking-the-talk/

The following was emailed as a response to a user query about the techniques mentioned at
http://phase.org/blog/2640_blog_archive_part_two_a_better_url


The first thing to know is that any PHP script can be called with a URL that continues after the script filename,
eg: (these URLs don't work, they're just examples)
http://www.phase.org/journal.php can also be called with the URLs
http://www.phase.org/journal.php/journalname/stuff/things/etc
http://www.phase.org/journal.php/?id=wechsler

As far as the webserver is concerned, these are all simply requests for journal.php with 'some stuff' added on the end.

When you pass arguments to a PHP script with a query string (?key=value&key2=value2 etc), you can (as I'm sure you know) read these as $_REQUEST['key1'] etc. (You could also read them as $key1 etc, but this method is dangerous).

When you extend the URL with slashes, you can read the extra parts as $_SERVER['PATH_INFO'] (at least on Apache, I've not tested on IIS). This will give you one long string, ie:
$_SERVER['PATH_INFO']='journalname/stuff/things/etc';
You can then split this apart with
$pathparts=explode('/',$_SERVER['PATH_INFO']);
and then read $pathparts as an array - use print_r to examine it.

This now lets you avoid query strings, but you've still got a URL with ".php" in it. To get rid of this, you need to rename the script (eg to just 'journal'), but you also need to tell the webserver to treat that script as a PHP file (because it won't usually know, if the extension is missing). How to do this can vary, and you might not be able to do it if you don't have full control over your webserver.

I use the Apache directive "DefaultType application/x-httpd-php" in a .htaccess file in the directory that contains the renamed scripts.

However, even if you can't do this, you can still use directory-style access, just keep .php in the script name.

Well, that last-but-one post proved to be rather more prophetic than I'd intended. Rest assured that, like 99.999% of London's population, I survived July without getting blown up. In fact, it's been another case of "too busy doing to document":

I've launched another site off this codebase: http://www.londongamelist.org/

Calendar: there's a graphical calendar view in place now, which can synchronise from iCal (and thus, indirectly, from most PDAs).

Filestore: Improvements to the upload system - in paticular fixes to handle the fact that MSIE uses some very strange mimetype declarations on file upload (image/x-png and image/pjpeg in place of image/png and image/jpeg, for example) and the ability to set icons manually (as they can't generate for Really Big images).

PHP code quality: I've turned on E_NOTICE and E_STRICT error levels on several sites using this codebase, which means I've had a fairly major task of updating code to the strict PHP5 way of doing things, and made sure all variables are pre-declared. A long task, but it should cut down on future debugging. (there are probably a few stray notices floating around this site - do let me know!)

AJAX: Many ajax tutorials advise you to use text rather than XML as your ajax page format, but also advise that you use the line
http_request.overrideMimeType('text/xml'); to force the behaviour of your XMLHttpRequest object
The new Firefox 1.5 betas throw errors on this in the (now misnamed) JavaScript Console (it now serves as a fully-fledged error console) which I've explained in updates to this page on DevMo:
http://developer.mozilla.org/en/docs/AJAX:Getting_Started
These errors can be pretty tricky to debug, particularly when they appear as JS errors...

XSS: Nasty stuff. Cross-site scripting occurs when you accidentally let a user manipulate the content of your website, either by submitted content or URL manipulation. This codebase caught most of those tricks, but at a recent talk by Rasmus Lerdorf at PHP London I discovered one that I'd completely missed: PHP_SELF. Needless to say I've done quite a few bugfixes on that one now!

MySociety: I've recently joined the MySociety group (http://www.mysociety.org/), and submitted a few code fragments - this is probably going to be a focus of my coding efforts from here, although I'll still be working on this site's codebase, particularly if there's user interest. For a start, there's a couple of external libraries that I can now jettison, due to PHP5 improvements.

I've also been to Linux World Expo, looked at picking up some LPI and MySQL certs, bought an iPod Nano (yes, they *do* scratch far too easily, Apple!) and started listening to Podcasts. Busy life.