10/9/10

Eager boolean operators in JavaScript

In most programming languages like Java, JavaScript, C and C#, the boolean operators && and || perform short-circuited evaluation. Essentially, this means that a program, when evaluating boolean operators, will only evaluate as many arguments as is necessary to determine the value of a boolean expression. Sometimes, however, this is not the behaviour we want.

In this article we will look at how to implement non-short-circuited or “eager” versions of the JavaScript boolean operators && and ||.

As mentioned earlier, most programming languages have short-circuited boolean operators. Short-circuited operators are possible due to certain properties of conjunctions and disjunctions in Boolean logic. In Boolean logic, for a conjunction x1 && x2, if the first argument evaluates to false, the whole expression will be false, regardless of what the other argument evaluates to. Similarly, for a disjunction x1 || x2, if the first argument evaluates to true, the whole expression will be true, regardless of what the other argument evaluates to.

For a more concrete example, consider this snippet of JavaScript:
var str = null;
if (str !== null && str.length > 0) {
alert("String is not null or empty");
} else {
alert("String is null or empty");
}


If executed by a browser, this snippet will alert String is null or empty. It does not throw a TypeError when it hits str.length because str.length is never evaluated. The JavaScript interpreter skips it once it sees str !== null evaluates to false. This is an advantage of short-circuiting.
What if we don’t want to short the circuit?

Although short-circuiting is almost always what we want, there are rare instances where we may want to have non-short-circuited or eager evaluation. Consider this example using jQuery:
var validate = function(element) {
if (element.val() === "") {
element.addClass("invalid");
return true;
} else {
element.removeClass("highlight");
return false;
}
};

var form = $("#comments-form");
var name = form.find("#name");
var email = form.find("#email");
var comments = form.find("#comments");

form.submit(function() {
return validate(name) && validate(email) && validate(comments);
});

In this example, we are validating a comments form for a website. In order to do that, we use a validate function to verify that an HTML input is not empty. If it is empty, then we add a CSS class to highlight it and then return false to indicate validation failed. Conversely, if it’s not empty, we remove the highlight CSS class and then return true to indicate validation succeeded.

Unfortunately, this code will only highlight the first field that fails validation. Why? Because the && short-circuits the first time it evaluates an argument to false and thus the rest of the validations will not be executed.

So how do we force them run? Well, if we were using a language like Java or C#, we would simply use the & and | operators, which are eager versions of the short-circuited && and || operators. Unfortunately, although JavaScript does provide the & and | operators, they return numeric (0 or 1) instead of boolean (true or false) results.

Eager boolean operators in JavaScript
(Aside: In the following section I use the arithmetic operators * and + to emulate eager boolean operators. You can just as easily use the bitwise operators & and | in their place. The only reason I used the arithmetic operators was because I thought they would make for a more interesting article).

So we’ve established that JavaScript has no eager boolean operators. JavaScript does, however, offer us two arithmetic operators that can almost accomplish our goal: * and +. These operators, in conjunction with JavaScript’s type coercion semantics, produce the following truth tables when fed all combinations of true and false (&& and || are shown for comparison):

In the table you’ll notice that * and + operators yield almost identical truth tables to the && and || operators, with the exception being that the short-circuited operators produce boolean values (true and false) while the arithmetic operators produce integer values (0, 1 or 2). Moreover, unlike the boolean operators, the arithmetic operators are eager.

That means that, in order to use the arithmetic operators to implement eager boolean operators that produce the exact same results as the built-in boolean operators, we need to map 0 to false and map any non-0 values to true. Fortunately, this is exactly what JavaScript’s type coercion does when forced to coerce a number into a boolean, which can be accomplished by prefixing the number with two ! operators.

And there we have it. We’ve implemented eager boolean operators using JavaScript’s type coercion, the arithmetic operators * and + and the logical not operator !.

Conclusion
Boolean operators in most programming languages are short-circuited: they evaluate their terms left to right and stop (i.e. short-circuit) once they can definitively determine the truth value of expression. This behaviour is usually what we want. However, in some rare cases, we want to evaluate all terms in a boolean expression. Such an operator is called an eager boolean operator.

Although JavaScript does not provide eager boolean operators, they can be emulated by using a combination of JavaScript’s type coercion semantics and arithmetic and logical operators. Sphere: Related Content

9 fine Python development tools

A wide-ranging flock of Python IDEs offer great options for Windows scripting, GUI applications, Web frameworks, multilanguage development, and more

Python has certainly distinguished itself as a go-anywhere, do-anything language. As a language for desktop application development, it can be found behind the Resolver One spreadsheet. As an embedded script language, it's inside the highly respected Blender graphics package and the Rhythmbox media player. It is the binding glue of the Sage open source math package and the Portage package management system of Gentoo Linux. On the Web, it powers the highly popular Zope, TurboGears, and Django frameworks. The list goes on, as any quick Google search will show.

Object-oriented and dynamic, Python encourages rapid, iterative, and almost exploratory development. But good Python development starts with a good Python IDE. In this roundup, I examine nine Python development environments, many open source, but some commercial. They are Boa Constructor, Eric, ActiveState's Komodo, Oracle's NetBeans, Aptana's Pydev, PyScripter, SPE, Spyder, and WingWare's Wing IDE.

[ Check out the top open source development tools in Bossie Awards 2010: The best open source application development software. | Keep up with app dev issues and trends with InfoWorld's Fatal Exception blog and Developer World newsletter. ]

Of course, there's also IDLE, the IDE supplied with Python itself. Written using the Tkinter GUI toolkit, IDLE opens instantly into an interactive console. It has no notion of a project, nor is there an included GUI builder. Its configuration is minimal, dealing mainly with fonts, colors, and shortcut keys.

Its debugger is bare bones too, but it gets the job done. Activate the debugger, and when you run an application it starts up immediately. Buttons let you single-step, step over, or step out; also, you can set breakpoints by right-clicking the line in the text editor. While debugging, IDLE provides a view of the current stack and locals. You can activate views for source and globals; from the source view, IDLE will highlight the line that is the current execution point. IDLE is a no-nonsense IDE for editing and debugging Python modules and files as single entities. Many Python developers need more.

This flock of alternatives makes clear that there is no single best Python IDE; the range of problems that these IDEs attempt to solve is too wide to permit a solitary, superior candidate. If you need an IDE that allows development in multiple languages, then Komodo is probably your best bet, though Pydev is a close second and will likely be as good once its documentation holes are filled. If you need an IDE for GUI development, then Eric and Boa Constructor are equally good.

Be prepared, however, to work out development details on your own; both Eric and Boa Constructor beg for additional documentation. For building single-module Python scripts, PyScripter is excellent, though limited to Windows. Linux and Mac users should check out SPE. And finally, if you need an analytics environment for working with the NumPy and SciPy math libraries, Spyder is the place to go.

Happily, many of these products are free open source, and those that are not offer multiweek trial periods. Your best bet is to choose one that appears the closest fit to your particular requirements, and then turn yourself loose on it.

Nine fine Python development tools: Boa Constructor 7.7 Good
Nine fine Python development tools: Eric 7.2 Good
Nine fine Python development tools: Komodo 8.6 VERY Good
Nine fine Python development tools: NetBeans 7.8 Good
Nine fine Python development tools: Pydev 8.2 Very Good
Nine fine Python development tools: PyScripter 8.3 Very Good
Nine fine Python development tools: SPE 8.0 Very Good
Nine fine Python development tools: Spyder 7.7 Good
Nine fine Python development tools: Wing IDE 8.4 Very Good Sphere: Related Content

10 Websites To See What HTML5 Is All About


HTML (HyperText Markup Language) is the most widely used markup language today. Tags, elements and angle bracket with a bit of help from CSS forms the building block of all that you see around on the World Wide Web.

It has been nearly 20 years since HTML started defining how we ’see’ the web. Now, it’s on the verge of its fifth major revision – HTML5. From its first avatar to its fifth, the web itself has been transformed. HTML5 is an attempt to meet current needs which is heavily based around dynamic and multimedia rich content. But how will it change the way we look and work with the web?

The web of today is increasingly interactive and media rich. Applications are moving online. HTML5 covers all of these needs by introducing ways to embed videos and audio, draw simple graphics, work with apps offline; do all these without plugins, and of course, do it all faster than before.

It is after all a standard. Thus, it’s nigh impossible to sweat the details in a few lines. So here are ten resources that give you an insight on HTML5. Let’s get to know it, because we all might be using it soon.

WTF is HTML5

Research group Focus breaks down the latest specification and gives it straight up as a simple to understand infographic. Within ten minutes, you will get a helicopter view of what HTML5 is all about and why we should care.

Dive Into HTML5

Written by Mark Pilgrim, it’s a hand-picked look at select features of HTML5. It’s put together in the conversational style of the ‘For Dummies’ series. If you want to grasp the basics, this is a must read.

And there’s ‘Professor Markup’ to guide you along with answers to common queries. Each chapter is supported by demos and examples. But do check the mentioned supported browser(s) before trying them out.

HTML5Doctor

At HTML5Doctor.com, you have articles that are meant to help you hit the ground running. For example, using the very first article, you can set your webpage to be recognized as HTML5 by browsers. The fourth post helps out with designing a blog with HTML5.

The HTML5 blog also has a Q&A section, where you can upload your questions. Some of the answers come on the homepage as ‘Your Questions Answered’ posts.

HTML5Rocks

This HTML5 web resource asks that you be well versed in coding. If you are, there are a few tutorials that help you understand how HTML5 features can be coded. You can experiment with HTML5 at the Playground and even reuse some of the code samples in your own project.

The tutorials are few but comprehensive. Each is also tagged with the icon of the browsers that meet the HTML5 feature requirements.

HTML5games



HTML5 is giving Adobe a few headaches because it promises to do away with Flash (at least for some apps). Will HTML5 prove as effective as online games as the Flash based games we have become used to? Check it out for yourself by playing the HTML5 games that are hosted on this site.

You can also read a brief mention of this HTML5 games resource in our directory.

HTML5 Readiness


All browsers are readying themselves for HTML5. The half wheel on this site is a visual indicator of each browser’s groundwork so far for HTML5 and CSS3. You can mouseover the 28 features and gauge the stage from the color coded spokes.

From the looks of it, Safari and Chrome are more down the path than the others.

HTML5 Element Reference Table


Periodic Table of the Elements, a project by Josh Duck is a quirky look at the 104 HTML5 elements (and 2 proposed elements) using the layout of a periodic table. Clicking on each cell brings up a brief description with additional references.

You can also read a brief mention of this resource in our directory.

HTML5 Test


At a glance you can tell your (browser’s) score when matched up against HTML5 ready features. As soon as the website loads up, the score from a maximum of 300 gets displayed. You get bonus points if the support extends beyond what the HTML5 specification might lay down.

The scores are also color coded which indicate anything between what’s great and what’s not available on your browser.

HTML5 Gallery

HTML5gallery.com parades other websites that are using HTML5 in some form or the other. You can click on the thumbnail and reach the websites on view. It is also a learning platform to see how web developers are implementing HTML5.

Each website profiled has a small note that explains how an HTML5 element has been used to design the website.

YouTube HTML5 Beta

The ‘Video’ element is touted as one of the bestselling points of HTML5. You can test it out on YouTube by opting to use the HTML5 player instead of the Flash player for most videos. Your tryouts will also tell you how your current browser is coping with HTML5 videos. Do read the Notes section to note the finer points on the beta effort from YouTube.

Everything says its exciting days ahead. These ten websites on HTML5 are just the early starters of a breed that’s going to explode soon. Hopefully, they will help to put you in the right element to work with HTML5.
Image: justinsonmia
(By) Saikat is a techno-adventurer in a writer's garb. When he is not scouring the net for tech news, you can catch him on his personal blog ruminating about the positves in our world.

Similar Articles

* 3 Great Websites To Learn About Art And Its History

* Really Painless Web Development Using Microsoft WebMatrix
* Top 6 Resources To Become A Better Freelance Writer
* 6 Best Web Design Blogs To Follow
* Create A Mobile Website From Your RSS Feed With WINKSite
* How To Run A Speed Test On Your Website
* 6 Websites To Help You Learn Japanese Online For Free
* Geotag Your Website or Blog Updates with GeoRSS & MapQuest Sphere: Related Content

7/9/10

Agile Development: The quickstart guide to doing it right

Lots of places don’t do agile right, in fact most do it wrong. There are a number of factors why, chief among them I believe is that the percentage of people who have worked somewhere that nailed it completely right is so low that there aren’t enough to go around and lead by example. So here are a few of the tips I’ve been given or learnt over the years, credit has to be given to all the wonderful places I’ve worked over the years. The ones that have done it well, and the ones that have done it poorly… you can learn just as much from reflecting on where it all went wrong.

Back to basics
Are you already using agile, or are you planning on implementing it in your company? If you’re reading this chances are something isn’t working as well as you’d like and so it’s time to reset. First thing to do is throw out your preconceptions on agile, ignore almost everything you’ve read in practitioner manuals, because at it’s core it’s really quite simple. Most problems stem from people picking only certain features of agile or trying to let a tool guide the process for them.

Refocus on the manifesto
It’s worth going back and looking at the agile manifesto and then questioning each aspect of your process to ensure it is giving priority to the values on the left side. If it’s not, remove it.

Throw out the tools you’re using
Scrumworks? Urgh! VersionOne? Atlassian? They’re all cut from a similar cloth. If you’re not already doing agile correctly, they will just get in the way. So until you’re up and running, get them as far away as possible.

For planning, all you need are some index cards and a pen. If that doesn’t offer you enough protection for your DRP or wont work because you’ve got a distributed team then use a wiki. Again, avoid at all costs anything more complicated until you’ve got it all working.

Planning
The great thing about stripping it back to basics is that it re-aligns focus on the things that are important, you’re not distracted about calculating backlog points and predicting velocity. Instead you are collaborating with your customer to understand what they will consider working software.

You need to become the customer
I mentioned in my last post, nobody cares what tools you use, your customer is trusting you to make the right decisions to do the best job. You can only really do that when you understand the requirements completely, a full and deep appreciation for why the work is being done and not simply what needs to be implemented.

Your customer is unlikely to be a software developer so their scope, as broad as it is will be, is limited by their experience. They also can’t possibly know upfront all the questions you want answered and all the seemingly irrelevant detail that would feed in to giving the best user experience possible. Expecting them to bridge this gap is completely unrealistic, instead the gap needs to be bridged by having the developers move towards the customer. The developer needs to be completely bought in to the product, understand the problems it solving, and how they’d best be solved.

This is precisely the reason why tools like cucumber don’t work, they are bridging that gap from the wrong direction. And if we go back to the agile manifesto, that approach carries the potential to skew the priority back towards contract negotiation rather that customer collaboration.

Customers don’t write the stories
And neither do business analysts, development managers, or project managers. I’d go so far to say that if you want to be “agile” and you’ve got a team of BAs then you should fire them all. Anybody getting between the developer(s) and the customer is stopping the developer from becoming the customer. Crucial detail will be missed and misplaced assumptions will go unchallenged. Sure, you’ll still get a product out the door but it will either be what was agreed (which is often different to what is actually wanted) or it will not be as awesome as it could have been.

The people that get between developer and customer have the best of intentions, they’re trying to let developers focus on writing code. I’ll say it again though, they’re job isn’t to write code it’s to deliver a solution. It’s a false economy to think you’re saving the developer time by taking on the requirements analysis on their behalf, it will take longer for them to appreciate 3rd hand the requirements than the half day it would take to know them first hand and write them up themselves.

Pull up your sleeves, get out a set of index cards, some pens, and some A4 paper. No laptops, no ipads, no technology. Force the customer to explain the issue without computer aids as though you know nothing. Each of you draw up wireframes, compare the differences between them to see what assumptions have been made, write the card together. Make it a tactile experience.

The result should be a concise description or the business problem, and the end-user benefit you’re trying to deliver. Never more than a couple of sentences (maybe a few footnotes for pertinent implementation details that can’t be neglected).

The most important stories get done first
This part of it will be difficult, if not impossible, for the developer to make a judgement call on. It is often very difficult even for the customer, but it has to be done. That being said, it shouldn’t be as difficult as many people make it. If you’re working on a new app that’s not yet been released I’ve got a diagram to help you evaluate priorities. It’s also worth keeping a focus on getting the smallest usable feature set out the door as soon as possible. Eric Ries said a good way to get to that point is to outline the minimum features you think you need before you can release it publicly, and then you can probably reduce it by 90%. You always need much, much, much less than you ever expect.

If it’s an existing app and you’re having problems prioritising it might be worth speaking to some users. Just make sure you keep focus on doing the smallest and simplest thing possible to meet the requirement.

A good way to gauge if story is really the next most important thing to get completed is to double the estimate, if that changes the priority in the customer’s eyes it wasn’t as important as they thought.

If you’re not doing the work, you don’t get to estimate it
It took me a long time to appreciate this as a manager, I was often guilty of sizing up tasks based on how long I thought it would take me to complete it. Then someone else picks it up and I wonder why it took so long. Truth is it would always take me longer too but that is too easily forgotten, do you really have a load factor of 1? I thought not.

How easy or difficult a task is has so many variables: familiarity with the code base, how recently you’ve solved a similar problem, access to an existing solution, etc. and each of those come from a fairly personal experience. Sit down, with a pair to rationalise the decision if you like, and come to a conclusion you’re happy with. The important thing is that you are consistently optimistic/pessimistic on your approach, it’s less important to get the estimate right than it is to be wrong on a relatively consistent basis. Everything averages out very quickly when it comes to the planning.

Don’t mention time
Do everything you can to avoid estimating in hours or days. No matter how much you explain things like load factor when you see a story with “2 days” or “8 hours” written next to it people can’t help but think that is how long the task will take. So you can’t be entirely shocked when 3 days in they ask how that task that was meant to take 2 days is going. Avoid the awkward situations completely by not setting the expectation in the first place.

Estimate in points, or jelly beans, or anything other than hours or something that can be tracked by a watch or calendar.

Estimates aren’t guesses, but they’re not accurate either
Don’t take a look at a story, close your eyes, and come up with a number. Take the time to analyse in detail what is required, what systems you’ll need to talk to, read the 3rd party API/integration docs. Spend a whole day or more if you have to and write up a detailed list of tasks in the order you expect to tackle them, including any decisions on implementation you may come to, and put them at the bottom of your story card.

Time invested here pays for itself two-fold later in the iteration by maintaining focus on the minimum set of tasks that need to be completed and setting your direction each step of the way. Plus it gives you confidence that the task is achievable and unlikely to blowout.

When it comes to putting a number (points, jelly bean, whatever) on an estimate I now always advocate a 3 point system. 1 point roughly equates to a half day of effort (ssshhh, don’t tell the customer ;), 2 points a whole day, 3 points 2 days. The reason being that it’s too difficult to consistently estimate small tasks, if you estimate an hour for something and you get snagged and it takes you the first half of a day (not unusual) you’ve blown the estimate by 400%. At the other end of the spectrum you’ve got two problems, either a story that isn’t really implementing the most simple solution possible or a broad scope that it’s difficult to really sit down and estimate in the detail required to be confident in it.

If the tasks are obviously much smaller than 1 point, I’ll try and group a couple of similar smaller tasks into a single 1 point task. If a task is bigger than 3 points, I’ll split it into separate deliverable parts. “But it’s really a 5 point task, everything has to be delivered at once!”, Bollocks!

Estimates for work that isn’t scheduled are worthless
They might be useful for getting a high-level idea of when some story might be delivered, maybe, if it’s ever actually scheduled. Remember one of the core tenements of agile is responding to change, and nothing impacts priorities like actually giving software to users. So it’s not uncommon for that really-urgent-we’ve-gotta-have-it next feature to get bumped for a glaring hole a bunch of users pointed out. And then bumped again for something else. It comes back to expectation management again, and putting an estimate next to it adds a misplaced finality to where the story sits in the priority queue.

The other issue is that each iteration of development feeds back into next. Some previously completed work may make a later story easier to implement, or it may actually highlight that something is more difficult than originally expected. Either way there are enough factors that can completely invalidate whatever estimates you come up with.

Execution
Once the planning is done, it is down to actually starting the work. It’s not always a matter of ticking boxes off the work list though, the most effective teams I’ve worked in have shared some common traits.

Start mid-week
Few people love a Monday morning and getting motivated can sometimes be an issue. If you compound that difficulty by expecting everyone to be in the right mindset to sit down and write stories, well it doesn’t always work out for the best. There is no real long-term cost associated with moving the start of the iterations from a Monday to a Wednesday, but allowing me to be productive first thing after a hazy weekend by allowing me to look at a list of work and just crack on with it is a huge win.

Have a fixed iteration length
I prefer to work with 2 week iterations, 1 week feels too short and 4 too long.

There is a comfort that comes from having some order to the world, and the importance of the general psyche and morale of a team is often underestimated. On those times that a story does take longer than expected, it’s nice to know that you’ve still got a week or so to make it up. It’s also great to have a regular sense of completion in predictable intervals, the dreaded risk of being stuck on the same task that is going to take months never materialises (partly because you never schedule a story with more than 2 days of effort, right?). There is a lot of subtlety in the impact this predictability has on developers which makes it hard to pinpoint all the benefits, but when combined they all make for a happier environment.

From a management perspective, it’s great to have some clarity on when things are going to be delivered. And I’m not just talking about for the next fortnight, but you know on a specific day each and every fortnight something is going to be delivered. So does the rest of the team. It makes planning much easier, people know the most convenient windows to take holidays, and which days to stay home are particularly inconvenient because you’re planning with the customer.

Developer attention is a premium
Jason Fried from 37signals has a great video explaining why you can’t work at work that I’d recommend watching. When you’re deep in the coding zone nothing is more disrupting to your flow than someone coming and interrupting you to ask you for opinion on something completely unrelated. That 10 minute interruption can take 30 minutes to recover from fully to get you back into the same state. If they happen 2-3 times a day that is up to 25% of the workday productivity lost to distractions.

And as Jason mentions in the video the very action is basically a way of saying “Hey, my immediate needs are more important than whatever you’re focussing on, give me attention”. It’s selfish and unhealthy for productivity. Remove phones. Email people if you have to, but don’t expect a response that day. Inter-team or important but non-urgent questions should be posed via an instant messenger, chat program, or something that doesn’t demand immediate interruption (like phones and meetings do). What about things that are urgent? Well that needs to be assessed on a case-by-case basis, but the truth is truly urgent things happen very very rarely. Almost everything can wait at least 4 hours.

It’s here that a good development manager can work wonderfully, acting as bouncer and preventing direct access to developers while they’re busy coding and filtering the urgency of all requests.

Pair Programming
A fairly contentious part of agile is pair programming. When it works well, it’s hugely efficient and carries lots of additional benefits (higher quality code, quicker delivery, lower documentation requirements, less “single developer” business risk, etc.) but when it’s implemented poorly it’s just a waste of a resource. In most places I’ve worked it sadly falls into the latter camp, and it’s because people are just paying lip-service to the practice and working it as mentor/tutor type role rather than a pair actively writing code together.

That’s not to say it doesn’t also offer great potential as an approach to training, the important thing is both participants need to be as involved and active in writing the code.

Owning the workspace
The biggest problem when it doesn’t work is normally down to the setup of the workspace. Generally Person B brings their chair over to share Person A’s desk, and Person A stays almost exactly where they were to begin with. It’s guaranteed to fail. Here’s what you need to do:

•Share a single monitor, you both need to be looking at the same screen. They’re big enough these days that you can see everything you need with appropriate window management.
•Each person gets their own keyboard and mouse, no co-piloting on shared inputs and it’s not fine to have one person with a keyboard and the other person using the laptop it’s connected to.
•Divide the screen in half, then mark that virtual line on the desk with some tape or a marker… right down the desk. Each person gets their side and never should they encroach on the other half.
It might all seem a bit contrived and naff, but it makes a difference to the efficiency of the operation. Body position plays on the sub-conscious and determines whether you feel comfortable using the keyboard in front of you. Sitting too far to the side and you start to feel that you’re a spectator on somebody else’s computer, too far in front and you feel like you’re a pilot and co-pilot rather than peers.

Everyone is in their own context
You’re going to know what programs you need open to get the job done. For me it’s an IDE, a terminal or 3, and a web browser or the running application. Agree on a common screen layout where everything can be seen at the same time, and then mandate it across the entire team (I’ve found Divvy really useful to keep it consistent). It means that when you come to someone else’s computer it doesn’t feel foreign, it’s just like being on yours but at a different desk.

Most importantly though is that even though you’re both working on the same task together on the same machine, you’re probably both in slightly different head spaces and thinking about a slightly different aspect of the problem. Quickly switching between applications to satisfy your own curiosities can have a terrible impact on whatever it was your pair was thinking about. Being able to see everything at the one time prevents you from unexpectedly switching context on each other.

Make sure you get a monitor large enough to make it work. Given you’re going to spend ~8 hours each and every day staring at it, it’s not sensible to buy a cheap screen.

Silence is golden
This is difficult to nail until you’ve developed a good working relationship with your pair, but the best communication between the two of you is the non-verbal. For the same reasons above, you can never be sure what the head space of your pair is and interrupting them to mention a typo could completely break their train of thought. Instead wait for an obvious pause or context switch and simply point it out. Eventually it may get to the point where you know each other well enough that even more subtle queues like a shift in posture give away that you need to re-think the code you just wrote. But at least you can then do it once you’ve got your current thoughts committed to screen.

80 characters is enough for anyone
Like many of the tips on here, they’ve either been passed on to me by Graham Ashton or refined further by working with him for 3 years. This one seemed particularly pedantic to me, but I went along with it because it really wasn’t that difficult to adhere to and he was adamant that he’d accept nothing less. And it’s this, no line of code should ever extend beyond 80 characters.

I now try to enforce it wherever I go.

It took me a long time to fully appreciate how important it is, at least a year, but code readability is drastically improved and that is always a good thing. When pairing though, it is mandatory. You don’t have the luxury of scrolling indefinitely to the right to read the full line of code to see the intention because doing so will interrupt the flow of your pair. Worse still, it actually takes the bulk of the code off the screen.

It also prevents a nasty scenario where something important that drastically changes the intention of a line of code is cleanly hidden by the window edge and the real flow of the application is the opposite of what you thought was happening. Say something like the following:
def my_example_method
raise "Here is an example of something you would think gets raised!!!" unless
coder_reads_this_part?
end

Refactor any line that stretches over 80 characters. It only requires a little thought, makes the intention clearer, ensures you can see all the important code at once, and stops you from having to break someone else’s focus.

Test Driven Development
Tests first, then the code. It’s not just about ensuring you’ve got stable and working code, it’s about keep focussed on doing the smallest thing possible to make the tests pass.

This approach works great with pair programming, and here’s the formula: One person writes a test, the other makes it pass. Once it’s passing the person that just wrote the code writes the next test and hands it over. While one of you is trying to write a crafty test to keep the other busy for a while, your pair has already devised a cheeky way to have it always return the value you want and has devised a new test to catch you out.

It’s generally a quick back and forth iterative process where you’re each trying to outfox the other with edge cases or overly simplistic implementations that pass because the tests are robust enough. It’s almost game-like, and eventually you both come to a stalemate. At that point you sit back and realise the story is complete, and with excellent test coverage.

Don’t break the build
A continuous integration server is a must as is some form of reporting of any errors raised in production, and problems on both must be treated with a suitably high priority. Anything that is deemed “flakey” and raises an alert for non-legitimate reasons needs to be fixed immediately. As soon as the team starts to doubt the notifications, legitimate problems start to slip through too. It’s a bit like the broken windows theory, if you allow these things to go by without action then it breeds complacency and soon some failing tests become accepted as the norm.

That all means that before any developer commits any code back upstream, they run all the tests… all the time. They are there for a reason, and it’s inexcusable to not be running them. The CI box is there as a backstop only.

Deliver at the beginning of the end
If you’re starting your iterations every second Wednesday then code needs to be completed at the close of play the Monday immediately prior. That means you’ve got a full day to deploy the code to production and catch any unexpected errors. The other benefit of the mid-week approach is that on the rare occasions it all goes pear-shaped people are more inclined to stay back a little late and fix it. Trying to keep people back after hours on a Friday isn’t just difficult, they’ve often already mentally checked out and off on their weekend and you run the risk of actually making a bad situation worse.

If it all goes seamlessly and you roll-out and have a day free, fantastic! Time to look at that ever growing list of bugs your users have been sending back that haven’t been getting scheduled into an iteration. You can always find small tasks to fill the day that are incredibly useful but not super urgent.

Avoid bringing planning and unscheduled work forward, you’ll skew your workload for the next iteration and run the risk of over-stressing developers and/or over committing. The benefit of maybe squeezing in a day of development isn’t worth the additional risks.

Review
The code is released, people are using it, and everyone is happy. Now it’s time to look back and see how things went, what went really well and what could be improved upon.

Run a retrospective
Before the planning session, get all the team in the room to talk about the good and the bad of the previous iteration. Do it quickly, you should need an absolute max of 30mins but you can probably do it in much less. Get index cards or post-it notes of two different colours, one for “went well” and one for “needs improvement”, and everyone has to write at least one thing on a card of each colour. There are no limits to how many things people can list though, allow it as a cathartic forum for all issues to be brought out in the open.

Put all the cards up on a board (grouping similar topics), so everyone can see the balance of opinion on how things went. Ensure everyone understands what all of the cards mean and what they are referring to. Now everyone gets 3 points to spend on the “needs improvement” cards, they can spread the points out across 3 cards or put them all onto a single card (or the obvious option in between). The 3 cards that have the most points the team agrees to make a concerted effort to improve upon during the next iteration.

It’s only after a couple of iterations with consistently re-appearing top rated problem that I’d consider going back at looking at some of the tools I mentioned throwing out at the top of this article. And even then, only if you’re certain they’re the best way to fix the most pressing issue facing your team. If I’m honest, I don’t think it’s going to come up.

Split partially-completed stories
Occasionally a story only gets partly done, and there is much gnashing of teeth when people have to work out what to do with it. Because you’ve been taking an iterative test driven development you’ve got a bunch of code that is written, tested, and passing even though the story is incomplete. Fantastic!

Look at what is left to complete and write up a suitable story to reflect it, then go back and revise the previous card to reflect what was actually done. Now look at the original estimate and try and work out what percentage of the story you’ve completed, apply the appropriate number of points to each story (keeping in mind that if you’re following along with my 3 point/jelly bean system, 3 points = 2 days. So if you’ve done half you’ve got 1 day on each or 2 points on each).

That way you get an accurate reflection of both what was completed in the previous iteration and a fair idea of what remains, based on your original estimates (remember, we wanted continual optimism. No revising estimates retrospectively).

Calculate your velocity
Now you know how much work you got completed, you can with a fair degree of confidence predict how much you can do next iteration. After a few iterations, you can look back and calculate the average for the past two or 3 to flatten out any particular fast or slow ones and get a good feel for what is achievable.

Under-promise, over-deliver
No kid likes waking up Christmas morning to find the present they asked Santa for to be nowhere under the tree. Likewise, no customer likes being told you’re going to give them something in two weeks and for it to not materialise. So if you’re uncertain, you’re better off scheduling too little work and then pulling an extra story in later in the iteration than over committing. It’s not just about managing customer expectations though, it’s about limiting stress on developers by keeping the targets realistic.

Conclusion
These are the things that have contributed to making it work at places I’ve enjoyed working at. The most important thing is to measure overall success of your process as it’s ability to create a happy and productive environment, good things will naturally flow from it. At the very least you should apply the principles of agile to the implementation of the process itself. Nothing is set in stone, pause at regular intervals to review the success, adjust the most important aspects to give people what they want. Sphere: Related Content

Some random thoughts on programming

As I near the end of my third decade programming, I keep finding myself coming back to pondering the same kind of questions.

■ Why are some people good at this and some are not?
■ Can the latter be trained to be the former?
■ What is the role of language, development environment, methodology, etc. in
improving productivity?
I’ve always been partial to the “hacker”. The person who can just flat out code. For some people this seems to come easy. I have been lucky enough to work with a few people like this and it has been awesome. As you talk with them about a problem they are already breaking it down and have an idea of how the final system will work. I don’t mean that other aspects of professionalism, experience, etc. aren’t important its just that I believe in the end there is a raw kind of talent that you can’t teach. My wife can sing, I can’t, and not for lack of practice. She practices, but she can also belt out a perfect tune without hardly trying.

What are the core talents that makes someone a good hacker? I think there are two:
1.The ability to break down an idea/system into an abstract set of “concepts and
actions”
2.The ability to understand how these “concepts and actions” execute in a
particular environment.
Note that I didn’t say anything about coding. Coding is actually the least interesting part. If you can do the others well, coding is just another kind of typing. Coding is the thing your fingers do while your mind is flipping between 1) and 2). Learning a new language is less about learning the syntax than it is about learning the execution model.

Writing good, correct code involves not just conceiving of how something should be 1) but also understanding all that can happen 2).

I honestly don’t know if these things can be taught.

In 30 years I have only seen a few examples where a mediocre programmer became good or a good one became great and zero examples of a mediocre programmer becoming great.

What does this mean for people who want to build large software systems? The easy answer is to only hire great people. Which is fine when you can do it but not everyone can be or hire the best so what is left for the majority?

What should be done to make things easier for people aren’t in the top 5%?

There are lots of nice things that have been invented like source code control, higher level languages, garbage collection, etc. which reduce or abstract away some of the problems. And while sometimes these abstractions are leaky, they are really one of the few weapons we have against the beast called Complexity. Complexity is the key problem we face, especially if we are not wizards because:

A persons talent with regard to 1) and 2) above determine how much complexity they can deal with.

When I was in gradual school at UNC-CH Fred Brooks (the mythical man month guy) used to talk about Complexity as being a major thing which makes programming hard. Its not something easily cast aside:
Quoting Dr Brooks:
The complexity of software is an essential property, not an accidental one. Hence, descriptions of a software entity that abstract away its complexity often abstract away its essence.

I wish I had appreciated everything Fred said when I was a cocky grad student as much as I appreciate it today… sigh.

Some things are hard because they are hard and not much can be done about them. Still our goal needs to be to not try to make it worse than it is.

To this end, some people say: Do the simplest thing that will work.

I agree but prefer this formulation: Strive for the simplest thing that can’t possibly fail.

If you think about what can go wrong you are more likely to find the problems than if you think about what will go right.

I wish I could say we were making progress in reducing complexity but we seem to be adding it into our programming environments rather than removing it. One of the things I have thought about recently is the complexity of these abstractions themselves. Programming languages of today are so much more abstracted and complex than what I learned on (Assembler, Basic, Pascal, C).

For example the world (or at least the web) is currently engaged in writing or rewriting everything in Javascript. Not because JS is so awesome, but because 15 years ago it got stuck into browsers, then we all got stuck with browsers and thus we are all stuck with JS. Sure there’s lots of server stuff done in various other languages but really how long can that last? How long can one justify having to write 2 parts of an application in 2 different languages?

The future looks like node.js.

I am not a JS hater. Its just that I used to think Javascript was a simple language. As someone who wrote 3D Game engine code in C++, my thought was “if these webkids can do JS then it can’t be that complicated”. After building a webapp using Javascript/Dojo/svg to create a realtime browser for genetic information, I realize how wrong I was.

Sure its all nice and simple when you are doing this:
$(“#imhungry”).click(function() { alert(“food”);});

However a lot of complexity lurks just under hood. And its the way it all interacts that concerns me. Crockford called it Lisp in C’s clothing. This should excite a few people and scare the crap out of the rest.

JS has lots of opportunity for expressing yourself in interesting ways. Expressiveness is good, but too much of it can create more problems than it solves. Back when C++ was a bunch of source code you downloaded from AT&T and you had to compile the complier yourself before you could use it, we grad students stayed up late eating bad fried chicken and debating the merits of O-O programming. It was at this point I began to wonder if the productivity gains of O-O were more the offset by the time spent debating about what good O-O practice is. This tape has played in my head more than a few times in my career.

Its not all bad of course: On balance, garbage collection is a win and I see lots of benefits to using closures in JS, but the underlying complexities of things escape most people and their interactions are understood by even fewer. If you have to slog through this (and you should) to understand what happens when you call a function, you are cutting out alot of people from fully understanding what is going on when thier code executes.

I hope that some of this will be address by standards of practice. Things like Google’s guidelines seem to be a good start. As with C++, I think that the best hope for improving productivity will actually be to agree on limits on how we use these tools rather than seeing how far we can push the envelope.

Well except for wizards and we know who we are.


15 Responses to Some random thoughts on programming
Jason P Sage says:
September 7, 2010 at 7:49 am
Right On. Nice Article… I’m on year 28 of programming – and I really liked this article.

P.S. I am a JS hater and .Net hater too (though good at both) because P-Code and Compile On demand are Slower and Make it necessary to obfuscate your code if you have intellectual property concerns.

Jonathan Palethorpe says:
September 7, 2010 at 7:52 am
22 years ago when I started in programming I could mess about at home doing a bit of BASIC and at work I wrote green screen sales and invoicing apps in COBOL ’74 using ISAM files. Nowadays programming is many times more difficult- ISAM files have been replaces with all singing all dancing SQL databases with a host of frameworks designed to access them (no in-language stuff as in COBOL with ISAM) – the green screens have been replaced with name your weird and wonderful GUI framework of choice (WPF in my case) and the languages themselves are now many, varied and unreadable to those not in the know.

I much prefer doing development work now but there are times when I still hanker after a bit of green screen/ISAM/MOVE W01-FOO TO W02-BAR

Michael Fever says:
September 7, 2010 at 9:24 am
Yeah, I’ll have to admit that was pretty random. I am an ok programmer, not great, but better than mediocre. I approach every job the same however by looking at what needs to get done and then by what is going to be required later to maintain it. Sometimes being OK is all you need to be.

Bob Gibson says:
September 7, 2010 at 9:47 am
Very good points.

I can relate… I started programming, in Basic & Fortran, … 40 years ago.

Jeffrey Lanham says:
September 7, 2010 at 10:13 am
Right on. After working in computers and programming for 27+ years, I have to agree. Programming is, unfortunately, something you inheritly understand or you don’t. You can teach semantics, but not the thought processes that go along with it. That being said, great programmers start as mediocre programmers, but quickly rise. Mediocre programmers tend to rise to somewhat competent and then stay there. That’s, unfortunately, the way it is.

Great article.

Alan Balkany says:
September 7, 2010 at 10:15 am
Sometimes the “hacker” is just someone who’s very familiar with the problem domain. An average programmer may appear to be a “hacker” to their coworkers when they’re re-implementing a system very similar to what they implemented at two other jobs.

They already know what works best, and the mistakes they’ve made in previous implementations.

I think most people don’t program as well as their potential because the best practices haven’t been communicated to them. For example, I see many programmers who don’t seem to know that reducing function length is a good way to keep complexity manageable.

The best practices are still being worked out; the field of software development is still in its infancy, compared to more mature disciplines such as mathematics and mechanical engineering.

Oisín says:
September 7, 2010 at 10:47 am
Alan is spot on here. Perhaps the idea that many programmers overlook useful, basic heuristics for programming (such as “shorter functions imply more manageable code, so break down your functions until they are small”) ties in with one theme of the OP, that programming has become more complex in terms of frameworks and maybe language expressiveness.

When I started programming, it was on the horrible bare-bones no-frills Commodore 64 Basic interpreter, followed a couple of years later by a lovely language called GFA Basic on the Atari ST, and then by 68k assembly and C. Maybe those formative years of just playing about, writing programs with minimal focus on libraries, industry standards and “frameworks”, were helpful in learning how to solve problems and write code.
But these days I spend more time reading articles about flashy programming/library techniques than actually thinking about and writing programs.

It could be the case that novice programmers nowadays jump too early onto this wheel, without spending the time doing simple, pure programming, so that they don’t sufficiently develop these key abstractions and heuristic tricks for managing complexity.

Joel Wilson says:
September 7, 2010 at 10:54 am
Couldn’t agree more about programming skill or JS. The best programmers I’ve ever known generally don’t have a comp-sci education (myself included). In fact, the single best I’ve ever known personally was an English major a couple of papers away from his Doctorate. He found programming and never looked back. As far as JS, I agree that its “appeal” (for lack of a better word) is simply because it’s the only trick in town. As a language it sucks for anything more than basic functionality. Although I use jQuery professionally, I find it sad that we have to resort to bloated frameworks and layers of abstraction to try and hide the basic weakness of the underlying language.

Leon says:
September 7, 2010 at 11:39 am
Oh, I guess for that post you’ll earn a lot of flames from mediocre guys who think that they will be great one day. Especially nowadays when everyone and his mother see themselves as coders because they can hack up some javascript/ruby code

Maintenance Man says:
September 7, 2010 at 1:12 pm
Say it ain’t so. Us mediocre programmers need some hope for the future. Heheh.

CollegeGrad says:
September 7, 2010 at 1:38 pm
I majored in Mathematics (not applied) and only became interested in programming in my junior year of college. I only started to really learn quality programming skills after leaving college. I find that I often know exactly how to solve a problem before the veteran programmers at my job even have a chance to really think about it. The problem for me is spending the time making sure my code is well structured and maintainable. When working with other programmers, I end up being the one explaining the problem and how we’re going to solve it over and over. My own experience, which I’ll be the first to point out is very limited, has indicated that it’s my experience with the theoretical aspects of computer science and complexity theory, as well as all my time spent proving abstract theorems, that gives me an advantage over the people who just know how to code. Maybe if programmers focused less on languages and more on things like linear algebra, abstract algebra, graph theory, etc. then they would be much better at their job. just my novice opinion.

Ryan says:
September 7, 2010 at 2:29 pm
“Do the simplest thing that will work.”

Not sure if you will even read this, but:

That is a great statement *but* I think it is critical to mention what I consider a serious and way too often overlooked problem. Lets say you start with a great code base. Then you amend/change it ‘as simply as possible’ to add a new feature. And again for a new feature, and again. Every step being done ‘simply’ actually adds unnecessary to the complexity of the program. What really needs to happen is the simplest implementation that incorporates *all* features, including the ones that had previously been implemented. And this takes more time and more effort in the short term, but boy does it save on time and effort in the long term.

Giles Bowkett says:
September 7, 2010 at 2:46 pm
Since my comment disappeared into a black hole with no feedback, I can only assume it’s gone into some moderation zone, which means I should assume people will read it, which means I should probably be less obnoxious about it, so: Java was designed to prevent people from fucking up. It did not; instead it prevented people from writing great software. That’s in the nature of design approaches based on minimizing the damage that the incompetent can do. The incompetent can fuck up anything; the only thing you achieve with standards and practices is restraining the so-called “wizards,” who create all the advances that save everybody else so much time. Exercising control to prevent incompetent mistakes is a sucker’s game; freeing up the language to “give them enough rope” results in great libraries which make everybody more productive (or at least, everybody who uses them).

I’m rehashing a debate for you here from 2006. It comes from the days when people were saying Rails would never be accepted in corporations, and, more to the point, your advocacy of standards and practices to prevent people from fucking up is the same argument that was coming from the people who said Rails would never be accepted in corporations. Their logic was clean and their conclusions were false; to anyone who was reading those debates, your argument was, I’m sorry, completely discredited almost five years ago now.

Sorry to go off in such detail on your final paragraph, which was kind of a tangent to your overall post, it just drives me fucking crazy that people settle these arguments definitively, only to see them revived a few years later and started over completely from scratch. “Those who don’t study history are doomed to repeat it,” etc.

admin says:
September 7, 2010 at 2:47 pm
I think it depends on how draconian the limits are. I don’t think the google guidlines for instance are any kind of crushing blow to innovation. I just don’t think that just because a language allows something you should necessarily do it.

admin says:
September 7, 2010 at 3:03 pm
Ok. Thanks for the longer comment. I’m learning alot about how bad I am at making my point.

Putting people on rails (sorry couldn’t resist) isn’t going to make bad programmers good. Crap can come from anything. That wasn’t what i wanted to say.

I’m more interested in keep some kind of boundary so that what results is maintainable enough by other people.

I Actually pushed for doing a recent project in python/django rather than java etc because I value the benefits which some of these newer, better langauges give you.

I just think with great power comes great responsibility. The cool meta-programming trick you did is awesome and personally I will give you props for it. I just hope the next guys can figure it out too.

Also I think that not all “rope” as you say is equal. Some things allow for great expressiveness and freedom while not adding much to complexity. I don’t use ruby so I’m not an expert on that, but in python there are function decorators, its a cool syntax for people to add some functionality. But its pretty explicit when its used. @login_required etc. Javascripts closures seem to be able to happen almost by accident or at least its not as explicit to someone looking at the code for the first time. Doesn’t mean we shouldn’t use them. I did today, but we need to be aware of what we are doing. And not i am not trying to start a python/javascript debate. Sphere: Related Content

Are microkernels the future of secure OS design ?

MINIX 3, and microkernel OSs in general, might show us the way toward the future of secure OS design.

Date: September 6th, 2010
Author: Chad Perrin
Category: Application Security, Security

--------------------------------------------------------------------------------
Andrew S. Tanenbaum created the MINIX operating system in the 1980s as an instructional tool. He wanted to provide an aid to helping people learn about the design of operating systems, and did so by creating an incredibly simple — in some respects, too simple for any serious use — Unix-like OS from scratch, so that his students could examine and modify the source code and even run it on the commodity hardware of the time. He and Al Woodhull co-authored a textbook that focused on the MINIX OS.

Since then, 1.5 and 2.0 versions of MINIX have been created as well, with the same goal: to serve as a study aid. By contrast, the current iteration of the MINIX project, known as MINIX 3, is being developed with the aim of providing a production-ready, general purpose Unix-like operating system. In Tanenbaum-Torvalds debate, Part II, Andrew Tanenbaum describes the relationship of MINIX 3 to its forebears:

MINIX 1 and MINIX 3 are related in the same way as Windows 3.1 and Windows XP are: same first name. Thus even if you used MINIX 1 when you were in college, try MINIX 3; you’ll be surprised. It is a minimal but functional UNIX system with X, bash, pdksh, zsh, cc, gcc, perl, python, awk, emacs, vi, pine, ssh, ftp, the GNU tools and over 400 other programs.

Among Tanenbaum’s key priorities in the development of MINIX 3 is the desire to improve operating system reliability to the level people have come to expect from televisions and automobiles:

It is about building highly reliable, self-healing, operating systems. I will consider the job finished when no manufacturer anywhere makes a PC with a reset button. TVs don’t have reset buttons. Stereos don’t have reset buttons. Cars don’t have reset buttons. They are full of software but don’t need them. Computers need reset buttons because their software crashes a lot.

He acknowledges the vast differences in the types of software required by these different products, but is optimistic that the shortcomings imposed by the necessarily greater complexity of software in a desktop computer can be mitigated to the point where end users will no longer have to deal with the crashing behavior of operating system software. As he puts it, “I want to build an operating system whose mean time to failure is much longer than the lifetime of the computer so the average user never experiences a crash.”

There are already OSs that meet that goal under extremely limited, tightly constrained circumstances, but he intends to provide the same stability and reliability under the harsh, changeable circumstances of general-purpose consumer computer use. It is an ambitious goal, but his approach is well thought out and shows great promise. Given that many of the stability issues we see in operating systems using monolithic kernels result from the fact that a crashing driver can crash the OS kernel itself, his intention to solve the problem by moving drivers out of kernel space may be on the right track.

In fact, if a driver fails to respond to what he calls a “reincarnation server”, which monitors the status of active drivers, the reincarnation server can silently swap out the hung or otherwise failing driver process for a fresh process that picks up where the original left off. In most — if not all — cases, the user may not even know anything has happened, where a monolithic kernel OS would likely crash immediately.

While his primary interest is in reliability, rather than security per se, there are obvious security benefits to his microkernel design for MINIX 3. Without going into too much technical detail, there are mechanisms in place that restrict a driver’s access to memory addresses, limiting it to its own assigned address space in a manner that in theory cannot be circumvented by the driver. Drivers all run in user space, keeping them out of kernel space, whereas kernels loaded as part of a monolithic kernel design can typically touch any memory addresses at all.

A number of countermeasures have been invented and employed in various OS designs over the years, but keeping drivers out of kernel space entirely certainly seems like the most effective and inviolate protection against driver misbehavior. As a result, this may all but eliminate the possibility of buffer overruns and similar problems in kernel space.

Further, mediated contact between drivers and other user space systems in MINIX 3 extends the strict privilege separation of Unix-like systems to protect other processes than the kernel process, as well. In short, whole classes of common system vulnerabilities may become extinct within the family of OSs that may develop in the wake of MINIX 3.

MINIX 3 itself is still in development, but it is currently a working OS with many of Tanenbaum’s intended reliability assurance features already implemented. You can download it from the MINIX 3 Website and boot it from a LiveCD though, as Tanenbaum states, you should install it to a partition on the hard drive of a computer if you want to do anything useful with it.

It will not replace MS Windows, MacOS X, Ubuntu Linux, or FreeBSD right now, but it may well be the future of general purpose OS design — especially secure general
purpose OS design. Sphere: Related Content