“How hard can it be?” – on coding, chess and elo

Introduction

It happens to all programmers. After some preliminary work, you’re at the point where you can see the solution with your mind’s eye, and all that’s left to be done is to write the code. You split it into steps that you think will bring you success, and start coding. Quickly, you realize it’s not so simple, and a few days later you get a version running but you’re not happy with it. It’s fragile, big, and performing awfully. It has the elegance of a drinking giraffe, and you feel disappointed. Many programmers say to themselves at this point: ‘How hard can it be?’.
Even worse, their managers, who can’t code to save their lives, will ask the same question.

Let me share an insight that comforts me at times like these. I will try to answer that exact question.

A Lower Bound

I will cheat a bit, as trying to establish exactly how difficult programming is is too difficult for me. So I will settle for a lower bound. Let’s find something that’s actually easier than programming. Well, chess seems to be a good candidate.

Chess is a limited game. Its entire universe consists of only 64 squares. There are two armies of maximum 16 pieces, each piece following some simple rules. Players take turns. Each player moves a piece (exceptionally 2) and tries to mate his opponent. In the starting position, one has only 20 possible moves, and a complex position has some 50 possible moves (I have seen composed positions with more than 200 moves) of which maybe 10 seem actually playable. A game can last 200 moves, but a typical game hardly reaches 50 moves. In the end, you can only have a finite amount of possible positions.

Programming on the other hand is far less limited. You can combine algorithms, data structures, programming languages and paradigms in almost an infinite number of combinations. As you write code, the only thing holding you back is that you have to produce something that’s acceptable to the syntax of the programming language you’re using. Other than that you’re free. A programming language like Java has a simple (compared to C++ at least) grammar, but still is very rich in possibilities. You can take the grammar to build small classes. You can then take these classes, and combine them like Lego, to build more complex building blocks, ad infinitum. Event better, your playground (the hardware you work against) becomes more and more powerful so you can try things that were impossible just a few years ago. Compared it to chess, it seems that in programming you can combine pieces and build new ones, enlarge the board, and so on. Come to think of it, it’s safe to say programming is as least as complex as chess.

If someone would still have doubts, I can say that we build chess playing programs that will beat any human. How far are we on programming programs? Ok, Chess is indeed a lower bound. Programming is at least as hard as Chess. It might be a lot harder, but that is too hard to prove, so I’ll settle for ‘at least as hard’.

About Chess

Let’s talk about Chess some more. It is actually quite a mature game. The first professional supposedly was Abu-Bakr Muhammad ben Yahya as-Suli (854-946). He was the strongest player of his time, and author of a book describing a systematic way of playing Shatranj. Apparently, in the Arab World, one can still occasionally hear the complement “he played like Yahya as-Suli”. But he played one of chess’s predecessors.

Modern chess has been around for more than half a millenium, and people are quite surprised to learn that some of the standard manoeuvres have been around that long.
For example, every chess player knows the Lucena position shown below


which is attributed to Luis Ramirez de Lucena, who wrote a chess book published in 1497 (you read correctly: 1497!).

There are a great many chess books around. Google books answers intitle:chess with more than 200000 results. Only a minority of these books concern beginners as most of these target club players and above. It’s also funny to note that such a limited game is rich enough for lot of new books to be published every year.

About Chess Players

One of the interesting things about chess is that every player has an elo rating. This allows you to see exactly how good a player is, and it allows you to calculate the odds of a player winning a game against another player. Fide keeps records of the ratings, and has an online database you can browse. For example, let’s take two of the best players in the world, and compare them.Top dog Magnus Carlsen currently has a rating of 2835. If he’d play the number one US player Hikaru Nakamura, who currently has a rating of 2759, he will score about 0.60, or score about 6 points in a 10 game match.

import math
def expected(r_a,r_b):
    d = r_b - r_a
    e = d / 400.0
    n = 1 + math.pow(10,e)
    return 1.0/n

print expected(2835, 2759) 

A person’s chess rating evolves in time. If one scores better than one’s rating predicts, the rating will go up, and vice versa. Improving your chess skill equals an elo gain. For example, Anish Giri’s evolution can be found here. In fact, elo is so prevalent, that people use it to refer to an opponent: it is even more important than the opponents name. If a chess player wants to show his game to a collegue, he will say something like:”I’ve beaten a 1900 yesterday. Let me show you how it went”.

Wanna Play?

As it turns out, chess is too hard a problem for most people to master. A master (see International Master) has a rating somewhere above 2400. It is a level only a few people ever achieve (about 1 or 2 percent of chess enthusiasts world wide).

Personally, I can testify it is very difficult: I started playing at the age of 17, invested countless hours, read (at least bought) more than one hundred books, and I only reached a level of 2000. There are some defects in my game I just cannot seem to eliminate, so I must limit my ambition. Talking to people about the reasons for this, they say the main reason is that I started too late. Anyway, if I put in some more effort, maybe I can reach 2100. Maybe.

So You Think You Can Code?

Ok, let’s go back to programming. We asserted before that it is at least as difficult as chess, and then learned chess is actually very difficult. So programming is really really hard. A way to improve is by doing it, analysing where you went wrong, and iterate, hoping you improve while doing this.

Just like chess, reading a book about it will not make you a better programmer. If you’re lucky, and read a good book on the topic (be it chess or programming), you will get some insights in what you are doing wrong. But these insights themselves will not magically improve you. So you need to put in hard effort. Some of the things I learned that will help you do that effort are:

  • Tackle a new type of problem (a chess engine, a rendering engine, a distributed key value store, a compiler, …)
  • Learn a new programming language
  • Learn a new programming paradigm(FP,OO,Declarative)
  • Learn a new platform(GPU,FPGA,…)
  • Learn from a better programmer

Master Coders, Anyone?

Whatever the activity, some humans will be better than others. As with chess, in programming, there will exist some people that have reached ‘mastery’. But, I’m a sceptic. I haven’t seen any. Moreover, as long as we don’t have anything in place like programmer elo I can claim they haven’t been observed. Actually, programmer elo can function as a discriminator for authors too, just like it does for chess books (chess book written by a non-master are rare, and being a master is not a guarantee you’ll write a good book either, but that’s an entirely different discussion)
I’m quite annoyed by the books that come out telling me how to code or to organize the software development process, from people who are clueless. Long gone are the days I used to attach value to a thing merely because it was written down in a book. In fact, the more somebody tells me how to do it, the more suspicious I become. This is a direct result from my interaction with chess masters. Sometimes I ask advice from a chess master on a problem I faced, and most of the time they answer in a sort of fuzzy wordings like “I think you probably should try to arrange your pieces something like this …”. The same position, when shown to a lower rated player often leads to an explanation like this: “you need to first take here and then there, and push that pawn”. How can it be that the same position is unclear to a master, while it is clear to a patzer ? I think it comes from respect for the complexity of game. If you see more, you fear more.

Closing Words

Basically, I think two things are worth remembering from this post. First, programming is really really hard. Second, it would be beneficial if there would be a programmer elo. It would at least prevent some authors of gathering enough courage to publish a book on programming. It would certainly allow better hiring decisions, and maybe could help you earn the money you deserve for your efforts. Or are you overpaid?

have fun,

Romain.

Post Scriptum

Some people remarked something like programmer elo couldn’t be done because programming isn’t a head-to-head competition. As it turns out, this is not necessary. Sites like chesstempo.com show that you can also compete against puzzles. What happens there is that a competitor gets a puzzle. If he solves it, he collects points from the puzzle and the puzzle loses rating points. If the user fails, the puzzle collects points from the user. This way not only the users get sorted, but also the puzzles, which allows the site to always present users puzzles that more or less fit their level. The same concept can be used for any kind of test/examination. The good thing is that people get a tool to calculate their level and can work to improve it. Exactly the kind of feedback loop one needs.

About these ads

15 Comments on ““How hard can it be?” – on coding, chess and elo”

  1. Arturo G. S. says:

    I don’t think an elo-like rating is applicable for programmers for two reasons: programmers don’t compete with each other, and the programs they code are not the same; these two facts make it really hard to evaluate the programmers, or at least I have no clue of how to do it.

    For example, if we have two “perfect” pieces of software, say a word processor and a chat client, each one coded by a single programmer, how can we say which programmer is better?

    At most we can evaluate against single snippets, but how? performance of the code? its elegance? lines of code?

    I really enjoyed reading your article, but I’d really like if you elaborate on how to achieve the programmers rating.

    • rslootma says:

      You’re right. I should elaborate this. Elo was designed to establish relative strenght of competing people. But sites like chesstempo.com show how to rate people without having them competing head to head. What happens there is that people get a puzzle to solve. If they solve it, they get rating points and the problem looses points. On the other hand, if they cannot solve the puzzle, they will lose points, and the problem will gain points. This way, you establish the difficulty of the problems and the people who solve them at the same time. If programmer elo is too difficult to realise, you can always ask people their level on project euler.

      • Arturo G. S. says:

        That’s very interesting! competing against puzzles.
        I’m not really convinced that such an evaluation can tell how good a programmer is in general, but in your simplification of programming being at least as hard as chess, it fits nicely.
        Also, as you said, it would help job interviewers to figure out someone’s skills.
        I haven’t heard of Project Euler before… looks like fun!

      • rslootma says:

        Caution: Project Euler is addictive!

  2. elo is an international rating. The vast majority of players in the US have a USCF rating instead. The calculation is slightly different but still the same idea. Probably should have mentioned both systems just to be inclusive. Rest of article is quite interesting and will provide material to ponder—always good!

    • rslootma says:

      Yes. the UK (still?) has an alternative system as well. elo still rules in popularity, but there are many other systems trying to achieve the same thing. There is even a competition of rating systems:
      kaggle.

  3. Franklin says:

    Regarding chess: a US National Master is rated 2200 (USCF, not FIDE) or higher, and there are about 1% of the rated chess population at that level. I achieved a USCF rating of 2057 at around age 15, then completely quit chess. 20 years later, in 2005, I started playing again. My first results in 2006 were miserable, and I was in danger of falling below 2000, but I started studying chess very seriously (for the first time in my life, actually), completely changed everything about how I played, and started getting better and better. By around 2007, I was rated 2197 and almost made Master, before I failed (for reasons I won’t go into here). The only point I want to make is that it was possible for someone nearing age 40 to make massive improvements at chess, and if I did not have higher priorities right now, I believe I know what I would need to do to get over 2200 confidently, fixing many serious holes in my knowledge, experience, and psychological flaws that prevented me from performing much better during my short-lived comeback to chess.

  4. While I think a rating system for programmers would be interesting, and the proposals made in these comments provide some interesting possibilities, I think being a good (professional) programmer embodies more than just coding skills. It also included being a good team-player, planner, teacher… And these are typically skills that are not that easy to measure in numbers.

  5. Mohammed says:

    I still think programming can’t be quantitatively measured in an accurate manner. I see programming as an art, not as a pure science(; not everyone is going to agree on that, especially entrepreneurs and business people engaged in the software industry), although it has many principles and foundations in science like mathematics and computer science. Programming is something that challenges the soul as well as the mind. You’ll find many programmers, in many cases, arguing about the philosophy behind some implementation or another, although both may exhibit comparable behaviour, performance, flexibility for extensions, …etc.

    How can you evaluate one programmer versus another? Fine, it isn’t an easy task. I should first consider the evaluator to be an experienced person in programming. The evaluator should follow his intuition and sixth sense and should rely on his experience and feelings. Yes, this sounds not scientific or quantitative, but experience represents abilities and skills that can’t be physically or accurately measured. And after all, there’s a personal touch. So one evaluator may like the way some programmer approaches and handles programming problems, while another evaluator mayn’t like that same way and prefer a programmer who follows another way, although theoretically both ways will lead to the solution.

  6. I’m a community blog curator for DZone (a content and link-sharing site for programmers). I wanted to talk with you about potentially featuring your blog on DZone’s content portals and maybe giving your blog and it’s content some exposure to a new audience. Send me an email and I’ll explain the details.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.