DigitalHobbit

Earlier today, TechCrunch’s poorly researched claim that Twitter is abandoning Ruby on Rails in favor of PHP or Java generated a lot of buzz in the Twitter and Ruby communities (the claim was later refuted by Twitter developer Evan Williams).

Of course, the article’s comments attracted the usual, ignorant TechCrunch trolls. Most took the opportunity to pitch their framework of choice (such as PHP, Java, .NET, or Django), which they claimed would of course magically solve all of Twitter’s scalability issues.

I have to say I am appalled at this level of ignorance. People just don’t seem to realize that Twitter is a complex messaging application and that the front-end is only a relatively small aspect of it. Even if one particular front-end technology happens to be faster than another one (and admittedly Rails, as much as I like it, is not the fastest technology out there), this fact is bound to be negligible compared to the real challenges in scaling the back-end, starting with the database (trust me, like many developers I’ve learned this the hard way ). Even for a typical web application (which Twitter is not), there are many performance improvements than can be implemented at that level (such as leveraging database replicas to separate writes from reads, or utilizing Memcached to cache queries and other data), all of which can be applied equally well to any front-end framework.

I’m not saying that it does not make sense to consider other technologies (there might very well be a breaking point at which it makes sense to evaluate Java or even rewriting parts of the system in C/C++), but in my opinion this should be considered a cost-savings measure when the application reaches a scale at which the cost of hardware far outweighs any savings due to increased developer productivity (think Google), and not a magic bullet for solving fundamental scalability issues (performance != scalability!)

One of the real difficulties in scaling Twitter lies in the fact that all Twitter hits are completely personalized and need to return fresh data, making it difficult to fully leverage caching. Also, since Twitter is a social application and the returned data is generated by each user’s social graph, there is no straightforward way to shard the database by user, as one might be able to do in a typical e-commerce or enterprise application (or pretty much any non-social app…). Without knowing more about Twitter’s internal architecture and their actual profiling results, it would be foolish of me to make any concrete recommendations — particularly silly ones like “Use technology XYZ, it will magically solve all your problems!” Too bad many of the developers out there don’t seem to realize this…

I just got back from an ice cream social with Ruby on Rails creator David Heinemeier Hansson (aka DHH). I had come across his blog post about this casual event last week and figured I’d make my way over there, particularly since the location (Michael’s Gelato & Cafe) in downtown Palo Alto is only 5 minutes from my house.

There were probably about 20 other people (mainly Rails developers) at the event, and since the space was pretty small and we didn’t have a dedicated room, it was initially difficult to actually speak with David. But towards the end of the evening the group started getting smaller, which made it easier to participate in the conversation.

In terms of what David had to say, I did not catch too many noteworthy items that are not already known in Ruby on Rails circles. But a few things seem to be key to understanding both the history and the future of Rails:

1. David created Rails to solve his specific problems. It wasn’t meant to directly solve every possible problem out there, and if there are useful features missing from it that simply means that he (or presumably the rest of the core team, now that it has become a larger community project) has not needed that particular functionality, not that it would not be useful.

2. Rails is (as we all know) a very opinionated framework. It makes a lot of reasonable assumptions about various things (such as the names of “id” columns), which enables developers to accomplish a lot with very little code. However, all of these opinions are purely about the internals of the system and therefore relevant for developers, but never affect the end user in any way. David is strongly against including any features in Rails that would result in any default application flows or otherwise affect the actual behavior of the application. That’s why Rails still does not (and probably will never) include a built-in authentication system.

Otherwise, he seems pretty indifferent about a lot of the details about how people are working with Rails. For example, Ruby and Ruby on Rails’ performance was good enough for him several years ago, so this area has never been too much of a concern of his. He also doesn’t feel strongly about using Test::Unit vs. RSpec. In his opinion, Test::Unit is good enough, and actually easier to understand for someone who does not have much experience with testing, whereas RSpec probably has more of a learning curve (and he did not seem to excited about the particular syntax). He felt that the most important thing that BDD and RSpec brought to the table is the “should” keyword, and that changing Rails to allow this in test method names made a big enough difference in readability and helped enforce the convention of using a single assertion for each test case. (I actually was not aware that you can use “should” in Rails test names; I’ll have to look into this.)

I probably missed a bunch of other nuggets of wisdom because the room was a bit noisy and crowded, but I’m sure that all of these have been posted on various blogs and mentioned in various keynote speeches before.

Anyway, it was nice to meet David in person. He seems like a great guy, and I definitely appreciate that he took the time to meet with a small fraction of his user base tonight. It’s refreshing to work with technologies that are owned and driven by real-world people like this, rather than large corporations that have lost touch with their user base.

As some of you may know, I have decided to leave Google and go back into the startup world. Friday was my last day at Google, and even though I normally don’t blog much about my job, I figured I was due for an update.

First off, Google is an amazing company. Especially for a company of this size (and impact), it is highly impressive that they have managed to maintain this kind of work environment, company culture, and integrity. On the most basic level, there are all the perks, from great health benefits to free food (there are about 15 cafeterias on the Mountain View campus alone, many of which offer breakfast, lunch, and dinner), micro-kitchens with free snacks (including fresh fruit or fresh cut carrots and celery), on-campus gyms and a beach volleyball field, and more. Another great thing are the many tech talks (usually there are at least a couple every day), which feature both internal and external speakers. There are also many opportunities for training. In the one year that I have worked at Google, I had the opportunity to take a one-day class on Agile Estimating and Planning, as well as an excellent four-day workshop on Design Patterns and Refactoring, and numerous Google and product specific classes. Then there are the various off-sites and team building events, which probably take up another good week each year. Not many companies send their employees to Disneyland for three days… And of course there’s Google itself, with its various products that have become so ubiquitous and synonymous with the web that life would be difficult to imagine without them. Anybody remember the web before Google Search came along? Let me give you a hint: it sucked! But even newer products like Gmail, Google Reader, or Calendar have caught on quickly and become established as best-of-breed applications, and I definitely felt proud to work at the company that has built all of these applications. And speaking of work, the actual work environment is also nice in many ways: Every engineer gets either two 24″ or one 30″ monitor, as well as a company laptop (either MacBook Pro or Thinkpad). Depending on which project you work on, you might get to work with innovative internal tools and frameworks (such as BigTable), and I have definitely developed an entirely different perspective on scale, which humbles any project I’ve worked on before. Then there’s the community. In general, Googlers are a great bunch, and very smart. There are internal mailing lists on pretty much any subject, and you can pretty much guarantee that you will be able to get a solid answer to whatever question you might have (work or non-work related). In fact, this reminded me quite a bit of Usenet, back when it was still popular and usable, and not totally overrun with idiots. Of course, keeping up with all of this can become quite a time sink as well… Google is highly engineering-driven, and engineers enjoy a lot of trust and power, which is a very different and refreshing experience from working at a more product-driven company.

So if everything is so great at Google, why am I stupid enough to leave? And part of me does indeed feel a bit guilty about not being able to fully appreciate and enjoy working at Google — after all, there are many people out there in the world that have dreary and monotonous minimum-wage jobs, without any benefits or perks to speak of. But in the end, I have realized that I am just much more of a startup person than a big-company person. Perks and everything are great, but this is ultimately not what motivates me. At an early stage startup, every single individual has a tremendous impact on the company (good or bad…), along with a much broader set of responsibilities (everybody has to wear many hats). Then, there’s the pioneering spirit, which is extremely energizing and contagious. These days, it seems like a lot of the true innovations are made at small startups, which have the benefit of being orders of magnitude times more agile and efficient than a large company will ever be. Sure, many ideas don’t go anywhere, but every once in a while, something new comes along that leaves a big footprint (and let’s not forget that even Google started out like this). Last not least, there is of course a significantly bigger upside to working at a startup. Of course the harsh truth is that most startups fail, but at least there is that 1 in a 10 chance of being tremendously successful (and the sense of actually being able to contribute to this chance). As a recent Google employee, I would have never gotten rich there, even if the stock had doubled or tripled in price.

There are a few other Google-specific problems I should mention as well. For one thing, it is unlikely to initially be able to work in an area that one is passionate about. Many of the Google products are exciting, but unfortunately I was unable to be passionate about my particular product area. That is not to say that there weren’t any interesting aspects about it, and I do have a lot of respect for the team I worked with. Overall this is less of a problem later, as it is generally encouraged to switch projects every 1-2 years, but this first year makes a big difference, particularly for experienced engineers that have a good understanding of what kind of things they enjoy working on (or perhaps more importantly, don’t enjoy working on) or what kind of environments are a good match. I feel that the hiring process should be improved to better take this into consideration, although this is admittedly a difficult logistical problem at Google’s scale. Another scale-related problem: Due to the sheer size of the code base and the vast number of Google-specific tools and frameworks, it also takes a very long time to learn how to actually become productive at Google, which can be frustrating at times.

But overall, I feel privileged that I had the chance to work at Google. I’m sure they will still be around for a long time, and 20 years from now I will be able to tell my grandchildren “Oh, Google, yes, I worked there once…”. I will certainly miss the Google campus, which had the vibrant feel of a University campus. I will also miss many things about the Google culture, and hopefully be able to take many of these inspirations with me into my future career.

Which brings me to my new job, which I am enormously excited about starting this Monday. I can’t say too much, as the company is just getting founded and still in stealth mode, but I am going to be a co-founder of a brand-new startup in the Social Networking / Mobile space, two areas I am very interested in. I have worked at small startups, but so far I’ve never had the opportunity to come in on the ground floor like this, so this should be a great adventure. My role is going to be that of Director of Engineering, but I expect to be wearing a lot of hats, ranging from architecture and implementation to being involved in the product direction, taking care of hosting and other IT stuff, and ultimately building a great team and helping define the company culture (although we will initially be a small core team). We will be working with some exciting technologies, including Ruby and a sprinkling of Ruby on Rails, which I am strongly looking forward to as well. Besides using a bit of Ruby at my previous job, I have mostly played with Ruby in my spare time, and I am glad about being able to finally use it again at my day job as well (though we will be pragmatic and use whatever tool makes most sense for the job at hand). I think I am officially done with Java at this point (but never say never… I am pretty sure I will at least use it again as a platform at some point in the future).

Well, this blog post turned out a bit longer than expected. But don’t worry, I am sure I will be busy enough at my startup that I won’t have a chance to bug you with any additional long posts for a while.

After having signed up for an Evernote beta invite a while ago, I finally received one this week.

Evernote is a next generation note taking service. What sets it apart from similar applications are the numerous ways of entering as well as retrieving notes. First of all there is the Evernote Website. It is simple and straightforward, and seems to be very responsive. You can create as many notebooks as you want. Each notebook acts like a folder that contains individual notes as files, and can be viewed using different perspectives (such as thumbnail view or list view), much like a native file browser application like Finder or Windows Explorer.

Notes can be entered and edited on the website, but there are several alternatives. For example, you can email a note to a special email address that is generated for you when you sign up. You can also use this mechanism to send notes from your mobile phone, using your provider’s SMS-Email gateway. However, I have been running into some technical issues when I attempted this from my mobile phone (I’m getting the following error message: “Your MSG could not be DELIVERED because InvalidPduContent”), so I need to look into this. Curiously, sending a picture via MMS worked just fine. Speaking of pictures: Evernote’s support for images is pretty amazing. It uses advanced text recognition to extract text from images, such that it becomes searchable. I have tested this with a low quality photo of a poster snapped from my mobile phone, and it worked as advertised. Very impressive!

You can also use a nifty bookmarklet that allows you to either submit an entire webpage to Evernote, or just the text that you have selected.

But perhaps most importantly, Evernote has a downloadable client app for both Windows and Mac. I have only tested the Mac client, which was released very recently. It seems to lag slightly behind its Windows counterpart in terms of features, but on the positive side it looks and feels like a proper Mac application rather than a quick port. It relies on a Sync feature to synchronize notes between the server and the client (either manually or in a configurable interval). Otherwise, the client offers pretty much the same functionality as the website. In addition, it comes with a clipping service that shows up in the menu bar (and registers some convenient keyboard shortcuts) and allows you to easily submit any copied text to the client, or even to clip a screenshot.

Evernote also has many ways to browse notes. It has a search box that allows you to find notes using full text search, but also using other criteria. For example, notes can be tagged (much like emails in Gmail), after which you can search for them by tag (or just click on the tag in the navigation bar). Notes can also be located via various attributes, such as creation or modification date, source (website, email, mobile, etc.), or whether or not they contain images or audio. This works from both the website or the native client. Evernote also has a mobile website, which I have not tested. This should be convenient when you need to look up a note while you’re on the road.

Overall, I am pretty impressed with Evernote. I have played with various note taking applications, and this comes very close to perfect in terms of features. I often have ideas that I need to capture, and right now I’m using a personal wiki for this purpose. However, I would prefer to work in a native application when possible, since this is generally more convenient. The combination of native client and website for Evernote is quite powerful, with a well-implemented synchronization mechanism. I just wish there were some wiki-like features, such as easy linking between individual notes. But the most important drawback is the lack of formatting options. The website only supports straight text entry, with no formatting at all. The Mac client supports rudimentary formatting (such as font, color, bold, italic, underline, alignment), but unfortunately this formatting appears to be lost when the note is uploaded to the website. The thumbnail view does show the properly formatted note, but the full view does not. And any edits on the website completely reset any formatting that might have been applied on the client. I don’t require any sophisticated formatting, but at the minimum I would need support bullet lists (which neither the Mac client nor the website supports), headings, and emphasis.

As it is right now, Evernote seems like a somewhat useful scratch pad to collect short notes, web clippings, etc. Particularly the mobile features might come in handy. With a bit of additional work, I think it could be extended to become a more comprehensive solution for organizing information, but right now I am going to continue using my wiki for this purpose.

If you would like to see it in action, there is a short Evernote Screencast.

The distributed Git version control system has been gaining a lot of traction lately. In contrast to traditional, centralized version control systems like CVS or Subversion, Git enables developers to easily fork each other’s repositories, pull patches from each other, etc. Even though I have not had a chance to use Git beyond pulling down some open source projects (such as Merb), it is clear that this enables much richer collaboration and development dynamics.

In particular, github seems to have quickly become a favorite service for hosting Ruby projects. github is still in invite-only beta, but it already looks very impressive. For example, forking an existing repository that is hosted on github only takes a single mouse click, and sending a pull request to the master repository or other forks is just as easy. Each repository has an RSS feed and a wiki, which turns github into a full project hosting service as opposed to just a version control service. In fact, github has been described as a social network for hackers. In addition, github offers convenient features such as tarball downloads.

The developers had announced that public open source projects would always remain free. Today, github unveiled the detailed pricing plans. Overall, the plans seem very reasonable. Each plan includes unlimited public repositories and public collaborators, but a limited disk space and number of private repositories and private collaborators (depending on the plan). Personally, I would have hoped for the smallest commercial plan (Micro, which includes 5 private repositories and a single private collaborator) to cost less than $7, perhaps $3 - $5. Many developers that work on non-open-source projects in their spare time would presumably be interested in such a plan, but $7 seems slightly steep, given that you can get a Dreamhost hosting plan for $10 per month, which supports Subversion, website hosting, shell access, and, with some effort, even Git hosting. Obviously the $7 github plan offers a significantly better user experience, but I am not sure if I could justify the expense as a small developer working on non (or not yet, anyways) profitable projects in their spare time.

But ultimately github’s main focus is clearly on open source projects, and it sure is great to have a modern alternative to Subversion based project repositories like SourceForge or RubyForge. In fact I went ahead and created my own forks of the various Merb sub-projects, in case I want to play with some Merb changes or submit additional patches. I also have some ideas (or even unfinished code) for other projects that I might end up moving to github as open source projects.

I believe that one of the smaller commercial plans would also be an excellent alternative for a small startup that wants to avoid the hardware and IT headaches of dealing with their own setup in the initial phases. In fact, by combining github with a hosted issue tracking service like Lighthouse, a VPS hosting service like SliceHost, and Google Apps for email, calendaring, document collaboration, and now also Google Sites for a wiki or intranet, it should make it much easier to bootstrap a new company without a massive IT investment.

I have a few github invites, so leave a comment if you’d like one.

This morning, Apple finally announced details about the long-awaited iPhone SDK. The SDK is available for download starting today, and it sounds pretty intriguing!

It comes with a full application stack that includes a custom version of Cocoa (Cocoa Touch), and APIs to access all the iPhone specific features, including hardware-accelerated 3D graphics, the accelerometer, camera, contacts, as well as location-awareness. This sounds a lot more complete that I expected.

The tools side is covered very well, also: The SDK includes a custom version of the Xcode IDE, Interface Builder, Instruments (a profiler), and an iPhone Simulator, so you should be able to develop iPhone apps using essentially the same toolset as for regular OSX applications.

As expected, Apple will control the distribution for all iPhone applications through their App Store. Applications can be purchased and downloaded right from the phone, or via iTunes. Developers get complete control over the pricing, and Apple offers a 70/30 revenue split (70% for the developer), which seems pretty reasonable. Developers will also be able to make their applications available for free, in which case Apple takes no cut. It sounds like Apple will generally allow any application to be published, but there are some (mostly obvious) limitations (such as pornography, VOIP over cellular connections, or anything illegal).

The SDK itself is available for free on Apple’s developer website, although you have to have to register as an iPhone developer. However, in order to be able to submit applications to the App Store, developers need to join the $99 iPhone Developer Program. Apparently the developer program will initially be available on a limited basis, but I assume that most developers will be accepted in to the program later. Since the App Store is not scheduled to launch until June, this does not seem like a major issue, and in the mean time the iPhone Simulator that is included in the SDK looks to be a pretty exact copy of the actual iPhone, so this will have to do.

Users will need a firmware upgrade in order to run custom applications. I assume that this will be released around the same time as the App Store (although hopefully a bit sooner, for testing purposes). iPhone users will be able to upgrade for free, but iPod Touch users (like myself) will be charged a nominal fee (the exact amount has not been announced), supposedly due to differences in the way Apple accounts for the iPod Touch, since it does not come with a subscription plan.

The SDK download proved to be a major pain, since Apple’s website was hopelessly overloaded and it was impossible to even get the download to begin. I eventually got lucky and found the SDK on BitTorrent. Apple really should have made the 2GB download available on BitTorrent in the first place…

Anyway, I’m planning to play with the SDK a bit over the next few weeks or so. One thing I was hoping for is that the SDK might include support for RubyCocoa, since I’m not all that thrilled about the prospect of using Objective-C (or any C-derived language for that matter…), but based on my initial research things aren’t looking promising. Ruby is supported on jailbroken iPhones, so it seems like it should be feasible to support RubyCocoa as well. This will be the first thing

Oh, there were a few cool game demos as well, including an iPhone version of Spore! I am most interested in building my own iPhone apps, but I am also excited to see what kind of applications and games will be released by third parties. I am currently using several useful applications on my jailbroken iPhone Touch (such as an ebook reader and an unit converter), and it seems like it should be reasonably straightforward to port these to the official SDK, so I bet that most of these applications will be available soon, either for free or for a moderate price.

For more details, check out the following links:

• Engadget’s live blog post from Apple’s iPhone SDK press conference
• Another live blog post, on TechCrunch
• Full QuickTime video of the event
• A good summary on TechCrunch

I was deeply saddened to find out that Gary Gygax passed away today, after struggling with health issues for several years. Gary Gygax co-created the Dungeons & Dragons role-playing game, and thereby invented the entire role-playing game genre.

Like any good geek, I grew up playing role-playing games during my teens. Even though we did not actually play that much D&D in particular, all the RPGs we played were directly descended from it, and none of them would exist without Gary Gygax. The first RPG I ever played (this must have been around 1986) was Das Schwarze Auge (The Dark Eye), a relatively simple German role-playing game that was strongly inspired by D&D. After my friends and I grew out of this, we explored many other games. Some of the ones I fondly remember are RuneQuest, Traveller, Warhammer Fantasy Roleplay (probably my all-time favorite role-playing game; not to be confused with the Warhammer table-top game), Paranoia (“Trust The Computer. The Computer is Your Friend.”), Vampire: The Masquerade, and KULT. We even participated in a Live action role-playing game once (yes, and I’m not ashamed to admit it. ). We hung out at our local game store (luckily the city I grew up in was big and progressive enough to actually have a game store that stocked RPGs) and bought dice (the store owner would always give us the option to role each die against him for double-or-nothing) and miniatures.

There were phases where we played every weekend, and phases where we played less, especially during the 90s, as most of us started to become more interested in going out to bars and clubs (and the opposite sex…). Still, we played at least occasionally during college, until I got married and moved to the US in 1998 (gosh, has it really been almost 10 years?) and started my first job.

For some reason I never got back into role-playing after that, although the thought to make an effort to find a new group did occur to me a few times. But I still like to nostalgically think about the great, both intellectually and emotionally stimulating (and simply fun) times we’ve had playing RPGs. Many of my German friends are still regularly playing role-playing games these days.

And of course D&D has similarly influenced computer games, and many of my favorite computer and video games (like The Bard’s Tale, the Ultima series, or the Elder Scrolls series, not to mention the myriads of AD&D computer games) would not exist without Gary Gygax.

For all of this I am truly grateful. Gary Gygax, rest in peace.

A short story about how the music industry has changed (though not all of them have recognized this yet).

Earlier this evening, a Twitter message alerted me to the Ghosts instrumental album that was released by Nine Inch Nails today, completely announced. I had been curious about Trent Reznor’s plans, as he had previously voiced his discontent with (and freedom from) the music industry.

So when I checked out Nine Inch Nails’ Ghosts website, I was not disappointed. First of all, there’s the music itself. Ghosts I - IV is an instrumental music collection - a first for NIN. I was able to listen to it right on the website, and based on my initial impression, this is quite possibly the best material that NIN has released in a long time (I wasn’t a big fan of any of their albums after the 1994 release of The Downward Spiral). It certainly sounds like great music to code to, so I’m uploading it to my iPod for tomorrow.

But more important is how the music is being made available. The first 9 tracks (Ghosts I) are available as a free download, in 320kbps mp3s, as well as a 40-page PDF booklet. The full 36 tracks (Ghosts I - IV) can be purchased on the website for the extremely fair price of $5. A 2 CD set will be released for $10 on April 8 for those that prefer physical media. In addition, a deluxe edition and ultra-deluxe limited edition are also available for $75 and $300 respectively for those that have way too much money to spare…

On a slightly sour note, the NIN website appears to have been hammered when I tried to download Ghosts I. But a few tweets later I found out that Ghosts is also available for download on Amazon (for the same price of $5), which suffered from no such issues.

What is interesting is that it’s not just the music distribution mechanism that is changing. It is also how music is being discovered and marketed. As far as I am aware, there was no marketing around this release, or even any sort of formal announcement. Instead, music is being discovered virally, by word of mouth, blogs, and even Twitter. Interesting times!

Today, I attended a tech talk by ThoughtWorks’ Ola Bini on JRuby (JRuby Wikipedia entry here).

I keep hearing great things about JRuby (even Matz has good things to say about it), and it’s nice to hear this from the horse’s mouth as well (Ola is one of the JRuby core developers). Apparently the latest version is highly compatible with Ruby 1.8.6, although applications that use native C extensions are not supported. It’s also impressive how far JRuby has come in terms of performance. Until about a year or so ago, it was still significantly slower than the standard Ruby 1.8 interpreter. Not only have they caught up since then, but the latest JRuby version running on the latest JVM (in server mode) is actually several times faster than the Ruby 1.9 interpreter (aka YARV), which in turn is several times faster than Ruby 1.8 (apparently up to 50x in benchmarks, but more like 1.5x on average). That’s quite an achievement!

The tight integration between JRuby and Java is very impressive. You can explore this by using jirb, JRuby’s version of irb. In this console, you can use all the standard Ruby classes and constructs, as well as Java classes and libraries (Ola demoed this by creating a Swing window and a button, with a listener implemented in Ruby). It really seems quite seamless and powerful.

Due to time constraints, Ola did not talk much about JRuby and Rails (I suppose his JRuby on Rails book would describe this in sufficient detail), but it sounds like it is pretty well supported by now. In fact, ThoughtWorks is shipping Mingle, a shrink-wrap agile project collaboration tool built in Rails and deployed on JRuby. Ola also cited several other large scale JRuby deployments at Oracle and SUN. In many cases, using JRuby proved to be a way in the door for development teams that wanted to use Ruby but whose IT organizations had standardized on Java and were not prepared to support another runtime.

Ola mentioned the GlassFish gem as a convenient way to deploy Rails applications using JRuby, although he recommended to mainly use it for development and testing, but not necessarily for production. There are other tools that create a WAR file for a Rails application, which can then be deployed on any standard Java application server. I did not catch the name of the tools that Ola mentioned, but Goldspike appears to be one of them.

About 2 1/2 years ago, I prototyped a scripting engine that ran within the Java based backend for our multiplayer mobile game. At the time, I dismissed JRuby in favor of Groovy and BeanShell (we tried both to see which one we liked better), because JRuby was by far the slowest scripting language to run in Java. Today, there’s no doubt I would have strongly considered using JRuby instead.

It’s weird… somehow Java still feels like a heavyweight system to me, with its compiler, JVM, app servers, etc. But in reality, JRuby is now not only faster than the native Ruby interpreter, but also has a smaller memory footprint. And because it supports proper threads (native Java threads), you should actually have even better opportunities for optimizing the deployment of your Ruby web applications (rather than having to juggle a bunch of Mongrels…)

I will definitely have to take a closer look at JRuby and perhaps try it out with Merb. According to Ola, Merb is supported even though it does use some native C extensions. Sweet!

Update: Ola’s JRuby tech talk is on Youtube now.

I had the pleasure of attending a presentation by Yukihiro Matsumoto (aka “Matz”) on Ruby 1.9 today. I had heard that he is a very nice guy, and he definitely came across as very charismatic. He is not particularly fluid in English, and in previous interviews I had heard with him, he often used an interpreter. But I was impressed that he actually delivered the presentation (and Q&A) in English.

He mainly talked about Ruby 1.9 and some of the improvements as well as incompatibilities that it is going to bring. I am not going to summarize those here, but a good summary appears to be here (note: this site appears to be down right now; use the Google cached version instead).

One of the things that stuck out for me was that Enumerator is now an inherent part of the core and no longer a module that needs to be required. Even more importantly though, Enumerators can now be chained together. For example, you were previously able to call each_with_index to iterate over the elements of a collection and their corresponding index at the same time, but you could not apply this to other methods provided by the Enumerable module, such as map. In Ruby 1.9, you can achieve this by chaining several Enumerators like this: [1,2,3,4].map.with_index {|x, i| ...}.

And of course there’s the new Ruby VM (YARV), which is supposed to bring some speed increases (up to 50x in extreme cases, but Matz expects more like 1.5x-2x on average). But then again, there are other related projects such as JRuby or Rubinius.

Anyway, I have not really followed Ruby 1.9 development much, but I might have to check it out a bit. Oh, it also sounds like Ruby 2.0 is not that far off any more (probably coming out some time this year)…

Update: Matz’s talk is on Youtube now. You can even see me (or at least the back of my head) at the beginning and end of the talk, in the second row, towards the right.

Heroku recently announced a public beta. Heroku is a Rails hosting service with a twist: They offer a fully integrated development and hosting environment. You develop your application using a web based IDE, and it gets automatically deployed to Amazon EC2.

The IDE works surprisingly well. You get a typical file browser that allows you to create new files / folders or open, rename, move, or delete existing files /folders. The actual editor performs syntax highlighting and automatic indentation, but does not seem to offer any other convenience features (like auto completion or snippets). Whenever you save a file, any updates are automatically deployed and appear on your website. You can choose whether your website is private (only accessible to yourself and other users you have authorized) or public (accessible to anyone), although there does not currently seem to be a way to disable the Heroku toolbar that shows up at the bottom of the page.

In addition to editing your application using the Heroku website, you can both import and export your application as a tarball. You can also take snapshots at any time, and revert to these later, which should come in handy before implementing experimental changes but does not replace actual version control. The Heroku team has announced plans to implement some sort of version control integration in the future, possibly using Subversion. This would be a major boon, as it would free developers to use whatever tools they like, without having to use the cumbersome import / export process (not to mention the inherent benefits of using version control).

The deployment of your application is handled in a completely transparent fashion. Behind the scenes, Heroku leverages Amazon EC2 (Elastic Compute Cloud), which is supposed to allow your application to automatically scale as needed.

Right now, beta accounts are free. Heroku is still finalizing the pricing structure, but they have stated that they’re hoping to implement a model similar to an electrical company, where you are charged based on the resources you have consumed. In this case, resources would be your computing cycles, bandwidth, database storage, etc. They have also stated that they are planning to keep small and low-bandwidth applications free forever. If the seamless scalability that Heroku promises actually proves to work out, this would be an ideal way to start a new application, without having to commit any money to hosting plans upfront.

At this time Heroku is invitation-only, so you can either sign up on their website (although I have found that it may take a week or so to get an invite) or ask an existing member for an invitation. The Heroku mailing list works great for this purpose, or feel free to leave a comment on my blog if you’d like an invite.

Note that beta accounts are currently restricted from initiating any outgoing connections, which makes it impossible to build many types of applications (such as fully integrated Facebook apps). You can however request for your application to be blessed, which will allow you to enter the pilot program for paid apps that support outgoing connections, custom domain names, and higher bandwidth. Heroku requests that the application be in use and under active development when requesting the blessing, although I am not sure this makes sense (certain types of apps might not be able to run at all in a meaningful fashion until they are allowed to open outgoing connections). But I am sure that the Heroku developers will use their best judgement when making this call.

If you’re a Rails developer, you should definitely check it out! I wonder if they are going to support other frameworks (such as Merb) in the future…

Merb 0.9.0 was released 5 days ago. It looks like the download instructions on the homepage have also been updated for 0.9.0. You can now install it as a gem or from source, by checking out the code from the Merb Git repository.

Aside from a more modular design, Merb 0.9 also introduces a more flexible application layout. Previously, the layout was essentially the same as for a Rails app. This is still the default, but if you create your app using merb-gen my_app --very-flat or --flat, you get a significantly smaller layout that might be better suited for small applications.

In fact, a --very-flat application consists of a single file with the following contents:

  Merb::Router.prepare do |r|
    r.match('/').to(:controller => 'my_app', :action =>'index')
  end

  class MyApp < Merb::Controller
    def index
      "hi"
    end
  end

  Merb::Config.use { |c|
    c[:framework]           = {},
    c[:session_store]       = 'none',
    c[:exception_details]   = true
  }

That’s not quite as tight as a true micro framework like Sinatra, but it’s pretty close. For very small applications, I have occasionally used Ruby’s built-in WEBrick server, but a very flat Merb app seems quite a bit simpler, as well as more future-proof, as this can be easily transformed into a full app with a more sophisticated layout if the app grows beyond its initial minimal scope.

Oh, by the way: I have contributed my first (admittedly tiny) patch to Merb (part 1, part 2), which adds support for dynamic layouts by allowing you to write something like layout :determine_layout in your controller to specify a method that is used to dynamically determine the layout to use, rather than hardcoding it on the controller level or having to pass it in each render call (yes, this works just like in Rails).

Yesterday I set up Markdown support for my blog, as I was getting tired of typing <li>, <a href="...">, etc. I’ve had my blog for almost four years now, and I can’t believe that I have not done this before. It finally allows me to focus on the actual content while I’m writing a blog post, without having to contort my fingers to type the necessary HTML tags.

I’m using the PHP Markdown script, which can be used standalone but also functions as a WordPress plugin. You simply copy the script into your wp-content\plugins folder, activate it on the Plugins tab, and you’re all set.

I am currently evaluating MarsEdit, a very nice blogging application for OSX. When configuring your blog in MarsEdit, you can set the Preview Text Filter to Markdown, which will give you a nice realtime preview of your post, the same way it is going to appear on your blog.

See here for a brief overview of Markdown syntax, and here for the full Markdown syntax reference.

Earlier today, Alex Payne blogged about side projects, which prompted me to write down my own thoughts on this matter, as I had been planning a blog post on this subject for a while. Alex states some of the benefits of having a side project, such as keeping you learning, dealing with different types of problems than during the day job, having fun, making new friends, and potentially even being profitable (all of which I completely agree with). Alex also mentions that side projects are a way of showing that you care and that they ask candidates about this in their job interviews at Twitter.

I tend to ask the same question when interviewing engineering candidates, and it’s definitely a big plus when a candidate starts to excitedly talk about a side project they’re working on. However, I would tend to generalize this a bit further. Ultimately, I am looking for a strong sign of passion, but there are several equally relevant things that a developer might be passionate about:

• Coding, which might manifest itself in a side project (either a project of their own, or contributing to another project, perhaps open source).
• Technology. Perhaps they like to examine the latest web or desktop application frameworks, testing tools, or programming languages. They might not be working on a concrete project in their spare time, but they clearly like to follow what’s going on in the software development arena. They probably get their actual coding fix at work, but being aware of the technologies out there will broaden their horizon and help them in making informed decisions.
• Software Engineering, i.e. things like best practices, process (such as Agile Development), or design patterns. They might not be interested in working on an actual spare time project (or lacking the time to do so due to family obligations), but they love to keep up with the software engineering discipline by reading books and blogs, discussing these subjects with their peers, etc.
• A Domain relevant to the job. For example, my previous job was at a mobile games company, and being able to convey passion (and ideally knowledge) about either the mobile industry or gaming was definitely an advantage.

Personally, I tend to go through phases. I always have ideas about various potential projects, and I maintain a personal wiki that allows me to capture and flesh out these ideas at any time. Sometimes an idea from this list seems exciting enough (either immediately or after maturing for a while) that I start playing with it by writing some code, although admittedly very few of these projects end up seeing the light of day. At other times, I spend more time exploring various tools and frameworks, or perhaps learning a new programming language. Granted, this is often triggered by a concrete project, but I often end up embarking on a mission to try out various new web or desktop frameworks, etc., and the concrete project gets left by the wayside for a while. Yet other times (for example during times in the past were I had very little spare time due to a long commute and a newborn daughter) I have spent very little time in front of the computer and a lot of time reading software engineering books instead. But a domain that I can relate to and get excited about is generally something I look for in all my jobs. Life is too short to work on boring applications…

It can be difficult during the short time frame of an interview or even a phone screen, but it’s worth trying to figure out what makes a candidate tick. An actual side project is a huge plus in my opinion. If they don’t have a side project (or at least a good idea for a spare time project that they would work on if they had the time), and they don’t manage to convey any enthusiasm for technology or the domain of the position they’re interviewing for, that’s definitely a big red flag. I am always surprised when I come across such a candidate, but it happens way more often than I would expect…

So where was I? Ah yes, in yesterday’s post I left off with me going back to Ruby for my personal projects.

My standard choice for web development has been Ruby on Rails for a while now (although I haven’t had a chance to try Rails 2.0 yet). However, lately I’ve been playing with Merb. Merb is essentially a lightweight Rails replacement, originally built to address some specific (largely bloat and performance related) concerns. For Rails developers, Merb will seem quite familiar (at least superficially), as it uses the same application layout as Rails, with the usual directories for controllers and views, etc. But unlike Rails, the core framework is very lightweight, with additional functionality being provided by plugins. Unlike Rails’ slightly awkward plugin framework, Merb plugins are actually just standard Ruby gems that can be installed either as part of the Rails installation or inside the Merb application. In particular, Merb does not include an ORM framework for its model layer. There is a plugin that adds support for ActiveRecord, the ORM framework that Rails uses as well. But as an alternative, Merb also offers plugins for DataMapper and Sequel. And of course very simple applications might not need a database at all.

I am particularly interested in DataMapper, which offers several improvements over ActiveRecord. Most importantly, it is thread-safe. But it also moves the definition of a model’s properties from the database into the Ruby class. The fact that ActiveRecord infers all attributes from the database has always seemed slightly unnatural to me, so moving the definition into Ruby code makes more sense to me. I do however miss ActiveRecord’s support for migrations (anyone remember when ActiveRecord did not have this feature yet and how big a difference it made when it was released?).

One thing I should mention is that Merb is currently undergoing a major transformation. The 0.5 version (which the Merb website still refers to) has been end-of-lifed (0.5.3 was the final release), and development has shifted to the 0.9 version. 0.9 takes an even more radical approach: The existing Merb gem will be split into merb-core (the actual core, very lightweight), merb-more (support for generators, mailers, the Haml template language, etc.), and merb-plugins (things like ORM framework support). Merb is also adopting the Rack interface, which is becoming the standard way for Ruby web applications to plug into any compatible web servers (similar to Python’s WSGI).

The 0.9 release is expected to come out any day now, and you can follow its development on github (Merb shifted from Subversion to Git) and the Lighthouse bug tracker. If you want to get more involved with Merb, Michael Ivey wrote an interesting two-part article on contributing to Merb (and part two). There is also a pretty active and helpful community on IRC (#merb on Freenode). Merb was originally implemented by Ezra Zygmuntowicz, who founded the Engine Yard hosting service, and with the recent investment into Engine Yard, Merb (as well as other Engine Yard supported apps such as Rubinius) now has several dedicated developers working on it and seems to have a bright future.

Right now I am using Merb to build a simple Facebook application. I’ll blog more about this later, but so far it seems like a good choice for this type of application. The transition from Rails is relatively easy (although this is likely more complicated if you are relying on a lot of the Rails magic such as the various helper classes), and the Merb community is very helpful. Deploying a Merb app is not much different from deploying a Rails app. I am currently deploying my Merb app-in-progress via Capistrano to SliceHost and running on Nginx and Mongrel (mainly following these instructions).

In my current application, I am also giving Haml a shot for the first time. It takes some getting used to (after being indoctrinated with HTML for over a decade…), but you quickly learn to appreciate its elegance and conciseness (it really makes you realize how much redundancy HTML carries around).

One thing I haven’t tried yet is RSpec (so far I’ve been using good old-fashioned Test::Unit), but the trend seems to go towards RSpec, so I’ll have to check it out one of these days. It’s amazing how quickly these technologies shift in the very dynamic Ruby world, and sometimes quite hard to keep up when I’m only using it occasionally in my spare time…

It’s been a while since I last blogged about any technologies I’ve been playing with, so I figured I’m due for an update. I am going to split this into a two-part series: First I will play catch up and talk about the technologies that I’ve played with over the past 6 months or so. In the second article, I will go over some of the technologies that I’m currently exploring, or that I am planning to explore next.

Last Summer, I was contemplating getting into Python, which I had used briefly at my job. I had been more of a Ruby fan up until then, but I figured I’d give Python a chance and try to focus on a single scripting language. Therefore, I decided to explore some of the more popular Python web frameworks. I am not going to go into great detail here, but below are some brief notes. Also keep in mind that it’s been about 6 months since I’ve played with these, and that some of them are evolving rapidly, so the situation has likely changed quite a bit.

• TurboGears combines several components (such as the SQLObject ORM library, the Kid templating system, and the MochKit JavaScript library) into a full web application stack. This seems very promising, but when I looked at TurboGears, its future appeared to be a bit uncertain, which is why I decided not to explore it in more detail. Since then, the team has announced that the 2.0 version is going to be based on the Pylons framework, but I am a bit unclear on what functionality TurboGears would add on top of Pylons (which itself combines multiple components into a web app stack).


• Pylons: I really like the philosophy of this framework. Instead of a full, tightly integrated stack, Pylons mainly consists of a lightweight core that integrates with other best-of-breed libraries to form a custom stack. Instead of being tied to a built-in ORM framework, Pylons supports either SQLAlchemy or SQLObject. It also supports many templating frameworks, such as Mako or Genshi, and of course AJAX helpers like Prototype or jQuery can be plugged in as well. I really wanted to like Pylons, but in my brief experiments things weren’t quite so easy to plug in as I had hoped, requiring modifications to various Pylons files (the instructions for which I had to hunt down on the web and in mailing lists). If it can overcome this limitation and evolve into a slightly more coherent framework where subcomponents can be cleanly picked “a la carte”, this would be quite promising. However, it looks like the last release (0.9.6.1) was in September of 2007, so there does not seem to be much momentum at this point…


• Django has been gaining a lot of traction and is being hailed as the “Ruby on Rails” for Python. In my brief experiments, it definitely seemed to have a lot of potential. Unlike Pylons, Django includes a full web application stack. It includes its own ORM support (pluggable ORM support would have been nice, as both SQLObject and SQLAlchemy seem to be powerful and mature) and a (deliberately) very limited template language, but other templating systems can apparently be plugged in as well. The documentation is pretty good (and the first Django book was published two months ago). It is however lacking some of the convenience of Rails that I have come to accept as standard, such as support for database migrations or easy deployment via Capistrano. The last official release (0.96) was in April of 2007, which is a while ago… Still, Django seems like a popular web framework in the Python community, and I might very well take another look at this at some point in the future.

This is where we get to the somewhat anticlimactic resolution of this post… Ultimately, I realized that I was happier working with Ruby and Ruby on Rails. Although Python is a perfectly fine language, I still prefer Ruby. And there are just too many convenient features that I had gotten used to in Rails (such as migrations and Capistrano) that made it difficult for me to pick one of the Python frameworks over Rails. Therefore, I decided to continue using Ruby and Ruby on Rails for my personal projects for the time being.

If you’re an experienced Python programmer, any of the frameworks above are certainly worth a look, though.

In my next post, I will go over some of the technologies that I am playing with right now, or planning to explore in the future.

Today, TechCrunch posted an article about a stealth job site that makes companies pay candidates to interview with them. NotchUp is currently in beta and invitation-only, but apparently the site will allow candidates to set their own price (although a calculator is provided that suggests a price based on experience, current salary, etc.), in an attempt to lure the kind of passive job seekers that many companies would be interested in finding.

This is certainly innovative, trying to solve a real problem, and I like the idea, but ultimately I have a hard time imagining this working. I don’t doubt at all that potential job seekers (even ones that are not actively looking for a job) will be tempted to put their resume up on NotchUp. But how are companies supposed to tell the true, previously passive rock-star candidates from the legions of mediocre candidates or even worse, candidates out to make a quick buck by exploiting the NotchUp system? It sounds like they are going to implement some sort of ratings system to prevent abuse, but I remain skeptical. I am also curious about the logistics. Most companies perform a phone screen before inviting candidates for an in-person interview. Do candidates get paid if they don’t pass the phone screen? The full amount or just a fraction? Or do only in-person interviews count (which would be open for abuse on the employer’s side by subjecting candidates to a longer and more thorough phone screen)?

If all these obstacles can be avoided, this might turn out to be interesting from the employers’ perspectives, although I still wonder if small startups would be willing to pay $600 per candidate (especially given that presumably only 10-25% of interview candidates actually end up getting a job offer, depending on the quality and rigidity of the resume and phone screens). Then again, paying 10 times $600 for 10 interview candidates is likely still a lot cheaper than paying a recruiter.

Oh, it seems like the site is still having some stability issues: Somebody posted the NochUp beta username / password on TechCrunch, and now the website keeps timing out. But I guess that’s why it’s supposed to be a private beta, so perhaps we should cut them some slack…

Anyway, it will be interesting to keep an eye on how this shapes up.

As you may have noticed, I haven’t been blogging much lately. I am hoping to change this and start posting more regularly again. In the mean time, I have adopted Twitter, so please feel free to follow my Twitter stream.

A few days ago, fantasy author Terry Pratchett announced some sad news on Josh Kirby’s blog (in a post titled “An Embuggerance”). Apparently he has been diagnosed with a rare, early onset form of Alzheimer’s. His blog post sounds optimistic under the circumstances:

Folks, I would have liked to keep this one quiet for a little while, but because of upcoming conventions and of course the need to keep my publishers informed, it seems to me unfair to withhold the news. I have been diagnosed with a very rare form of early onset Alzheimer’s, which lay behind this year’s phantom “stroke”. We are taking it fairly philosophically down here and possibly with a mild optimism. For now work is continuing on the completion of Nation and the basic notes are already being laid down for Unseen Academicals. All other things being equal, I expect to meet most current and, as far as possible, future commitments but will discuss things with the various organisers. Frankly, I would prefer it if people kept things cheerful, because I think there’s time for at least a few more books yet :o) Terry Pratchett PS I would just like to draw attention to everyone reading the above that this should be interpreted as ‘I am not dead’. I will, of course, be dead at some future point, as will everybody else. For me, this maybe further off than you think - it’s too soon to tell. I know it’s a very human thing to say “Is there anything I can do”, but in this case I would only entertain offers from very high-end experts in brain chemistry.

Terry Pratchett has been one of my favorite authors ever since I first discovered his Discworld series about 20 years ago, and his cheerful satirism has made my life richer numerous times. I wish him all the best and sincerely hope he pulls through this.

So I went out and bought Leopard last Friday when it was released. I even got a free T-Shirt for being one of the first 500 customers at my Apple store. This was the first time I can think of that I ever went out and bought an OS on its release day (or any piece of software for that matter, perhaps aside from a computer game or two). I was worried there might be a long line, but my Apple store was pretty deserted (this was at 8PM). Maybe I just had the right instinct to go to the much smaller store at Stanford Shopping Center rather than the one on University Ave.

Regardless, I installed Leopard the same day, opting for a fresh install instead of an update. The install was very smooth, and about an hour later I had a nice upgraded Leopard system.

There are plenty of in-depth Leopard reviews out there, so I’ll refrain from boring you with another full review (aside from the fact that I haven’t played with Leopard enough to warrant a full review). Instead, here are a few bullet items:

• New Finder: I love it! The new aesthetics are quite pleasing, but it also packs some nice new features. Quick view (triggered by pressing ‘Space’ when a file is selected) is very convenient. Cover flow mode should be useful as well, although I probably don’t manage enough documents on my system to really benefit from this (since most of my work is online these days). It has a neat Easter Egg, by the way. The new sidebar is well organized and deals well with network shares. I haven’t used the new search shortcuts yet, but they seem convenient. Finder now includes a Search box at the top. This itself is useful, but the fact that the search results are global by default (as opposed to limited to the open folder) is confusing to me. I’ll have to figure out a way to change this.
• New Dock: I’m still making up my mind about this, but so far my sentiments are more on the negative side. The visual mirror effect is nice and the dock locks pretty polished, but this probably gets old very quick. Running applications are now indicated with a small blue dot underneath, which is hard to spot. It also takes up more vertical space, which is quite valuable on a widescreen display. I might end up moving my dock to the side. It does not use the new 3D look in that mode and actually looks quite nice.
• Stacks: Definitely a nice feature and some cute eye candy. Makes for a pretty application menu if you drag your Applications folder to the dock. I only wish there was a way to recurse into subfolders in that mode, perhaps by hovering over them with the mouse for half a second.
• Safari 3.0: Probably warrants its own blog post at some point, but I haven’t really used it enough at this point (still loving Camino). But this is a significant upgrade and I’ll have to take it for a serious spin one of these days. The new tab functionality works great, including dragging tabs, a feature that I miss in Camino (and that Firefox has had for ages). Safari finally has an incremental find functionality, which is very well implemented. It also seems a lot snappier than the previous version.
• Dashboard: You can now very easily turn part of any website into a Dashboard, simply by selecting it in Safari. This feature works as advertised and should prove to be very useful.
• Spaces: Nothing ground breaking (after all virtual desktops have been standard in Linux for over a decade…), but still a very elegant and nicely integrated implementation of a virtual desktop.

I was originally going to talk about the developer tools and particularly the improved Ruby support, but this post is already getting much longer than I intended, plus I haven’t had a chance to test drive the new RubyCocoa support. So expect a follow-up post on this topic soon.