[Update: since these benchmarks were run, performance of Restify has improved substantially to where its performance is on a par with the others. I have not got around to updating the charts in this article to reflect that].
In the cloud today, performance is more important than ever because it can be so easily correlated with cost. If my app can handle 100 requests per second using a single CPU in the cloud, then that means that it will cost me 10 cpus if I need to handle 1,000 requests per second. In the cloud, 10 CPUs cost 10x as much as 1 cpu.
PerfectAPI is my open-sourced Node.js-based service API module. It is designed to be used as an API server that’s easy to write and easy to use. It is mildly RPC-biased, although it includes many REST principles and exposes similar HTTP endpoints. I wanted to run some benchmarks in order to validate its performance.
I chose to compare to Express.js because PerfectAPI uses Express.js and so it could not possibly perform any better. (So its a good baseline). I chose to also compare to Restify.js because it is a similar product.
Benchmarks were run on a small Amazon EC2 instance using ApacheBench running on a separate EC2 instance. I used TJ Holowaychuk’s guidance to plot the charts. Ok, on with the results…
First off, Express.js is a performance god. Although I wasn’t able to replicate these results, my tests regularly averaged a very respectable 1,600-1,700 requests per second. My sample app for PerfectAPI did around 1,000 requests per second. Restify.js, I’m sorry to say, was much slower – around 250 requests per second. Memory usage for all 3 was very low, as expected with Node. The text results are all here, with the associated code here.
Here is a chart showing all three (note, this chart is out of date since Restify performance issues were resolved):
As you can see, Restify.js is not in the same league. I’m sure it is nothing structural, it probably just needs some performance tuning. Excluding Restify, the same chart looks like this:
You can see that as expected, Express.js is faster. PerfectAPI is no slouch though, and scales similarly. I don’t think the upturns at the end mean anything significant – to prove that to myself, I doubled the run to 16,000 and you can see that the performance is consistent over the length (although it has another upturn at the end).
Well that’s all I have really. Please comment below if you have questions or observations.
FYI, Restify has been working on the performance issues: https://github.com/mcavage/node-restify/issues/107
Thanks for the heads up Steve. I reran some benchmarks using latest Restify from npm and it is indeed much improved! Some preliminary results here https://gist.github.com/2361848 show it is now on a par with the others (and perhaps even better).
Would you please post a correction or update the charts to reflect the new performance numbers? Fair or not, this site is a top-ranked Google search result for “restify performance”.
I added a note to the top of the article.
[...] Like what the description says in their github repo, it’s a fast and minimalist web framework for node applications. Express handles post/get request, session [...]
Why do you use -k (keepalive) in the benchmark?