Bit Pushers
Our storage cluster offers over 3 petabytes, not to mention no shortage of angular momentum. What you're seeing is mostly 1Tb disks (up to 240 per rack) served by Netapp appliances.

Although 3 petabytes may not seem all that much (you could imagine ordering 1000 WD 3Tb drives), once you consider the requirements for fault-tolerance, power, cooling and floor load, deploying this much storage is not trivial.

It's a little disorienting to think about where all this started (full article).

Out-of-focus images are made creamy by the beautiful bokeh of Canon lenses.

The compute cluster, currently largest in British Columbia (as of Jan 2011), consists of 420 nodes each with 12 cores and 48GB RAM, totaling 5,040 cores and 20TB RAM. Each node has 160GB local /tmp space and all nodes are tied together over an Inifiniband 40Gbs network. The nodes all have access to a dedicated storage system over the Infiniband Network running GPFS with a total 700TB of usable scratch space. The filesystem is served by 8 IBM x3850 servers. All nodes are running CentOS5.4 and are using open source Grid Engine 6.2u5 as their scheduler.

The storage and compute clusters are in separate rooms.

Technical note: How were these photos created? Take a look at the lighting setup for storage and compute clusters. I photographed our lab with the same lighting method (Genome Pushers set, example setup).
68 photos · 1,034 views