Jonathan blogs on Sun Constellation System

Sun CEO Jonathan Schwartz offers his perspective on the Sun Constellation System this week, including some interesting specs on the first deployment at TACC:
TFLOPs: approximately 500 TERAFLOPs
Magnums: 2 (>2000 4x IB ports each, expandable to 6,912 ports)
Thumpers: 72 (1.728 PB)
Metadata storage: STK6450 RAID (9.3 TB)
Tape storage: STK SL8500
Storage/Data Management: SAM/QFS
Racks: 82
IB NEMs: 328
Pegasus blades: 3936
Aggregate memory size: 123 TB
Number of cores: 62,976
Total racks: 94
Approx footprint: 2,037 sq ft
Approx power: 2.4 MWatts
IB cable length: ~14 kilometers
To put that in perspective, their computing facility will be about half the size of an NBA basketball court. Not exactly small – and in fact, likely the largest on earth.
And for those curious as to why we settled on 3,456 ports…
Begin forwarded message:
From: Andreas Bechtolsheim
Date: June 28, 2007 6:58:59 AM PDT
To: Jonathan Schwartz
Cc: John Fowler
Subject: 3,456
We implement a 5-stage fabric, and with a 24-port switching element the maximum number of ports is n*n/2*n/2, or 24*12*12 =3456.
Other Infiniband switches in the market today are 3-stage fabrics and they have n*n/2 or 24*12 = 288 ports.
Now you can build a 5-stage 3456 port switch with 12 288-port switches and 288 24-port leaf switches but you end up with 300 boxes occupying about 456U of rack space or 12 racks, and 6912 cables. We use one double rack with 1152 cables, so it is 1/6th the space, 1/6th the cables and 1/6th the weight.
On Jun 28, 2007, at 6:36 AM, Jonathan Schwartz wrote:
so – why 3,456 ports?
Jonathan’s Blog


About Rich
FlexRex began his life as a cartoon character I created a Sun Microsystems. As the world's first "fictional blogger," he appeared in numerous parody films that made fun of the whole work-from-home thing. Somewhere along the line, the Sun IT department adopted FlexRex as their spokesman in a half-dozen security awareness films for employees. So when I left Sun recently, I started FlexRex Communications, a Marketing company in Portland, Oregon.

Comments are closed.

%d bloggers like this: