Episode 5x19, Building the Ultimate White Box Server for under $2000 Options

[Brian Sierakowsk...]

Hey guys, great episode!

I've actually been looking into virtualizing our infrastructure using VMware and Datacore, so this episode was extremely timely.

My questions: Given the option (and the budget), would you of preferred to get a prefab server from dell or whoever your vendor of choice is?

We're currently looking into a two server configuration, do you think two $1000 servers could perform as well as 1 $2000 server? I like the idea of initially starting with two so that we can load balance, and when its time to upgrade we can move the VM's to the first server while replacing the second.

We're crunching numbers right now, and we're trying to deal with the curve of "how cheap can we do this now" with "if we go cheap now, how is this going to effect us in the future." So, we may opt out of vMotion, but then if we ever decide to host customer solutions here, that's something we're going to have to upgrade to, etc etc.

Also, our voice servers use an internal USB dongle, we use a little bit of internal cable management to hold it down like so. Place one or two of these on the side or bottom of your case and you should be good to go.

Again, the episode is great, very inspiring that with enough determination a $40,000 project that is immediately turned down can turn into $20,000 that may get green lighted.

[WhollyMindless]

And thanks for putting out what Systm *would* have put out.

(and "no thanks" for making me late to work... again)

[Seshan]

And thanks for putting out what Systm *would* have put out.

(and "no thanks" for making me late to work... again)

Should of watched it last night You can get it on iTunes before Rev 3.

[jaggededge]

heh right... systm failed me. Used to watch Hak.5 back in 06 and lost interest after the Rev3 thing, glad I started watching again you guys rock. Keep it up!


----------------
Now playing: The Good, The Bad and The Queen - Herculean
via FoxyTunes

[Darren Kitchen]

Spec out your ultimate $2000 white box server for an opportunity to win awesome swag from the hakshop including the last Hak5 shot-glass. Detail the parts and why you picked 'em here and we'll pick a winner based on performance, value and creativity. Post your entries here!

[Xipher]

This isn't a <2000 server entry for the contest, just my own personal build I did recently

http://secure.newegg.com/WishList/PublicWi...Number=10870732
Also including a Intel PCI gig-e NIC since the onboard NIC isn't supported. Also have to configure the SATA controller in AHCI mode just as a note.

No RAID but it comes in under 800. Only downside is that if you want to use the PCI-E at all you will need a PCI video card as well since the onboard gets disabled the moment you use the PCI-E slot at all.

It's working great so far for my own personal tasks, and I'm also using it to prep for QuakeCon (one of the volunteer staff). The shuttle form factor makes travel with it easy so taking it with me from Iowa to Texas shouldn't be an issue.

[3TeK]

Antec 300 Case (just cause i like mine so much)
TYAN Thunder n3600B Dual 1207
Dual AMD Opteron 2352 Barcelona 2.1GHz Socket F 75W Quad-Core
6 sticks of 4GB DDR2800 Kingston ECC Buffered memory = 32GB
4 Seagate Barracuda 1TB hard drives = 4TB
PC Power & Cooling 750w PSU
Samsung 22X DVD+/-RW (sata)
2 Dynatron Socket F heat sinks
4 Evercool 120MM case fans (80CFM)

Grand Total: $1,862.47 shipped to my house :-D

[TechLife]

As much as I'd love to build a rack for my computer gear, it's just not practical when you live in an apartment. After seeing this Hak5 episode, I decided to scope out my own iSCSI solution. For YEARS I've been looking for a Mini-ITX solution, but it's near impossible to piece one together without a PCI/PCI-E Raid card, and good luck fitting that into a Mini-ITX case.

Well, I found the solution the other day, which has now sparked my summer tech project. Check this.

Mini-ITX RAID5 FreeNAS - $810 - iSCSI NAS that fits on a shelf.
Case - $205 - http://www.logicsupply.com/products/es34069 - extra $60 for a higher watt power supply, might be a good idea with the 4 drives.
Mobo - $129 - http://www.logicsupply.com/products/j7f2we_1g
RAID5 daughter card (attaches to mobo) - $39 - http://www.logicsupply.com/products/adpe4s
1Gb IDE Flash Drive - $29 - http://www.logicsupply.com/products/fdm40xdi1g
512MB 667 RAM - $12 - http://www.logicsupply.com/products/64mlq64v6j
4x WD 1TB Black 3.5" SATA HD - $99ea. ($396 total) - http://www.newegg.com/Product/Product.aspx...N82E16822136284

ESXi box - $510 - ESXi box that fits on a shelf.
X-QPACK Case w/420W PS - $80 - http://www.newegg.com/Product/Product.aspx...N82E16811144162
ASUS P5QPL mobo w/ onboard video - $80 - http://www.newegg.com/Product/Product.aspx...N82E16813131387
2x 4GB OCZ Platinum 2x2GB kits - $50ea. ($100 total) - http://www.newegg.com/Product/Product.aspx...N82E16820227298
Intel Core 2 Quad 9550 (2.83Ghz, 12MB Cache) - $220 - http://www.newegg.com/Product/Product.aspx...N82E16819115041
1GB IDE Flash Drive - $29 - http://www.logicsupply.com/products/fdm40xdi1g

Total is $1320.

You could go iCore 7 I suppose, but that hardware (chip, mobo, ram) hasn't quite hit the pricing sweet spot yet. Its getting close though. I haven't tested the hardware for compatibility yet, so if anyone knows of any red flags with this combo let me know before I find out that hard way?
One downside to this is that there are two separate boxes, but hey, you can stuff em just about anywhere!

[jona]

There are 3 configurations here: A white box ESX server, a white box ESXi server, and an entry level SAN. Here are my white box servers < $2000. The entry level SAN is not less than $2000, but it is 6 times cheaper than an Equalogic SAN and with more total space.

ESXi Server
ASUS P5Q Premium LGA 775 Intel P45 ATX Intel Motherboard – Retail - $169.99

Note: 16GB DDR2 1200, 10x SATA 3Gb/s, Quad 10/100/1000 Mbps Nic (Quad Port is Great for Virtualization)

Intel Core 2 Quad Q9550 Yorkfield 2.83GHz 12MB L2 Cache LGA 775 95W Quad-Core Processor – Retail - $219.99

Note: Quad Core, Great Speed, Great Cache Size

SUPERMICRO CSE-833T-650B Black 3U Rackmount Server Case 650W Redundant Cooling, High Efficiency Power Supply 2 External 5.25" Drive Bays – Retail - $379.99

Note: 8 Hot Swappable HDD’s (Supermicro Rocks)

G.SKILL 4GB (2 x 2GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel Kit Desktop Memory Model F2-6400CL5D-4GBPQ – Retail - $46.99 x2

Note: This memory rocks and fills up the board with a total of 8 GB’s

Seagate Barracuda 7200.12 ST31000528AS 1TB 7200 RPM 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive – OEM - $89.99 x 8

Note: 8 SATA drives fills up the hot-swappable bays.

areca ARC-1210 PCI-Express x8 SATA II Controller Card RAID 0/1/1E/3/5 JBOD - Retail - $299.99

Note: Great Raid Card but manufacturer is not well known. Total array size = 7 TB

ASUS EAH3450 SILENT/DI/512MD2 Radeon HD 3450 512MB 64-bit GDDR2 PCI Express 2.0 x16 HDCP Ready Low Profile Ready Video Card - Retail - $20.99

Note: HDMI and HDCP ready

LITE-ON Black 52X CD-R 32X CD-RW 52X CD-ROM ATAPI/E-IDE CD Burner – OEM - $16.99
Note: Nothing special here

$1921.84

Final Note: Quad 1Gb/s Nic, 8 GB memory, Quad Core with 12 MB L2 Cache, 8 Hot-Swappable drives in a Raid 5 configuration with 6.5 TB usable disk space. Great setup for ESXi server (Since license can only have one CPU – Up to 6 Cores). I would utilize the server datastore to install VM’s.

ESX Server - license not included
First off I would like to say that a server should have redundant nics and power supplies.

Athena Power RM-4U4064XR4 Black 4U Rackmount Server Case 400W Redundant 6 External 5.25" Drive Bays – Retail - $306.99

Note: redundant power supplies and able to install 2 AMS DS-3151SSBK Backplane Moduels, which provides 10 hot-swappable SATA bays.

AMS DS-3151SSBK Aluminum 3.5" Black SATA SATA Backplane Module - Retail - $124.99

Note: The Athena case will accept 2 of these modules for a total of 10 SATA HDD’s, would need to purchase an additional SATA controller, or a SATA Raid Controller

SUPERMICRO MBD-H8DME-2-O Dual 1207(F) NVIDIA nForce Professional 3600 Extended ATX Dual AMD Opteron Server Motherboard - Retail - $369.99

Note: going to use onboard Dual NICs, SATA controller and RAID. Easy to upgrade later. 128GB max memory, 2 PCI-E 8x, 4 PCI-X

AMD Shanghai 2.4GHz Socket F Quad-Core Server Processor – Retail - $189.99 x2

Seagate Barracuda 7200.12 ST31000528AS 1TB 7200 RPM 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive – OEM - $89.99 x5

Note: HDD’s fill up the backplane module, which are all hot-swappable. RAID5 total array size = 4TB

Crucial 8GB (2 x 4GB) 240-Pin DDR2 FB-DIMM ECC Fully Buffered DDR2 800 (PC2 6400) Dual Channel Kit Server Memory Model CT2KIT51272AF80E – Retail - $164.99 x 2

Note: 16 GB configuration, 12 Slots Open, upgradable to 128 GB

Total = $1,961.88

Final Note: Motherboard has a lot of room for expansion (128 GB Memory, another 5 disk SATA backplane module, PCI-E, PCI-X, both great for RAID Cards. Can also add 4 additional HDD’s using the internal 3.5” bays. Using a USB CD-Rom to install OS. Intended for ESX Server, since this has 2 Physical CPU’s. Case is a 4U with redundant 400W power supplies.

Entry-Level SAN
PROMISE VTM610I RAID Sub-Systems – Retail - $4369.99

Note: Great customer support, RAID 50 compliant, 32 Lun Support, dual 400W power supply, 16 hot-swappable hdd’s. Not as scalable as EMC, Equalogic, or NetApp. Equalogic is the cheapest of the 3 manufactures listed and lowest cost is around $40,000 for the SAN.

Seagate Barracuda 7200.11 ST31500341AS 1.5TB 7200 RPM 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive (bare drive) - OEM - $129.99 x 16 = $2079.84

Note: 21 TB total available free space in a RAID 50 configuration.

Total SAN: $6,449.83

Final Note: 21 TB iSCSI SAN for $6,449.83. Just Awesome.

[3TeK]

ah shit you guys are gettin all serious..i just threw somethin together really quick..lol

[jona]

Enterprise ESXi configurations should be a well thought out, with equipment that meets the budget ($2k in our case.)

P.S. I'm getting a headache looking at your avatar 3Tek JK

[3TeK]

haha like that? lol

[Darren Kitchen]

@3TeK, up until now I hadn't really missed extensions all that much in Chrome. Now I have three options. 1) Find a way to build a "block image" extension for Chrome. 2) Switch back to Firefox and add your avatar to ad-block-plus. 3) gouge my eyes out with a spork.

Maybe I'll just block your avatar on the router level.

PS: Great submissions guys. Keep 'em coming! Loving the creativity in hardware selection. We'll have a field day on the show with these.

[yickerabbo]

@3TeK, up until now I hadn't really missed extensions all that much in Chrome. Now I have three options. 1) Find a way to build a "block image" extension for Chrome. 2) Switch back to Firefox and add your avatar to ad-block-plus. 3) gouge my eyes out with a spork.

Maybe I'll just block your avatar on the router level.

PS: Great submissions guys. Keep 'em coming! Loving the creativity in hardware selection. We'll have a field day on the show with these.

Dude, just hit the escape key on your keyboard and stop him in his tracks

[digip]

@3TeK, up until now I hadn't really missed extensions all that much in Chrome. Now I have three options. 1) Find a way to build a "block image" extension for Chrome. 2) Switch back to Firefox and add your avatar to ad-block-plus. 3) gouge my eyes out with a spork.

Maybe I'll just block your avatar on the router level.

THats why I like opera. I can just right click and select block content and click on things to make them go bye-bye. No addon or third party plugin needed.

[neg]

Here we go!

Neg's $2400 (Slightly over-budget but whatever!) ESXi Server

http://secure.newegg.com/WishList/PublicWi...Number=11925386

I'd throw the 8 drives into 2x 4-disk RAID 5 for a pair of 2.3Tb (formatted) data arrays.

If I had to keep it below $2000 I'd drop 4 of the disks out (new total: $2000) but keep the hotswap so I could add 4 more drives later as disk usage requires

I've checked and everything on this list is fully ESXi 4.0 supported, as long as SCSI-emulation is enabled on the Areca card.
If more NICs are required, $34 Intel PWLA8391GT (http://is.gd/1dDfA) can be added (up to 2)

Here's my reasoning on things that aren't obvious choices:

Mobo:
Picked it because it seems like a good mobo and has twin-gigabit intel NICs supported by ESXi, along with 2x PCIe x16 and 1x PCIe x8, + PCI and 2 PCI-X (lots of room for expansion).

RAID card:
Areca ARC-1222. 8-port (using miniSAS) RAID controller with almost limitless upgradeability, discrete HDD LED connections, HTTP config wizard and SCSI emulation of arrays provides 100% ESXi 4.0 compatibility according to the HCL.

Feels good man.

The Card Reader is so I can run FreeNAS and/or Windows Home Server in a VM and have an instant flash drive / memory card dump utility that scans for drives and dumps their content to a folder on connection.

The rest of it is pretty self-explanatory really. I chose the Kingwin hotswaps because they're compatible with my 3ware miniSAS breakout cables, the NZXT fan controller is pretty much there because I had a spare 5.25" slot - but it means the fans on the hotswap bays can be temperature controlled by the HDDs inside them, and the rear exhausts too - which is a major plus in keeping volume down.

The case was chosen as it's cheap, rackmount and has 8 5.25" and a 3.5".

CPU / RAM are self explained. CPU is a 920 because it's a 920, good bang/buck, and the RAM was the cheapest 1333-speed 12gig kit I could find - ECC isn't required on this board. I like OCZ gold series ram.

PSU is a CM 550W, i like CoolerMaster PSUs and it has the added bonus of being cheap.

3Ware breakout cables are to hook the high-density miniSAS ports on the Areca to the 4 individual SATA drives in each hotswap. I'll probably need a couple of molex splitters too but that's OK.

Flash drive. It's a 4 gig flash drive. Need I say more?

HDDs: cheap, 1Tb, 7200RPM, 32MB cache. 'nuff said.

GPU: cheapest PCIe GPU, nay, cheapest GPU I could get off The Egg.

--neg

[tamoneya]

http://secure.newegg.com/WishList/PublicWi...Number=11085152

Quick specs:
Dual Nehalem Quad core Xeons
4x1TB black
12 GB DDR3 RAM
dual intel NICs
3ware 4 port sata raid controller (with low profile adapter so it fits in the 2u case)

total: $1,952.83

[DarkBlueBox]

Im going to try to enter the contest. But us peoples in the UK only have a budget of £1,200 (€1,400) so gona be more difficult, but I like a challenge.

[franktank]

Ok, I'm lazy so I started off with a barebone kit... it also offers an integrated remote management thingy so I don't have to walk over to the server room as often to "touch" the device in case it's acting flaky or somtihng goes tits up. All prices are USD from new egg.

SUPERMICRO SYS-5026T-3FB 2U Barebone Server = $719.99
Has the chasis I want and a good board plus integrated remote management etc... con is no dual PSU. But in all my years of server managment, 15+ now, I've only lost one power supply so I'm willing to risk it. I would buy a second "hot spare" to stick on the parts shelf but did not include that in this build.

Intel Core i7 920 Nehalem 2.66GHz LGA 1366 130W Quad-Core Processor Model BX80601920 = $279.99
Great proc... would like to have gone Xenon but the budget didn't allow. Possible future upgrade as prices fall and could be paired with the upgrade in ram below assuming a budget of 2K every quater or two.

CORSAIR XMS3 12GB (6 x 2GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) HX3X12G1333C9 = $209.00
12 GB of ram, I wanted 24 but they didn't have the modules. I would add into my capex budget $ for an upgrade to 24GB when parts are available.

6x SAMSUNG Spinpoint F1 HD103UJ 1TB 7200 RPM SATA 3.0Gb/s 3.5" Internal Hard Drive - OEM = $539.94
Cheap. Should work fine and be plenty reliable in a RAID 1+0 array.

Sony Optiarc Slim Combo Black SATA Model CRX890S-10 - OEM = $34.99
Gotta have a CD/DVD drive. I hate lugging around an external and the chasis has a slim bay so why not.

StarTech SATABAY425BK 4 Drive 2.5in Hot Swap SAS SATA Mobile Rack Backplane = $70.99
I had money in the budget and this adds some expansion cpability. I would want to upgrade the PSU before utilizing all four bays but again, this is really just filler for that free 5.25 bay.

1x Seagate Momentus 7200.3 ST980411AS 80GB 7200 RPM 2.5" SATA 3.0Gb/s Notebook Hard Drive - OEM =$49.99
Had the money so I may as well add a disk to the StarTech, could be used for ISO file storage or whatever.

2x SanDisk Cruzer Micro 4GB Flash Drive (USB2.0 Portable) Model SDCZ6-4096-A11 = $29.98
2 internal USB ports... 1 for the OS and I'm not sure what to use the other for but I had the cash so I bought two. I wonder if you can mirror USB dirves?

Shipping = $46.80

Total = $1981.67

The down side of this configuration is the Intel ICH10R, I dont think it works with ESXi at the moment. I had fun putting it together though. So, I would be looking at installing a barebone install of CentOS and then just adding VMware server on top of that or going with Hyper-V. So, this is the problem with being a lazy admin... you should always check the compatability list.

[aperture kubi]

Honestly I have no idea what I am doing here, but this is what I would spec out: http://secure.newegg.com/WishList/PublicWi...Number=15826527
It's a mini-atx desktop motherboard and case I could throw in a closet or something while it does its thing.
Possibly throw a VM or two on it to mess with (At least mess around with a Steam game server) and storage.
My idea is to boot off of the removable CF card in the build instead of a usb drive that floats in the case.