The rules are:
1) All parts must be new.
2) NO Intel Atom processors
AMD E350 + E35LM1 CPU / Motherboard: $61.95 (Gigabit LAN, Integrated Audio / Video)
PortaTech.com - ASRock E35LM1 APU Combo
2 x 2 GB RAM (4 GB DDR3-1333) RAM: $18.88
1stChoiceMemory.com - Market Grade - 2 GB PC3-10600 / 1333MHz DDR3
Mini-ITX Case w/ 400W Power Supply: $31.99
CA-IM1207B BLACK MATX 350W iMicro CA-IM1207B 350W 20+4pin MicroATX Mini Tower Case (Black) - Desktop Cases - SuperBiiz.com
320 GB 7200 RPM Western Digital Hard Drive: $36.78
SATA 320GB Hard / Removable Drives at Pricewatch - Lowest prices, Sales | Page 1
Genius SATA Cable: $1.99
SDHC Reader: $2.99 (OEM PC World)
All for the grand sum of $154.58
Add a USB keyboard ($5.49), USB mouse ($5.49), and 23" LCD monitor ($109.99) in for a grand total of less than $270. And of course, it has to run Linux, being a budget build.
You could even have accelerated nVidia graphics in the system for under $300 (or less than $190 without the monitor / keyboard / mouse).