Home |
Search |
Today's Posts |
|
UK diy (uk.d-i-y) For the discussion of all topics related to diy (do-it-yourself) in the UK. All levels of experience and proficency are welcome to join in to ask questions or offer solutions. |
Reply |
|
LinkBack | Thread Tools | Display Modes |
#1
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Ok, not strictly D-I-Y i know, but I have been asked to come up with
some specs so that we can get quotes for a new power supply for our server room at work. W have 3 tall racks (30U from memory) round about 6 - 7 feet anyway which are mostly stuffed full of Dell servers. These are rated at 2 Amp (The 1U jobbies) and 5A (2U jobbies). I don't know if this is the maximum they draw at start up, or if they may take more at startup. What I do know is that the 30A breaker which feeds the radial circuit they all go back to (via a few UPSs) trips out if the power comes back on after an outage This can be a bit embaressing if we are all asleep in our beds and there is a power cut. The sequence goes something like Power goes off, UPSs continue to power equipment Backup generator kicks in MCB opens UPSs keep the servers up for 10 or 15 mins Then everything goes down. If we are here when it happens, we just put the breaker back in while the UPSs are holding up and live to fight another day. Anyway, to the question (I hope I haven't asked this before, I couldn't find it via google) Is there a rule of thumb for specing power supplies for these racks? Based on the rating plates, I could allow 2.5A per U, but I don't know if I need to allow extra over what it says on the rating plates of the servers, or if Dell have already done that. Does 60A per rack sound about right, way over the top? Way under? Assuming it is right, what would be the best way to provide it? A number of 16A or 30A Industrial sockets? Answers on a postcard please. :+)) |
#2
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
|
#3
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Andy Burns coughed up some electrons that declared:
I'd be tempted to split the load across multiple 16A outlets, to reduce the total inrush current per circuit, also switch to type-C MCBs. Agree. At Imperial College, we had this problem - had to go to type-D breakers in some cases, but the design considerations for putting type D in are onerous (eg you need a *very* low L-E impedance back to the supply: fortunately the 11kV transformers were round the back of the building and the College has a resident Electrical Engineer to worry about these things). Alternatively consider intelligent power distribution strips which can sequence the servers back on after any power cut, and have the advantage you can remotely power them off/on if required. APC masterswitches were good at this, and allowed for remote power switching via RS232 and ethernet. Think that range has been superceded, but it's likely APC still make something similar. Cheers Tim |
#4
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Tim S wrote:
Andy Burns coughed up some electrons that declared: I'd be tempted to split the load across multiple 16A outlets, to reduce the total inrush current per circuit, also switch to type-C MCBs. Agree. At Imperial College, we had this problem - had to go to type-D breakers in some cases, but the design considerations for putting type D in are onerous (eg you need a *very* low L-E impedance back to the supply: fortunately the 11kV transformers were round the back of the building and the College has a resident Electrical Engineer to worry about these things). Alternatively consider intelligent power distribution strips which can sequence the servers back on after any power cut, and have the advantage you can remotely power them off/on if required. APC masterswitches were good at this, and allowed for remote power switching via RS232 and ethernet. Think that range has been superceded, but it's likely APC still make something similar. Cheers Tim I'm assuming the ups's keep the pc's running and the breaker supplies the ups's ? Therefore isn't it more likely the ups's overloading the circuit when the power is returned ? they can have quite hefty startup currents, especially if the batteries are already under load and discharging rapidly. Pete |
#5
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Pete Cross wrote:
Tim S wrote: Andy Burns coughed up some electrons that declared: I'd be tempted to split the load across multiple 16A outlets, to reduce the total inrush current per circuit, also switch to type-C MCBs. Agree. At Imperial College, we had this problem - had to go to type-D breakers in some cases, but the design considerations for putting type D in are onerous (eg you need a *very* low L-E impedance back to the supply: fortunately the 11kV transformers were round the back of the building and the College has a resident Electrical Engineer to worry about these things). Alternatively consider intelligent power distribution strips which can sequence the servers back on after any power cut, and have the advantage you can remotely power them off/on if required. APC masterswitches were good at this, and allowed for remote power switching via RS232 and ethernet. Think that range has been superceded, but it's likely APC still make something similar. Cheers Tim I'm assuming the ups's keep the pc's running and the breaker supplies the ups's ? Therefore isn't it more likely the ups's overloading the circuit when the power is returned ? they can have quite hefty startup currents, especially if the batteries are already under load and discharging rapidly. Usually in a rack setup like this each server has two PSUs, one supplied direct from mains (with generator backup) the other supplied from mains via UPS. There is no initial disruption to the half of the PSUs being fed from the UPS, so no surge, when the mains goes off there will be a surge when half the PSUs come back on line when either the generator kicks in, or the mains is restored. If this surge trips the MCBs then you leave the servers on one PSU permanently, if this UPS happens to be on the same MCB they'll all die when the battery is exhausted. |
#6
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
I see now then...
So what you need is a delayed start on the psu's, same as on a sunbed where the tubes are powered sequentially otherwise it would exceed the 13A fuse in the plug. Pete |
#7
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
On Thu, 08 Jan 2009 11:12:25 +0000, Andy Burns wrote:
I'd be tempted to split the load across multiple 16A outlets, to reduce the total inrush current per circuit, also switch to type-C MCBs. Alternatively consider intelligent power distribution strips which can sequence the servers back on after any power cut, and have the advantage you can remotely power them off/on if required. Both, I'm surprised racks with so many servers don't already have master switches just for the sequential start up, the abilty to power cycle individual bits of kit without having to physically visit is just a useful side effect. B-) As for power, per rack 60A is 13.8kW, 3 racks 41.4kW I hope you have some good air con! Maybe buying a decent clamp meter and measuring what is actually being drawn would be useful. Not sure how they are affected by distorted waveforms produced by loads a SMPSUs though... -- Cheers Dave. |
#8
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Answers on a postcard please. :+)) It sounds like a classic case of your employer wanting to do it on the cheap the result of which could be that you get the blame when it all goes pear shaped. The whole job needs to be specified by a specialist professional in the field, which is not likely to be you average sparky who, with all due respect to inhabitants of this NG, will no experience in this sort of installation. If possible get three different quotes for the specification and the installation of it. Get references for similar work they have done and take them up. Peter Crosland. |
#9
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Dave Liquorice wrote:
I'm surprised racks with so many servers don't already have master switches just for the sequential start up, You can get "dumb" sequential startup strips too the abilty to power cycle individual bits of kit without having to physically visit is just a useful side effect. Which you may not need if your servers have "lightsout" control where you can turn them on and off over a dedicated network which is powered from the standby power of the PSU. Also with servers on multiple power supplies you need to have multiple master switches and power off all relevant outlets feeding a single server to reboot it. |
#10
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
In article ,
wrote: Ok, not strictly D-I-Y i know, but I have been asked to come up with some specs so that we can get quotes for a new power supply for our server room at work. W have 3 tall racks (30U from memory) round about 6 - 7 feet anyway which are mostly stuffed full of Dell servers. These are rated at 2 Amp (The 1U jobbies) and 5A (2U jobbies). I don't know if this is the maximum they draw at start up, or if they may take more at startup. What I do know is that the 30A breaker which feeds the radial circuit they all go back to (via a few UPSs) trips out if the power comes back on after an outage We have 50 odd racks in our machine room - and often had this problem. The Dell servers were by far the worst culprits along with the EVA storage. Sun and HP servers tend to be better. We have dual psu machine feed from two independant distribution networks and most Dell boxes appeared to run from one or other psu on not share the load between them. This tended to cause one side of the rack to be overloaded and cause the breakers to pop :-( Sorted by fitting different MCBs in the SAN racks and decent sequencing power distribution to the server racks. We use APC power dist (I think the models we have are http://www.apc.com/products/family/index.cfm?id=136 ) which give us remote (ssh or https) control of the outlets and also sequencing (can be configured with timings to allow things to boot in order). They also offer some load monitoring. The ones we have are zero U models that fit nicely into our APC racks - you might be better with a few of the rack mount versions if you don't have room to mount them. Not cheap, but pretty solid Darren (oh, we stopped buying Dell servers as well ) |
#11
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
In article ,
Andy Burns wrote: Usually in a rack setup like this each server has two PSUs, one supplied direct from mains (with generator backup) the other supplied from mains via UPS. Or even better, both feeds from UPS protected feeds, with gen backup. Of course, then someone buys a bloody sun server with one power lead or even worse, one with *3* power feeds. Stupid bloody machines. Thankfully, I think sun have stopped that now Darren |
#12
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
On Fri, 09 Jan 2009 09:04:07 +0000, dmc wrote:
In article , wrote: Ok, not strictly D-I-Y i know, but I have been asked to come up with some specs so that we can get quotes for a new power supply for our server room at work. W have 3 tall racks (30U from memory) round about 6 - 7 feet anyway which are mostly stuffed full of Dell servers. These are rated at 2 Amp (The 1U jobbies) and 5A (2U jobbies). I don't know if this is the maximum they draw at start up, or if they may take more at startup. What I do know is that the 30A breaker which feeds the radial circuit they all go back to (via a few UPSs) trips out if the power comes back on after an outage We have 50 odd racks in our machine room - and often had this problem. The Dell servers were by far the worst culprits along with the EVA storage. Sun and HP servers tend to be better. We have dual psu machine feed from two independant distribution networks and most Dell boxes appeared to run from one or other psu on not share the load between them. This tended to cause one side of the rack to be overloaded and cause the breakers to pop :-( Sorted by fitting different MCBs in the SAN racks and decent sequencing power distribution to the server racks. We use APC power dist (I think the models we have are http://www.apc.com/products/family/index.cfm?id=136 ) which give us remote (ssh or https) control of the outlets and also sequencing (can be configured with timings to allow things to boot in order). They also offer some load monitoring. The ones we have are zero U models that fit nicely into our APC racks - you might be better with a few of the rack mount versions if you don't have room to mount them. Not cheap, but pretty solid Darren (oh, we stopped buying Dell servers as well ) BAe Systems at Warton had a lot of problems with circuit breakers feeding computer suites. They eventually solved it by going back to fuses! We supplied them with some rather nice (if I may say it myself) fused distribution panels. -- Mick (Working in a M$-free zone!) Web: http://www.nascom.info Filtering everything posted from googlegroups to kill spam. |
#13
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
In message , dmc writes
Darren (oh, we stopped buying Dell servers as well ) Good man. PERC controllers are the spawn of Satan. -- Clint Sharp |
#16
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
In article ,
Andrew Gabriel wrote: In article , (dmc) writes: The Dell servers were by far the worst culprits along with the EVA storage. Sun and HP servers tend to be better. We have dual psu machine feed from two independant distribution networks and most Dell boxes appeared to run from one or other psu on not share the load between them. This tended to cause one side of the rack to be overloaded and cause the breakers to pop :-( Well, the whole point of splitting the power is so that if one supply fails, the other continues to run the servers. If either supply isn't capable of doing that, then there's no point in splitting the servers across multiple supplies. Yes, but the start up surge on the dual psu dells was all on one psu. Given we had plenty of them it meant seriously under speccing the power feeds and hoping they wouldn't all come up and suck power from the same feed. It was fairly random IIRC (at least if it was consistant which psu they would pick we could have shared them about). Sods law meant that 80% of them would come up on one of the feed tripping it out are which point they would all switch to the other feed and the surge would trip that. As I say, we switched to HP for winblows servers and it got a lot better (maybe just better psus?) and now we have all the racks sequenced anyway so the problem has gone for good (he says, touching wood/mdf ) Darren |
#17
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Clint Sharp wrote:
Good man. PERC controllers are the spawn of Satan. Don't suppose you'd care to expand on that? All my critical systems are on some sort of PERC... Andy |
#18
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
In message , Andy Champ
writes Clint Sharp wrote: Good man. PERC controllers are the spawn of Satan. Don't suppose you'd care to expand on that? Not really, apart from *never* listen to Dell support if they tell you to re-seat a failing/failed disk, demand an exchange or replacement. All my critical systems are on some sort of PERC... Ahh. Of course you test your backups regularly I hope. In fairness the newer PERC controllers do seem to be a lot better so far but.... Andy -- Clint Sharp |
#19
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
On 11 Jan, 00:05, (Andrew Gabriel) wrote:
Good stuff as he usually does. Many thanks one and all, and apologies for not being around for a couple of days to read the replies. It has given me lots of food for thought. |
#20
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
Clint Sharp wrote:
Not really, apart from *never* listen to Dell support if they tell you to re-seat a failing/failed disk, demand an exchange or replacement. People fall for that? When that happens to me I regard it as good luck that I've a warning of an upcoming failure! All my critical systems are on some sort of PERC... Ahh. Of course you test your backups regularly I hope. IT department look after that. TBH I trust at least one of them. Andy |
#21
Posted to uk.d-i-y
|
|||
|
|||
Server room rack power
In message , Andy Champ
writes Clint Sharp wrote: Not really, apart from *never* listen to Dell support if they tell you to re-seat a failing/failed disk, demand an exchange or replacement. People fall for that? When that happens to me I regard it as good luck that I've a warning of an upcoming failure! Far too regularly, despite having been warned several times, otherwise intelligent people in jobs that have much higher pay rates than mine do this and then call me out to mop up the damage. Plenty of other PERC disasters as well, but that's the most common. All my critical systems are on some sort of PERC... Ahh. Of course you test your backups regularly I hope. IT department look after that. TBH I trust at least one of them. Lol, good, but do your IT department actually test the backups by restoring to different hardware/disk space or do they just rely on verifying it? Andy -- Clint Sharp |
Reply |
Thread Tools | Search this Thread |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Forum | |||
Power For Server Room | UK diy | |||
Server Room Power Supply | UK diy | |||
Computer room static blowing server power supplies | Electronics Repair | |||
Anyone Need a Server Rack cabinet in the Orlando FL area? | Electronics Repair | |||
Cooling Server Room | UK diy |