A long time ago in a job far away, I was responsible for, among other things, a physical datacenter with a few hundred servers and core networking gear, all spread across several rows of racks. Redundant power end to end, physically diverse fiber paths, FM200 fire suppression, two factor access control, it was a setup to be proud of, to be sure.
Unfortunately though, the contractors who designed and built the datacenter … hadn’t really ever done anything like a datacenter before. Their expertise was more in the food services industry, specifically designing commercial kitchens. This meant that while their choices were always to code, they sometimes looked at design criteria from a very different mindset.
Warm servers? Put the ductwork above them like they’re cooktops and blow cold air everywhere. Of course!
Raised flooring? Installed, but used for absolutely nothing because they had run everything overhead. Power, data, air return - all in the ceiling.
However, the least-good of their design choices involved water. Specifically, the building had a boiler for common use, and they chose to use steam humidification in the datacenter. On it’s own, this isn’t the worst idea, but combined with the advanced age of the rest of the building’s steam infrastructure, as well as being an option this particular air conditioning manufacturer had never actually done before, and you have a recipe for a very cranky HVAC setup.
Most of the time when the humidification gear would kick in (which was only once or twice a year thankfully), it would only cause damage to itself. Success! But every once in a while, it would start to humidify and just never stop adding water to the air. “Sauna like” conditions in a datacenter are mildly unsettling the first time you experience them.
After the time we found moisture on a wall nearest the HVAC unit, the steam humidification was disabled, and we bought a $50 appliance from the local hardware store that never gave us trouble again.