Enormous bitcoin "mining" operation goes up in flames
The operation was run by Euro expats in Thailand, a "cooperative" called Cowboyminers. The entire facility was paid for in bitcoin, and wasn't insured.
It's all very William Gibson-esque: Massive computer power strung together on the fly by geeks, some of them no doubt bent:
[T]he fire highlights the possible pitfalls for any Bitcoin mining startup. Bitcoin mining's appeal is its relative simplicity—you just need computing power. But setting up a facility is more complicated than throwing a bunch of hardware together in a room. Every detail of the traditional data center is designed with cooling in mind, from the placement of vents to the hot aisle/cold aisle layout of the equipment.
And regardless of how the fire started at the Thai facility, it clearly spread, possibly because of a flammable acoustic foam. A traditional datacenter would have also had sprinklers—or in fancier iterations, gas fire extinguishing systems—that detected smoke or heat early on to minimize the damage. Of course, these systems don't come cheap.
Mining currency may no longer involve people, but it still does involve monster machines—machines that are very expensive and very destructible.
And then of course there's that awkward confluence between glibertarians, regulations, and regulatory arbitrage:
The bitcoin network infrastructure is split between data centers and no-frills hashing centers featuring high-density hardware and low-reliability power infrastructure, often housed in former warehouses. Many bitcoin entrepreneurs focus on building high-powered infrastructure at the cheapest price point possible. As industrial mining operations scale up, they are optimized for rapid changes in hardware and economics.
But there are tradeoffs that accompany this single-minded focus on cost. Data center veteran Mark MacAuley has been publicly warning about the potential for this type of incident in low-budget mining facilities. “The nuances between a mining facility and a data center are more obvious than I thought,” Macauley tweeted in September. “A data center cares if there is a fire, for example.” MacAuley warned that the design of some bitcoin mines created the potential for fires.
Most traditional data centers feature fire suppression equipment. These systems typically involved advanced systems to detect heat and smoke, including sensors using VESDA (Very Early Smoke Detection Apparatus). Early detection can allow data center operators to quickly extinguish a fire or smoke condition before it can damage entire racks and rows of expensive equipment.
When a fire occurs, data centers use several methodologies to extinguish the fire. These include “pre-action” sprinkler system in which water fills the sprinkler pipes only upon an alarm, reducing the potential that leaks in overhead pipes will cause water damage. Some data center forego water altogether, using gas-based fire prevention and “clean agents” – electrically non-conductive gases that don’t leave a residue. Most clean agents are hydrofluorocarbons (HFC), which lack the ozone-depleting characteristics of chlorofluorocarbons (CFCs) like Halon gas, which was once popular in data centers.
Will this incident prompt more bitcoin miners to consider hosting equipment in traditional data centers? It’s too early to say. Recent declines in the price of bitcoin have further tightened profit margins for miners. To remain competitive, industrial mining operations are making large investments in new hardware. Many businesses would seek to protect these investments with business insurance, but insurers will likely to want the equipment to be hosted in facilities with proper fire detection and suppression equipment.
Wait, wait. Data centers don't have a single-minded focus on cost? MBAs don't run them, like they run everything else?
Makes you wonder how much of "the cloud" is out there in Bangkok suburbs, or, if Bangkok is too pricey, Manila or Hanoi or Yangon or Jakarta. Of course, one obvious remedy here is never do business with a company that has "cowboy" in its name, but who are the cowboys, really?
UPDATE I've heard that one owner's house, 350 yards away, also went up in smoke (no link). So, one does wonder who didn't get a cut from the enterprise, and whether they were anxious to share their feelings....