It was not just the New England Power Network that suddenly broke down on the evening of last November 9th. A number of very fine theories also broke down completely. The power network was back in operation within a maximum of twenty hours; the theories never will be in operating order again.
One of the consequences of this is that the electric-power engineers of the world - not just of New England! - are now living in fear, in an exact and technical sense. In total ignorance, there can be no fear. (A primitive Amazon River basin Indio has no fear of nuclear explosions, radioactive fallout, or invasion by extraterrestrial aliens). In full knowledge, there is no fear; what you know and understand does not frighten you, because you know its limitations and possibilities, and know how to deal with it effectively.
Fear comes when you have learned that a menace exists, but you do not understand the menace, how it acts, what its limitations are, or what to do about it.
In that sense, world-power engineers, both in industry, in government, and in universities, are living in fear. The Great Blackout proves that a menace exists; they definitely don't understand it, nor, therefore, know how to control it.
Somebody wrote an angry letter to one of the New York City newspapers after the blackout, charging that the "greedy" power companies could have prevented the whole thing if they'd just been willing to spend a little time and effort and computer-time running a mathematical simulation of the behavior of the network. That if they'd done that they would have known that the Great Blackout could occur, and how to prevent it.
It was one of many letters damning the power companies for the occurrence; that one particularly annoyed me, because it attributed motive ("greed") to the power companies for not doing something the writer was sure they could and should have done.
A mathematical simulation works when, and only when, you can express the actual characteristics of the system being simulated in mathematical form that the computer can handle. "Can" in that statement needs the additional modification, however, of "can in practical fact". That is, theoretically a computer could be programmed to determine whether White, who moves first, or Black, who moves second in chess, would win if both players played an absolutely perfect game. Since chess is a perfectly logical "universe", in which we know all the laws, and all the "natural constants" - the moves of the pieces - a machine can be programmed to play out a perfect game, exploring all possibilities of all possible moves, of all pieces.
The difficulty is that even the fastest computers would take something on the order of ten thousand years to carry out the program.
The problem of that electric-power network differs in several ways:
1. We do not know all the laws of the real universe.
2. The power network problem does not involve a mere thirty-two pieces on sixty-four squares; it involves hundreds of generators, and thousands of load blocks, and scores of interconnections through lines that introduce various complex delays due to transmission time.
3. It isn't a "logical" system; unlike chess, the "players" don't take nice, neat, logical-sequential "moves" - the only kind of system a logical pattern can exactly represent. Things happen simultaneously, and logic can't handle true simultaneity.

There are three broad classes of mistakes; the Blackout revealed instances of all three.
Type I is the Unavoidable Ignorance type. So long as you're not omniscient, there will be things you do that you later discover were definite mistakes. A small child who sticks his finger into a live electric light socket makes an unpleasant discovery - that that's a definite mistake.
The Great Blackout itself was a result of a mistake of Ignorance; the network turned out to have characteristics that no one had ever imagined.
The Type II mistake is the Pure Goof. The individual does something, overlooking factors of which he was perfectly aware, but which he just plain forgot to consider in relation to the problem in hand.
Example: One of the major Boston hospitals had installed a magnificent new emergency power setup only about two months before the Blackout - a setup capable of supplying all the power needs of the hospital. No doubt the plant engineer threw the new plant switch ON with considerable pride and satisfaction...
It didn't run.
The fuel pump that supplied fuel oil to the big Diesel engine was designed to be electrically powered.
If the Diesel and the generator were running, everything was fine. If the engine were started before the power line failed, everything was fine. But there was no way to start the thing after the power failed.
They got it going by chopping a hole in the fuel tank, dredging out some fuel, carrying it upstairs, and pouring it into a funnel and down a long rubber tube. That gave enough pressure-head to force the starting fuel in.
Another hospital - New York this time - had their emergency power generator equipment in the basement. This was all right; the plant started up satisfactorily... but didn't run very long. The basement was below the level of the adjoining river, and was normally kept dry by continuous pumping by electric-powered pumps. Somebody had classified these pumps as nonessential accessories, and their line wasn't tied into the emergency-power board. Before it could be, the generator was drowned out.
Those are plain, all-too-human goofs, resulting not from basic ignorance, or rejection of responsibility, but from simple mental slips.
Neither of those two types can be classified as bearing any onus of guilt. Responsibility, yes - but not guilt.
Type III mistakes are those resulting from refusal to accept responsibility - by insisting that the responsibility "should be" someone else's.
By a sheer God-granted miracle, none of the millions of people affected by that blackout were killed. That is, perhaps, the most incredible aspect of the whole affair.
It just happened that that early November evening, the skies over all of that eighty thousand square mile area were perfectly clear, and that a brilliant, nearly full moon shone in the sky. Had it been a night of typical November cold drizzle, it's highly probable that hundreds of people would have died in air crashes that night. There were several hundred planes in the air over that territory... and abruptly all the lights went out on all the airports. Moreover, simultaneously all the radio navigation equipment went dead, the control-tower communications died, and the ground radars went black.
Commercial airliners aren't normally equipped to talk to each other. And the air traffic over that northeast corridor from Boston to and around New York is fantastically dense for such high-speed planes.
How the planes managed to unscramble themselves, realign the traffic patterns without ground control, until the still-functioning airports, that were lucky enough to have commercial power, took over is a major feat of pilotage on the part of all the airliner captains in the air at the time.
If there had been drizzling clouds at the time...
It wasn't that Newark, Philadelphia, and the other airports that offered the planes refuge were better run; they just happened to be lucky enough to be in an area the Blackout didn't reach.
That all those airports could have been set up without any provision for emergency power represents a true Type III mistake. They rejected responsibility for maintenance of power supply on the basis that the commercial power companies "should" maintain it for them.
That type of mistake does merit the assignment of guilt, not merely responsibility. It was inexcusable.
Note that the military bases throughout the area simply switched to their local emergency power generators, and continued full normal operations. The Bell Telephone system hiccuped slightly while storage batteries took over the load, and then their emergency generators were started up and the telephone system operated normally. Military communications functioned perfectly, on fully competent emergency power plants. The search radars that keep constant watch over America's skies didn't falter, nor did the communications links feeding the data to command centers.
But the commercial airports, with literally thousands of human beings depending on them for life, all blacked out.
Sure - I know. The power companies are legally required to maintain power supplies to their customers.
Another group of Type III errors also showed up - the major hospitals throughout the blackout area were, almost at once, busily screaming for help from police and other agencies for emergency-power generators. The police, like the military, had adequate emergency-power arrangements, and were able to carry on their work very competently, and did supply several hospitals with emergency-power generators. Commercial companies that sell such equipment also acted quickly to loan units they had in stock. And they had to loan some big units; any hospital which needed a one thousand kilowatt emergency generator rushed to the street beside the building was obviously badly in need of power.
There were babies being born by candlelight in great New York hospitals. A brain operation was completed by the light of a few flashlights. A frantic call came from one of the major hospitals for ice from a commercial ice manufacturer - ice to preserve the blood in their blood bank.
Oh, I tell you - New York's hospitals were in fine shape to handle any emergency that might come along.
New York City's Civil Defense system fell flat on its face, too.
The major radio stations of the city displayed ludicrous ineptitude in the emergency-preparations department - what a help they'd have been in a real emergency! Most of them stayed on the air... thanks solely to the fact that the transmitters were located in New Jersey, and the New Jersey power companies were not tied into the New England network. (The transmitters are across the river on the New Jersey meadows because (1) land is cheap there, and (2) the meadows are saltwater marshes, which gives the antennas an almost ideal ground to work against).
But the studios of the major network stations were all in New York, of course. And they were able to communicate with their transmitters in New Jersey only because the telephone company, unlike the radio stations, did consider the problem of loss of commercial power, and had emergency equipment. The announcers in New York were operating by candlelight, flashlight, or - in one case - by the light of an automobile lamp hitched to an automobile storage battery someone had lugged up ten flights of stairs.
The airports were just plain, miraculously lucky that their failures didn't kill anybody. The hospitals were equally miraculously lucky. Sure... they finished a brain operation by flashlight. Fine - skilled surgeons are human beings, and human beings, as organisms, have several hundred million years of evolution under the basic proposition "Lethal emergencies are normal; learn how to adapt anyway".
But modern hospital equipment is electronically operated; the anesthetist uses electronic sensing devices to keep track of his patient's heart functions - pulse, blood pressure, regularity. There's usually a cardiac monitor device watching electronically along with the anesthetist, and ready to send electronic impulses immediately if the heart suddenly slows, or shows signs of fibrillation. Many modern operation procedures are predicted on the fact that such equipment gives the surgeons a reserve back-up, so that if the operation requires it, the normal heart-stimulating nerves can be interfered with briefly.
Fortunately, at the moment of the blackout, nobody happened to be hitched to a heart-lung machine. If anyone had, he would, of course, be dead the moment the blackout hit.
That the hospitals were so helplessly dependent on commercial power lines is flatly inexcusable. Again - sure, the power companies are legally required to maintain service. Yeah... and maybe somebody should pass laws against earthquakes, tsunami waves, major hurricanes, and major meteor strikes. Or even some disgruntled paranoid with a suitcase full of dynamite in the nearest power substation. It doesn't require that the whole northeast be blacked out to interrupt the power to a hospital.
In the event of a major disaster, it's obvious that the hospitals would be most desperately needed.
And the Great Blackout demonstrated that the hospitals, under those conditions, would be plaintively bleating for somebody to supply them with emergency power.
It's inexcusable, because emergency power equipment isn't that darned expensive that they can't afford it. Flashlights are marvelous gadgets - but let's see you take X rays of a fractured skull with flashlights. Or keep baby incubators warm and the oxygen flow regulated with flashlight batteries.
I have a 3.5 kilowatt gasoline-powered generator in my garage. I bought it as insurance for my home in case of a power failure; it's been there for fourteen years and hasn't been run for anything but routine testing - but it was cheaper than insurance on the food in our freezer would have been. And it assured light, oil-fired heat, and some electric stove power, as well as freezer and refrigerator maintenance.
I checked our local hospital; they have a 35 kw generator system with an automatic relay cut-over that starts immediately if the power line fails. This supplies all the operating rooms, and the emergency ward. An additional 100 kw unit cuts in manually, as soon as the plant engineer reaches it. The two can handle the entire normal power load of the hospital.
But New York's great hospitals weren't ready.
Wonder if your local hospital would be prepared if the power failed? Or, when a disaster made it most needed, would it be crying for someone else to supply them with what they know they'll need in any major emergency?
Or does your area have a special dispensation from God guaranteeing that no disaster through which no power transmission system could survive will strike?

The question of "what" caused the Blackout seems to be settled; a "sensing relay", part of the protective system of the great Sir Adam Beck hydroelectric power plant near Niagara Falls, in Canada, was the immediate cause. It's essential to understanding the Blackout problem to know something of the characteristics of modern power plant machinery - particularly you need to know how extremely sensitive to overload the huge generators actually are.
A modern gigawatt generator, if short-circuited, can destroy itself within ten cycles - one sixth of a second.
That very simply explains why you can't leave the protection of the generating equipment to human engineers' reactions. Overload relays, to be of any value as protective devices, must react in appreciably less than that sixth of a second. Old and now obsolescent relays react in six cycles; modern gear can open the circuit within three cycles.
And that means a switch-gear massive enough to handle the short-circuit currents put out by a gigawatt generator - not merely the normal billion watts power, but the ruinous-overload level of power from the massive machine, with some thirty tons of metal whirling thirty-six hundred times a minute.
It's done with "solid-state devices" all right - solid masses of copper, magnetic iron, and steel. You don't use semi-conductors when you're working with currents of that magnitude.
But equipment that massive is not, itself, very sensitive to critically important changes; therefore, the massive heavy-duty switch-gear is operated by a smaller "sensing relay" which can be adjusted to the desired level of sensitivity and which - because it is relatively small and light - can react very swiftly. The sensing relay then slams a triggering current into the heavy switch-gear's operating coils, the whole sequence from first sensing to final opening of the line taking less than three cycles of the sixty-cycle current.
It was one of the sensing relays at the Sir Adam Beck that first kicked out, throwing its associated big switch-gear open. One of the lines carrying the Sir Adam Beck's power to the Ontario area loads opened. The four remaining power lines were, then, overloaded, since the peak load demand of an early winter evening was on the system at the time.
Naturally, the now-overloaded lines were kicked off by their associated protective relay-switches.
The Big Trouble began happening at that point-for the Sir Adam Beck plant's huge power output now had no place to go but down the heavy-duty tie line feeding into the New England power grid. The Big Trouble for the power engineers of the world started at that point also. Because theoretically, that sudden diversion of power into the system shouldn't have caused any trouble that the regulating equipment couldn't handle. Normally, in a power tie-in, if one generator starts to run a little bit faster than the dozens of others on that line - if it starts to get a little ahead of the others - the electromagnetic forces operating in the various generators tend to make the other generators stop generating, and start acting like motors. Remember that any generator is a motor, and any motor is a generator; it's the generator effect of back-EMF (back Electro Motive Force) that limits the current flow into an operating motor. If the other generators start tending to motor instead of generate, the too-fast generator suddenly finds itself trying to drive the whole system, and carry all the load. This tends to dampen its over-enthusiasm quite promptly; it slows down.
Equally, a lagging generator tends to be driven by all the other generators, and forced up to proper speed.
The trouble with this theory is that it doesn't adequately take into account some five hundred miles of power lines, plus the fact that it takes time, even at the speed of light, to go five hundred miles and get back again. And the high inter-connectivity of the power-grid network caused a wild sort of system of electrical surges to go ramming up and down and back and forth. Since there were n different paths through the multitude of interconnecting lines between any two selected power plants - no two paths being the same length, or having the same time-delay characteristics - the result seems to have been instant chaos.

One important problem - one that Professor Paynter of M. I. T. pointed out some years ago - has to do with the shutdown characteristics of power plants. Because of the immense mass and momentum of the water in the intake ducts of a hydroelectric plant, when the induction valves are suddenly closed down, for a brief time the hydro plant puts out more, not less, power. When a big modern steam plant is suddenly shut down, it behaves quite differently. The usual setup in a steam plant involves steam flow from main high-pressure, high-temperature boilers, through induction valves to the topping turbine, then to a reheater, then to the main turbines. Close down the induction valve, and the topping turbine is fairly quickly deprived of power. But not until the steam on its way through the reheater has been exhausted does the main turbine lose power. The interactions of hydro and steam plants on a tie line remain an unknown quantity, simply because the best theoretical analyses by different, and equally competent experts, disagree.
And therein is another reason why computer simulation. analysis of the power network was impossible. The best experts at General Electric - which knows something about the business - and at M. I. T. are in disagreement as to what the proper equations are, and have been for a decade or more. And a mathematical simulation is, of course, always somewhat less reliable than the equations it's based on.
So far as mathematical logic is concerned, it's as far beyond our present techniques as a dynamic analysis of the motions of all the stars within four hundred light-years of Sol. The computers are just about able to solve, by successive approximation, the behavior of the ten major bodies of the Solar System for a reasonable time in the future.
The power network reacts in a time scale several billion times faster than the Solar System, and has several hundred force-generating components, instead of only ten gravitating bodies.
That's why the Great Blackout itself is a Type I mistake - the result of unavoidable ignorance. There was no way to analyze beforehand how the system would behave.
The unfortunate fact is that, moreover, we still can't analyze it. The only thing the Great Blackout has taught us, basically, is that that interactive breakdown phenomenon is not as improbable as we were, previously, convinced it was. Because we can't solve the problem of the complex interactions, we can't determine what to do to prevent it in the future. Another Great Blackout remains perfectly possible - just as possible as the first. And, incidentally, by the nature of the problem, will become constantly more probable as the complexity of the problem increases - as the interconnected power system spreads wider and wider.
When you can't find a way to prevent a breakdown of something, the thing to do is to figure out a reliable, fast way of reestablishing it when it does break down. And that didn't exist as of November 9, 1965 - it took up to twenty hours to get things going again. That is one area in which the power engineers can act with knowledge and understanding; arranging better restart methods isn't in the realm of the utterly unanalyzable. However, until the Great Blackout demonstrated a realistic need for such arrangements, they appeared, to all sane power engineers, unnecessary, and expensive frills - economically unjustifiable. Like carrying built-in lifeboats on interurban buses because there might be a flood requiring lifeboats to reach safety.
The power company's start-up problem comes in two pieces: in-plant power-supply, and picking up the load again.
As stated above, the protective switch-gear operates in small slices of a fractional second; it has to. Imagine New York City's position if the switches hadn't opened, and Con Edison's great generators had been ruined. Gigawatt generating plants aren't something you pick up off the shelf from your nearest electrical supply store. They aren't made to order in a few days, either. It takes months, even if the most extreme hurry-up overtime methods are used. And what would New York City have done during six months without electric power? The quantity of power involved simply couldn't have been made up by other plants in the northeast. For one thing, the forced evacuation of several million New Yorkers would have imposed great abnormal loads on all those other northeastern plants.
The slow dimming of lights in the area - the lights flickering and waning over many seconds of time, as they did - seems to show that the generators didn't cut off the line suddenly. I'm told, however, that this appearance was caused by what might properly be called "putting down the load". As stated above, every generator is a motor, and every motor is a generator. When the generators were yanked off the lines millions of electric motors, all up and down that line, were spinning. Some were huge multi-thousand-horsepower units and some were tiny electric clock motors, but in their myriads, they'd been absorbing a major part of the output of all those generating plants. Some were driving electric trains; some spinning heavy machinery in factories, some were massive "synchronous condensers" idling on the lines to keep the line power-factor under control.
When the generators quit, all the stored momentum and kinetic energy of those motors started feeding back into the power lines. The slow dimming of the lights represented the slow braking to rest of all the motors on the lines.

Part of the problem the power company engineers faced when they did get their plants running again was that they had to pick up all that dropped load - they had to get all those motors spinning again.
Their most immediate problem, however, was that they didn't have any in-plant power to get started with. Picture mentally the engineer on the control desk at Con Edison's great Energy Control Center in New York City. Before him is the console, studded with dials, switches, push-buttons and scores of remote-reading instruments. From that console he controls the opening and closing of the great steam valves - speeds or slows the automatic stoker mechanisms feeding the massive boilers - adjusts the excitation to the generators - operates the entire Con Ed system of power plants.
But just at the moment - say five minutes after the Blackout started - he's standing in helpless horror. He can't see his console, because there's no electric light. When somebody brings a battery-operated emergency light, he can see it - but the instruments are meaningless. Many of them are servo-operated remote-reading instruments; with the death of the power supply, there's no correlation between what the instrument reads, and what the situation is. The purely electrical instruments all read zero - zero volts at zero amps producing zero watts.
The valves regulating the flow of steam, boiler water, lube oil, and fuel to the system are electrically powered - and like modern automobiles, they don't have hand cranks for use in emergencies. If your modern car stalls, you get somebody else, whose car is operating, to push you to a speed that will start your engine, and get going that way.
That's the plan the power companies had; if something did force them off the line - if their generators did have to shut down completely, so they lost their own in-plant power - they had tie lines to two or more neighboring power plants that would supply them with needed in-plant and accessory startup power. A neighbor could give them a push to get started again.
It didn't work this time, of course - because all the neighbors in New England were stalled simultaneously! The oil-fueled plants didn't have electric power enough to heat the oil which wouldn't flow - it's only slightly less viscous than road-tar when cold - to the oil pumps that couldn't pump without power. The coal-fired plants couldn't operate without power to operate the automatic stoker and ash-handling equipment. And they still needed electric power for valving and control systems, all designed to be failsafe, and shut down things at once if the power to the control systems failed (Without controls, how do you keep the boilers from exploding - or the turbines from running away, and bursting the equipment by sheer centrifugal force?).
The New York power system got started by reason of the fact that a small power plant on Long Island - the Rockland Power Company - had gotten off the tie lines quickly enough to save its in-plant power. The night engineer on duty when the blackout hit did what no computer system, for all its nanosecond speed of action, could have done. He broke the rules, threw away the instructions, disobeyed orders, and slapped open all the main switches on his board when he saw that all his meters were going far into danger zones - and the generators were whining down under some terrific overload. (The tie line to New York was trying to make Rockland's small plant power all the electrical equipment in New York City).
As soon as he broke the tie line to New York, his generators recovered, voltage went back where it belonged. He found that the outage was on the New York tie. (He assumed, naturally, that the line was at fault, not guessing that the whole Con Ed system had collapsed, along with everything north into Canada, east into parts of Maine and New Hampshire, and west as far as Detroit). So, since his own local-town lines were all right, he quickly restored the power to Rockland - an oasis of light in the great darkness!
It was the small Rockland power plant that, later, gave Con Ed the "push to start" they needed - the in-plant power to restart their own greater generating equipment.
Somewhat more amusing was the sequence that got the Boston area power plants running again. They, too. were in the spot of the no-hand-crank-and-all-the-neighbors-stalled-too.
M. I. T. being an engineering school, has a power plant of its own, partly because they need an electric power plant to train electrical engineers, partly because they need some special types of power to feed the many laboratories, and partly because laboratory scientists doing experimental work frequently need huge amounts of power for short, irregular and unpredictable times. That sort of load is something a commercial plant can't tolerate; it louses up the voltage regulation for every other customer, and produces violent and destructive surges in their equipment.
So, when the Cambridge Power Company found what the situation was, they hopefully called M. I. T. Did M. I. T. have its plant in operation? Unfortunately, no. It'd been closed down for the day. Could they get it started up? Well... they did have steam, by diverting steam from the dormitories and buildings for a while... but they didn't have start-up power either. However... hm-m-m... look, we'll call you back.
And presently, triumphantly, the M. I. T. plant fed start-up power to Cambridge, Cambridge could then feed start-up power to Boston Edison, which fed it down the tie lines to the rest of New England.
M. I. T.'s plant, being an engineering laboratory plant, and equipped with various types of laboratory-supply generators, had found a solution. But it must have been a somewhat odd sight to see the crew out lugging all those automobile storage batteries in to crank up a power plant!

April 1966

END