Did This Part FAIL Cause MV Dali Key Bridge Collapse?
Vložit
- čas přidán 25. 06. 2024
- The NTSB released an Investigative Update with Release Date 24 June 2024.
This report provides an update to the NTSB’s preliminary report issued on May 14, 2024, concerning the March 26, 2024, contact of the containership Dali with the Francis Scott Key Bridge in Baltimore, Maryland, and the subsequent collapse of the Key Bridge. Investigative activities have continued since the preliminary report was issued.
NTSB investigators have completed in-person interviews of the vessel’s crew. Onboard examination of engineering systems and testing of electrical systems has been completed. Documentation of the damage to the vessel structure is ongoing. Jeff Ostroff gives you an analysis of the suspect electrical part that the NTSB removed off the MV Dali ship to investigate the cause of the electrical failure that led to Dali crashing into the Francis Scott Key Bridge on March 26,
During the accident voyage, electrical breakers HR1 and LR1 unexpectedly opened when the vessel was three ship lengths from the Key Bridge, causing the first blackout (loss of electrical power) to all shipboard lighting and most equipment. While examining and testing the vessel’s electrical power distribution system and control circuitry, NTSB investigators (in coordination with vessel crew and parties to the investigation) noted an interruption in the control circuit for HR1’s undervoltage release. - Jak na to + styl
Hi Jeff thanks for the great video. I work in the railway signalling field and we use WAGO terminals extensively, even the exact same 2.5mm2 terminal that the NTSB has shown. WAGOs are one of the best terminals one can use and the main reason is that they are vibration proof. Since WAGOs were introduced in the railways, we seldom get any faults as a result of vibration which is a significant factor adjacent to a railway line. HOWEVER, there is an achilles heel and this is down to competence and training.
If your video, the WAGO film clip shows one type of terminal where the cable core is pushed into the cage clamp. The other type of terminal (aka the NTSB photo) is where the cage clamp has to be pre-opened and the wire inserted. This is where the the achilles heel comes in. WAGO provide an insertion tool which is made specifically to open the cage clamp to insert the wire. You are also trained to ensure that the wire is stripped to the correct length so that when it is inserted, it fully engages with the clamp.
The problems start when a person does not have the correct tool and decides to use a flat screwdriver. The problem gets worse, when a person decides to use an electrically insulated screwdriver. This is where the shaft is insulated and only the tip of the screwdriver is exposed. So what happens is that the screwdriver is inserted into the WAGO and the uninsulated part goes into the clamp but butts up against the insulated part of the screwdriver. This prevents the cage from opening fully.
So when a mult-stranded cable is inserted, especially a small 2.5mm2 type, it does not fully engage at all and nominally a couple of stands may be lucky enough to be clamped, but the rest contribute to a loose terminal connection.
When we first stated using WAGO's, loose wire terminations was a major problem and only through on-going training and ensuring the right tools were used was this problem eliminated. As I have already said, once correctly terminated, you never need to check it again - maintenance free.
Certainly going to be interesting reading the final report, but I will put a wager on that this was the root cause. Keep up the great work.
From my reading it isn’t that an undervoltage condition existed; rather that the circuitry to detect an undervoltage condition had a fault that caused the breaker to trip when it should not have, e.g. no undervoltage condition existed.
that was my interpretation - the system that CATCHES low voltage, had a connection problem, and dumped the breaker.
Kinda what I was thinking too. A fault in the fault-detecting supervisory equipment.
If the sensing equipment or breaker trip unit is fed power its sensing from this din rail connector thing and a wire vibrates loose or breaks than the breaker is gonna trip
"Low Voltage Release" is a common feature of power circuits that will automatically disconnect the loads from a bus when the bus is lost. The idea is if the HV bus is lost, you want to trip all the loads off the bus so when you re-energize the bus you're not overwhelmed by all the loads still connected.
But if the control circuitry 'senses' a low voltage (due to faulty connection) even when the bus is operating normally, would 'trick' the breaker into thinking there was a voltage problem and trip the HR1/ LR1 breakers.
I remember an incident at work where we had an emergency diesel start unexpectedly. It was caused by a blown fuse in the voltage-sensing circuit so the engine controls 'thought' there was a loss of bus and started. Fortunately it didn't try to energize the bus (which was still energized from main power) because the breaker controls used a different voltage sensing circuit.
If the bus is still energized from main power, doesn't the VSC detect this condition and terminate any connections or bridging?
@@1Patient Well that's the idea, but to 'detect' that the bus is energized, it uses a small relay connected to the bus. But that wiring is through small wiring and terminal connections, etc... If one of those wiring connections is faulty, it may drop out the relay even though the bus is still on. It's a matter of faulty 'sensing' circuits.
@@mikefochtman7164 I guess what I am getting at is, I'm thinking there should be a redundant system that detects the detectors fault and reports that back to the controller so it can continue on as it should. Having said that, if there was a ground fault and the main bus did have to shut down... would it melt the mains? I know, it might sound like a ridiculous question. 😃
Control voltage has nothing to the 6600 or 480. Low voltage typically is 24v that used to control the high voltage devices. That terminal block looks like a low voltage or control voltage terminal block. Usually less than 30amps at 24 dc volts
I agree.. Also, I really doubt they used aluminum wiring on a safety circuit; therefore I am inclined to favor a bad contact, possibly caused by too small a gauge of wire, i.e. design flaw
I don't know anything about large vessels, but for boats there's marine grade wire. It's tin-plated copper stranded wire similar to milspec aviation wire. I don't know of anyone that uses aluminum in harsh environments. However, copper clad aluminum (CCA) wire or devices that use it run rampant on Amazon. Hopefully they aren't buying wire off of Amazon :-)
The same blocks get used up to 1000V.
Control voltages can be between 240V and 12V both AC or DC.
I have run into 240V DC battery supplies to HV breakers.
These are spring loaded terminals which use springs to clamp the wire unlike the older screw terminal blocks.
@@allangibson8494 These are commonly used for connecting solid wire only. Animation showed stranded wire with a sleeve in there, which I personally would not trust that much. Connectors for stranded wires usually have screws or very strong spring clamps, you may have one at hand if it is bear to open even with a tool. Spring has to be strong enough to deform sleeves on stranded wire terminals and keep it constantly compressed to maintain reliable connection. In short, these types of spring loaded terminal blocks exist but aren't that common and quite probably have notably higher price tag.
Alu termination paste, I'm not a certified electrician but I know that stuff and use it regularly in places where Alu stuff connects with Cu/bronze stuff. Bushings or connectors may have it applied from factory, just know what type to order. Have repaired some 400V underground Alu cables with those and no issues ~20 years later.
If it was a WAGO 280-835 then it is rated for 800V and 20 Amps.
But yes, that is not the HV side, in this case used for the control side of the contractors.
And the video to explain it's function was the wrong one. It is not a clamp for direct push in. The spring has to be released with a tool before inserting the wire. From this they are not only allowed for solid wires but also for flexible wires without sleeves. But you should be trained to do that without mistakes.
Regards from from WAGOs home land.
Keep in mind that the electrical diagram used to illustrate the first NTSB report is neither a wiring diagram nor a schematic. What is shown as a high voltage breaker is actually a set of detection circuits operating a contactor. Each of low voltage, high voltage, no current, or excessive current could cause the control circuitry to open the contactor. In fact, there could easily be separate low voltage detection for each side of the contactor and also for the other side of the transformer. Most of these kinds of control circuits are designed to fail safely. And for that reason any fault should open the contactor separate the load from the power source.
A home breaker panel doesn't do any voltage monitoring for the ordinary thermal breakers. Those only protect against over current.
I'm guessing there is some sort of interlock between the high and low voltage transformer breakers as you can't have it trying to back feed.
If those are the screwless terminals I can see the advantage in a high vibration environment and they say they are better than conventional screw terminals but I've never really trusted them. For one thing it's quite easy to damage them by trying to release them without the right tool and it may not then be obvious that the clamping force isn't what it should be.
And let's not forget, that originally at the time of the accident, the DALI - NTSB reported the DALI was sailing into port and operating on LEG 2, and that in port Baltimore, just prior to DALI getting underway, the control circuitry was transferred to LEG 1 control operation.
My guess would be the voltage detection circuit would just "shunt trip" the main breaker. Rather old fashioned, but reliable, this concept is still used everywhere the power would need to be cut with a fire alarm system or leaky pipes underground at a gas station..
@@user-pf3ye6yi9n Why can't a system like this be designed to "back feed"? Any given transformer is capable to both step up and step down and is mostly limited by the current carrying limits of the windings. Somebody more familiar with this specific system might be able to clarify whether the symbolically shown transformer is intended for symmetrical use. In fact, what's shown as a transformer could conceivably be a motor/alternator sharing a single shaft.
@@mikebarushok5361 As the heavy loads are on the HV side I'm assuming it's not designed to feed the HV side from the LV bus. All the power sources are on the HV side.
Thanks for your continued analysis of the Dali disaster. I have worked on alot of power systems with DIN track stuff, and in this case I am guessing the breakers are shunt-trip types that are activated by the control system, of which the WAGO DIN terminals are a part of. Personally I much prefer screw-down terminal strips, as the pressure contact ones can be damaged by improper insertions and especially removals of wires. For instance, to remove a wire, you insert typically a small flat-blase screwdriver in the release slot, and then gently pull the wire out. Maybe someone didnt have the release pushed all the way back and just yanked hard on the wire, deforming the spring contact. Or maybe stranded wires were partially bunched up and not fully inserted. Assume this might be what NTSB is looking at. As for alminum wiring, theres no way that could be present in a ship. We dont even use it in data centers due to the corrosion potential.
Ships yes but data centers use AL wire all the time for high voltage feeds, its way cheaper than copper although copper is definitely better, Frugality lol
As a non-engineer medical equipment mechanic (retired) I enjoy your depth of knowledge and your attention to detail. Thanks.
I’m so glad you enjoy making these videos…
*…because we (the viewers) really appreciate watching them so much!* 🙂
You really dug deep… even using an AI image search to find the part!!! 🤯
Who else would even *think* to do that???
And who else would explain this thing so well to us laymen?
Nobody. 😒
I’m really thankful I’m subscribed to your channel so I can keep up with whatever’s next.
🙏
14 years of mechanic working on cars full time, I've seen some stuff. The thing with connectors is much the same from old to new or a terminal block on a ship. They rely on clamping force and making good conducting contact with the male and female part. Corrosion and oxidation can hamper conducting contact. But vibration/movement will cause friction erosion, removing material and lowering the clamping force. Metal can bend and you have a loss of clamping. With a loose male part in a female part you can still lean to one side and make good contact but it can also move away and not have contact. And thus you can not notis a 1ms non contact moment but sense circuitry is going to trigger of the non contact moment is long enough. Sadly the 1 wire at the wrong time can cause Dali to collide with the key bridge by opening the breaker contactor.
But, from my work of 27 years in electronics of fabrication machines, usually the screwed terminals fail, less so the spring loaded ones...
And as a reminder: In an earlier video Jeff said they should have used both transformers in close quarters. In this case the breaker tripping wouldn't have ended in a crash.
My work on fire alarm systems, typically 24vdc, I have encountered problems with this style connector in every installation I've encountered using spring loaded only connections. (always WAGO) particularly with small signal circuits (0-24vdc, 50ua to 20ma). This is usually not a problem in higher voltage/current AC applications.
Could it be a 4-20ma current loop circuit broke, leading the receiving equipment to trigger a breaker shunt vs signalling a disconnect?
Sounds like an intermittent loss of power to the UV relay caused a protective action to protect the transformer. Could be as simple as a loose connection.
But like i said if you have all 4 gen-sets running in sync you could loose one, tie breakers open and you have not even a blink!
@@ch3no2killz Could also run all four DGs, split busses, as looking at the single line diagram, then a failure on one side wouldn't de-energize the whole ship.
"an interruption in the control circuit for HR1’s undervoltage release." surely suggests an intermittent circuit in the CONTROL circuitry of that system. In other words, they've taken away that connector block with the connected wires to flex them, shake them etc to see if that was the cause of the "interruptioon"
A loose connection in the control circuit would be my guess. I’ve seen loose connections cause I2R power losses. This causes a voltage drop across the terminal and sometimes lots of heat that will melt the terminals.
It looks like the blocks are rated for 600vac. Could have been used on the LV side of XFMR or in the control circuit. If in the LV side the I2R issues would cause the voltage to trip depending on the LV connection location.
Yes. Loose connection. That is why Ntsb has taken out also part of wiring, and Ntsb did not take the wires out from connection points.
x navy, 40 yrs. as IBEW wireman, used many of this type blocks, nice pc.. My question is what idiot would spec AL wire on a water craft of any type and most of all on a ship used in salt water? One other point is you have redundant systems, in situations like this why wouldn't you have both systems up and running, you run into problems your tie breakers could dump the problem and you would be backed up with NO interruption. The mode they were in takes minutes to spool gen sets and sync up, time they didn't have. The whole thing stinks!
Beancouters. Having said this, worked on genpacks also marine never used these wago type termination blocks always used screw type with crimped on ferrules. Only some low priority DCS wiring was push-in terminated
@@dubro5009 In my 27 years of work on fabrication machine as an electrician I a had a few screwing terminals fail. Not a spring loaded one (so far).
I'm quite sure the AL-wire came from Jeff himself because he had seen the notions in the docs for the terminals.
When I was training in the RAF in 1977 we were taught to use a white paste, the name of which I have forgotten, when crimping terminals onto aluminium cables. I never actually saw any aluminium cables in my time working on aircraft though.
@@martinconnelly1473 A few colors and brands, No Ox is one and it's dark grey, just to keep oxidation down by covering it with special grease by keeping the O2 out, one of Al's main problems. Under voltage can also be from overload, your breaker of time delay fuse gives you time for heavy start loads, motors etcc.. The other is the one everyone thinks of first, instant trip is what you would call a ground fault Once again if you haven't been around salt water, only and idiot would use Al. Navy says, yeha right.
I've repaired many plugs in houses near the beach where the connection to the plug were through the push in connectors in the back of the plug. Even if copper wires were used in these connections, corrosion could occur in sufficient quantity to make the connection fail especially in a salt water environment
I have also seen the same failure where I live with these push in connectors. I re-wire them and use the screws on the outlets and switches. Then WACO sells these other spring clip things to use instead of wire nuts. I like the convenience they offer. Now I question their safety.
the problem is even worse for low voltage/current applications, such as control circuitry, where there isn't enough "Oomph" to push through a dicey connection
OK, An undervoltage release coil is designed to be powered at all times. If it loses power, it will release it's plunger to operate a lever. This lever will release a spring which causes the breaker to trip. A momentary loss of voltage will cause the breaker to trip and unless there is a method of recording what the particular control voltage is doing, there is no way to tell what caused the trip.
As to what caused the momentary drop out of the control voltage, there are any number of things. With the Terminal alone, I can think of three or four alone. The cages could have failed due to age or metal fatigue (they are only about 1mm thick). There could have been corrosion due to the sea air reacting with the copper wire or cage. I have yet to see a control panel wired with aluminium wire but for runs between different boards it may have been used. There could have been a hot joint which heated the cage or even started to burn out the terminal.
I noticed that the photo in the NTSB report is taken from the a catalogue showing dimensions. The terminal in question is also designed to have a small screwdriver insert to lever open the spring cage before inserting the wire and then releasing the cage to hold the wire against the clamp body however if someone was to really put pressure on the screwdriver while opening the cage it could bend the cage itself which would cause less force when holding the wire.
That said, the terminal may only be the first component in the control line when tracing back from the breaker. There could be an old fuse further back, a faulty control power breaker or any number of other components...
The downstream breaker most likely tripped due to the upstream breaker tripping. For HV/LV setups like this, it is recommended that the HV breakers do not CLOSE onto a load. so the control circuit would have most likely opened the downstream automatically.
Why the crew did not close HR2 & LR2 instead in case of problems with TR1 will also need to be looked at as a failure in operation. Doing this MAY have prevented the second blackout.
Regards
Was just thinking about Dali today. Thanks for sharing the report Jeff.
Per usual, great explanation that even a layman like myself can understand.
Interesting that they isolated this one part and sounds like associated wiring with it. Thanks for the video.
Very early on, from either the news or CZcams, it was mentioned that refrigerated containers were connected to the power system and there may have been in-port power issues. Would it be possible that the overall connected load (ship systems + auxiliary loads for refrigerated containers) caused the upstream breaker to trip? Great spot on the WAGO connectors. One would assume that all electrical components would be approved for marine use (you mentioned vibration). The control wiring would likely be insulated, stranded, 12-16 AWG THHN-like wire. One would hope the WAGO push-in connectors were approved for stranded copper wire if that was used.
I was thinking the same. The refrigerated containers on the 440v buss caused the overload.
@@bobbym6964 No, the refrigerated containers are fed from the 6600 volt bus BEFORE the TR1/TR2 transformers. The container have their own 440v step down transformers located closer to where the reefer containers are lashed. There are many 6600 to 440v transformers dedicated to reefer service. The main reason being, it's cheaper to buy a few step down transformers, than to run heavy guage 440v cables all the way from the rear of the ship's engineering spaces all the way forward to the reefer containers. You can send a lot more kilowatts down a thinner cable at 6600 volts.
The diagram shown on screen seemed to indicated the style of wire that was acceptable, and I did see stranded wire of various sizes shown.
The image shows a wago 280-681 of the cageclamp series. It can handle stranded wire of all types. But if aluminium wires are used, it seems like a special contact paste must be used.
I use those Wago terminal blocks every day. They are the best in the world if used correctly.
conversely, I tested Wago lever nuts in a corrosive environment, and even with high quality de-ox, they had a working lifespan of less than a year.
unless you get a counterfeit one.
Why would you put a wire under a spring to hold it verses sticking it under a screw and tightening. I have seen old "knob and tube" wiring still working great with all the connections using screws.
Spring-tension connections resist loosening from expansion, contraction, and vibration.
@@thebigmacd springs will loose their tension and elasticity during heating and cooling cycles under load.
They WILL get hot with just a lousy pinch of copper plated steel digging into a conductor.
Who in their right mind would use a device that connects wires contacting less area than the total circumference of the wire itself.
Give me a full 1/4 inch of conductor clamped under a screw clamp . The more contact area, the better.
I will die of laughter if a WAGO style connector is to blame for the ship hitting the bridge.
Jeff! I'm surprised. You should know that shipboard engine rooms are constantly hosting low level vibrations do to the running machinery.
Apparently they had power problems whilst loading reefer containers that drain a lot on load on the generators. I wonder if engineers onboard when fixing the problem of losing power ashore did something wrong or did a quick fix by bypassing certain electrical components to prevent power tripping. Not an Engineer just speculating.
As a previous person commented I still would like to know reason for a lot of smoke coming out of funnel from ship when power regained. I still feel that if ship had power problems whilst loading a proper investigation of root cause should of been made before departure. Maybe it was then I am incorrect. As an ex Captain on big ships there is a lot of commercial pressure placed on Captain of ship to sail on time to keep to sailing schedule. This is the dangerous part of shipping nowadays because often commercial pressures overrides safety concerns. Another assumption I am making is that maybe Chief Engineer and Captain were aware they had small problem onboard but thought it had been sufficiently fixed without having time to find out what caused problem.
Just speculating: They might have searched for the failure and found something suspicious that looked like the cause.
Don't always assume the worst in people.
And you should keep in mind: When a organisation like the NTSB searches for the cause they turn every stone, look for all the nooks and crannies. I don't always check every single wire in every terminal. To repair like this would take weeks (as does the investigation).
@DrKlausTrophobie not sure if your comment is directed at me. I have one question to ask you before I think it is necessary to expand my comment in more detail for your understanding. Have you worked on ship's at a senior level as either a Captain or Chief Engineer?
@@brianbarbarich1401 No ships, just 27 years of electrics and electronics in a manufacturing plant. Where the goal is 98% availability and (especially at the assembly line) every lost minute counts...
@DrKlausTrophobie OK similar environment where time = money. My point was that sometimes things on ship's are not 100% repaired before sailing because of the stressful working environment. Not finger pointing anyone on the Dali just generalising how the shipping industry is run nowadays. As a ex Captain I myself was guilty at times of sailing my ship when I knew inside it was potentially unsafe but did so because of tremendous pressure applied on me by the owner, charterer of my ship and the Company I worked for.
@@brianbarbarich1401 Thanks for clarification.
In my system engineering work I've used Wago parts extensively including similar din rail modules like the one pictured. They are well made and reliable. They are potentially useless if you do not properly tighten the screws. It is possible to tighten the screws down without being on the conductor(s) you want to secure. If the control circuitry for the buss breakers was intermittent it could be perceived by the system PLC as a low or no voltage and drop the buss connection. 😮
I have always figured that most of these old ships have gremlins in their electrical system. As these ships live in salt water conditions, I bet they have some wear and tear in those systems. Great job Jeff.
The connection points could suffer from oxidation.
Great explanations... thanks for sharing.
I am a retired ME so I am not as familiar with the ship’s electrical system. Another CZcams that is authored by a chief engineer on another ship provided a more complete electrical diagram (not from the Dali), which helped me understand a bit more about the ship’s electrical design. The majority of the ship’s mechanical equipment are powered off of the 480 VAC circuit. The notable exception is the bow thruster and the main ship propulsion. The bow thruster was not in use as it is only used when leaving a dock. Of course, the NTSB has the actual ship’s as built drawings and the maintenance records that they will be used to zero in on the failure. I’m wonder if the main generators may be the cause. Time will tell.
Pictured is a 3-pole or 1-to-2 wire expansion IEC terminal block. It clips into a rail, called a DIN rail, with other IEC terminals or any number of other components - it is a universal mounting system.
Over the years, I have used many thousands of IEC terminals of various sizes and configurations in the construction of industrial machine controls panels.
These terminals are typically rated at 300 to 800 Volts depending upon size and design. What is not shown on the drawing is the current rating in Amps, which determines the current rating in Amps.
The wires are attached to the terminals by one of two ways: A convoluted flat spring which is opened with a tool resembling a flat-blade screwdriver inserted into a hole at the top of the terminal for wire insertion/removal - or a screw & clamp terminal.
The drawing shows a spring-type terminal. The idea behind the spring terminal is rapid connection - push down on the insertion tool through the hole in the top, insert the wire through a corresponding hole, then remove the tool. You may imagine that the screw terminal takes more time and effort to make a connection.
But I prefer it as you can get a tighter more reliable connection with more contact area, particularly when ferrules are used on the wires (to avoid split strands or breaking strands) which is always best practice.
So what could have happened? This block would be in a low-current control-related circuit - with this block likely being rated at 8 Amps - but we don't know for sure (a width dimension is not shown in the drawing). If corrosion occurred between the spring or contact pocket, high resistance would develop creating heat in the joint. This could anneal the spring, making it lose pressure on the wire, causing an intermittent connection.
Only The Shadow knows for sure.
Thanks Jeff
That's pretty bad if this is the culprit. What do those terminal blocks cost, $5 if you order a box of 100? So $10 worth of some wire and a terminal block caused many deaths and billions of dollars of damage
It's not. Expect lack of proper maintenance, cost cutting. It is not fault of a part. It is rather obvious they did run system in way it was not supposed to. Normaly you should have both transformers side closed, use GEN 1 (or Gen2) and GEN 3 (or gen4) with HVR breaker open. When one sides generator dies for whatever reason, you close HVR, until you restart failed GEN or its backup partner. IT is very likely there was major undealt with fault in one of TR sides. Likely for months prior
@@klonik79 That makes sense, and explains why when they switched to the other transformer, then they had power just fine while stuck on the bridge for the last several weeks.
Jeff, you are wrong to call them "breakers." They are really CONTACTORS, relays, in other words,(not thermally operated circuit breakers) that are controlled by electronic circuits that monitor voltage and current conditions. It's the electronics that may have been at fault.
They are definitely not contactors. Contactors are used to frequently make and break load current. Breakers are switches that are rated to break much higher fault currents. They are often spring operated with motors to wind the springs after they have opened as opposed to contactors that most often are operated by an electromagnet directly. Large breakers do not have the control mechanisms to detect e.g. over current integrated unlike a small household breaker that is a self contained unit.
Good work Jeff, many thanks.
Glad you enjoyed it
Long time ISM was implemented in the shipping industry to avoid accidents. Whatever technical failure happens onboard ISM must following to avoid major accident.The purpose of investigation is to find out if the ISM had been followed but not the electrical terminal
I'd be surprised if a spring type connector is used for anything more then 1 amp. They do make constant contact & don't loosen up from heat cycles, but don't ever make good contact on stranded and/or large wire.
Love how you just have a circuit box hanging around
I guess no one was recording any power quality during this event. It would be nice to see if there were any major events. I would run the generators for 24 hours, while recording the output data. It feels like the human factor is greater than mechanical failure.
A number of comments here have speculated that the harsh salt water environment may have caused corrosion of the electrical connections. If that is a hazard, one would think that an environmental management system would have been in place in the electrical control room.
Equipment can do crazy stuff. This has nothing to do with this case but I worked on a reefer ship where its DC generators would reverse their polarity.
To cool the two hatches we ran three generators, Two MANs and a Detroit diesel. The MANs would reverse their polarity and turn the Detroit diesel into an electric motor by stopping it and turning it in reverse. An older engineer told me that he had seen connecting rods burst through the sides of an engine when this happened. Luckily that never happened to us.
I am an electrical engineer with 30 years of electric utility work experience. The 6.6 KV bus was energized and the two generators were tied together before the power failure. First, Synchronization and voltage considerations were not a factor before the event that caused the trip. In utility systems, we have electromechanical devices that leave a target indicating the tripping current path by a sensor. IE. low voltage, high voltage, over current, under current, change in frequency, bus fault, transformer fault to name a few that come to mind. When a breaker trips there is a lock out relay that has to be reset before the tripped breaker can be closed. Forcing insight and investigation as to the cause of the trip.
I would think on a large commercial ship that the protection scheme would mimic a utility system. From what you are saying a bad electrical connection on sensor wiring sounds reasonable.
Also, from what you read, I believe the 6.6 KV bus was never tripped or lost. Which begs another scenario, the opposite breakers and transformer should have energized after the initial trip. ????????????
Again, it is never a good idea to reset a lock out relay and close a breaker without investigating the cause.
Jeff,
You need to think of LR1 and HR1 in terms of their being relays and not so much as circuit breakers. The relays act more like switches linking parts of the buss bars.
I believe the components you were showing were from a low voltage breaker. M/V Dali was running a 4500ĶW and a 4000KW generator. These are 6.6 kV machines. The current ratings are about 618 and 600 amps. Bus ties will consist of multiple large copper ɓars, similar to 6 - 3" high by 1/2" thick. Special tools and protective clothing are required for 6.6KV work. I would be very surprised to see aluminum wiring used. IEEE 45 code, in 1997, even banned the use of aluminum for protective weave on cable in the shipyards. I have never seen "click" connections on ships. Considering size and the use of multistrand wire, I don't believe "click" connections will work.
There are two distinct, but highly integrated being discussed. The Power Distribution System (PDS) is known to most as a one-line diagram. The Power Mananagement System (PMS) is the computer system that integrates all the machinery and electrical equipment making it possible for the ship to run. You could have a faulty PC card in a relay causing it to trip. Or, you might have a glitch in the line code of the PMS sending an incorrect value causing the relay to trip.
In 1997 USCG rules required that you parallel generators using three lights. When the patterns matched you could close the breaker. We found it to be close to impossible using the lights. Too many variables, far more complicated systems. The lights remained, but we didn't have to use them.
Bob
A difficult problem to troubleshoot quickly in the dark.
Thanx for this information ... the details make me dizzy ... does any of this explain all the black smoke ... that comes from poor air/fuel mixture ... how does that effect voltage? ...
The most likely thing that would involve a terminal block is a loose connection. Loose connections are a lot more common with those than design flaws in the terminal block itself. The protection circuit is probably wired to be fail-safe where any break in the connection (normally interrupted by logic operating a relay, maybe logic directly, or a red mushroom button) - and a loose connection can also break the connection. Loose connection is also more likely to be intermittent. A protection loop shouldn't normally see very much current, so it shouldn't be seeing a lot of thermal cycling. But a ship is a big vibrating environment that would have a tendency to cause connections to loosen up. On connections with current going through them - you can get an early indication of the failure with thermography as the interface heats up due to increased contact resistance as it loosens up. (conversely - if there is not current, or not very much current going through it then it won't be generating enough heat to show up that way). It is difficult to scope and manage a maintenance plan to "go tighten ALL electrical connections" - but there are maybe a few ones that are important enough to have regular effort to check on them. Hard to say if this system would have been included in something like that - easier to deprioritize it because there is a redundant path.
What's the differnce between required condition 1 and condition 3?
Ships of that size don't "bounce around". Too much mass. There'd be some vibration worth consideration around and above the engines and possibly the prop shaft. Could be a concern depending on how and where the electrical cabinets are installed.
How does it knock out the entire system? Is the electrical system not redundant? It sounds like there is more to this story.
This point is tiny, minutiae, a fly. Almost nobody has mentioned the utter absense in either of the two original night time videos of big waves following the big collapse and big splashdown. The second video has audio but not a single sound is heard from the massive, heavy bridge collapse. It's eerie. Linked from the 90-part series on the many anomalies of the BKB event -- it's on the ORGanization (not the COMpany or NETwork) site called "CON" followed by "S" then "P" and with an "IRA" on the end.
As the voltage drops the amperage increases.
Watch a car with a bad battery as it turns over and you can hear the starter slow because of the battery the cables will start to overheat, the slower the starter turns the engine over the higher amperage demand on the battery.
A 20-amp breaker trips when there is a short or when the amperage exceeds 20 amps.
Those parts have contacts which and if the wire is loose or the contact is worn can cause arcing and lead to the breaker dropping out.
Very interesting.
According to the VesselFinder App, Dali has been moved again to the next port. Maybe now the ship's repair work start.
I don't trust any terminal without a lug. Those spring clamps can heat up if the wire is too loose. That is why I refuse to use the quick holes, and I will put the wire under the screw because I believe that more surfaces are touching each other.
Exactly, fault detection control interlock circuits. Wire terminations are made by WAGO among others. The protection circuits just perhaps thought something was wrong...
Ouch, so faulty control wire connection caused a remote trip of HR1? Insane how such a small piece caused such a large disaster. This could have been a sense wire.
These power clamps are intended for rigid cables.
But I don't think it's right to have rigid cables in a ship.
Think about the vibrations on the ship. These do not appear in the house.
However, flexible cables can be used with core sleeves.
I installed water heaters for 20 years and one sub division I worked in every house had aluminum wiring and I used that paste religiously 👍🇺🇸
There are multiple verified known failures in varied industries caused by "engineers" doing exactly what you cited not adhering to the published usage for electrical connectors,plugs,contactors ,and terminal blocks. many times the form factors are the same but the ratings vary within a series if parts. The Aluminum corrosion issue is well known in the electrical industrial arena. Will be interested to see what the final report says and will we reallyk now what failed?
A bad WAGO spring loaded terminal block... Who'da thunk it? ;) The WAGO blocks are a breeze for building/assembly, but my experience is that in severe duty (heat/cold/moisture/vibration) they are notorious for intermittent glitches. I've chased a LOT of problems on industrial gas turbines that use these things. And they can be a double barrel bitch to find the intermittent connection!
Interestingly - The equipment I used to work on was equipped with WAGO Screw-on terminal strips. The same thing as you posted, but they had screw connections instead of spring loaded connections and ferrules.
The other possible problem with WAGOS of this type is when assemblers or technicians install wiring directly into the spring loaded connector. Using the wire directly on the spring loaded contact surface can reduce the contact surface area, and heat does what it does. After a while you have a nice flakey connection. That happens much less frequently with the ferrules.
Good report, Jeff!
*WOW a Bridge destroyed by lack of paste? Or is it a bit of motion? Awesome video!!*
I'd be curious to see schematics and a functional analysis of how Dali sync'd generators. With 4 to use in different combinations, that has to be more complex than just a single pair master-slave design with reduced capacity fallback to either one.
I'm a fan of Amp PIDG spade lugs, with MIL-39012 oxygen free crimps and insulation retention. Not as easy to wire as those Wago blocks.....
Aluminum in a marine environment? Hydrogen replaces Aluminum in an electrovoltaic series, whereas copper replaces Hydrogen. IOW, even without a salt environment water over time can make Al wire turn to mush of Hydroxide flakes and powder. Why risk No-Alox or other brand equivalents?
Electricity in a marine environment needs also particular attention, with the humidity and salinity. This can corrode stuff big time. Some equipment might be suitable on-shore, but not on ship.
Under voltage protective relays are normal in any power distribution system to protect equipment from damage. Wago push type terminals are notorious with working loose or over heating from current. Poor craftsmanship like not stripping wires right length, broken conductor strands and how far the wires are inserted all play a roll. Sometimes, it's takes years to show up. I've seen first hand how hard problems like that can be to track down as the connections cool off and read good on the ohm checks but under load can go open circuit... Other factors like mech stressing the wires as a bundle on a movable door can lead to looses connection problems.
The (dis)advantage of uncontrolled automation. They saved one sailors wage and lost a ship, a bridge and some cargo.
Nobody in their right mind would use aluminum wire in this situation. That said, if the insulation of the copper wire was stripped to less than the spec (.31"-.35"), it is possible that the insulation could interfere with the conductor to terminal connection and cause an intermittent connection especially under vibration. (BSEE '67)
Is this terminal block type approved for marine usage (corrosive environment) ?
Or could a wrong part have been used here ?
Low voltage protection - likely part of a check to make sure ALL 3 phases are 'up'. This isn't single phase gear here ...
yep, there's motors on the lower voltage buses, as well, and if they single phase, all the smoke leaks out.
@@kenbrown2808 Yessir. We lost a rotary frequency converter (Motor Generator set) on a chiller in an HVAC system on a building back in '74 ... one phase dropped the other phase currents 'picked up' and eventually we had little copper balls on the floor when the copper field coils melted ... so, missing phase protection is VERY important!
In my opinion the contact area of the wire and the terminal contact is not sufficent to conduct the amps needed for the circuits.
There were reports that the electrical system was overloaded by the number of cooling containers. Might this be a reason why the voltage broke down repeatedly?
The cage clamp design IEC terminal block should have never been approved in favor of the NEMA screw terminal. Wake up DNV... German engineering!
Yeah, how many times have you retightened those screw terminals with vibration and hot cold cycles?
Failure to use a ten dollar tube of Alu-Plus Contact Paste resulted in the loss of a billion dollar bridge. That's what I call leverage!
Go back to your NTSB report video. The ship had the same problem at port resulting in power loss, low voltage causing the breaker to trip. There was either a problem with the generator or the breaker.
I didn't see any specs on those terminal blocks, but I certainly hope they were used on low voltage low current applications. Any type of electrical connection that is made with spring loaded contacts and not from clamping force from a bolt or screw is questionable at best. Personally I'm shocked this is even allowed on any ship, especially in US waters.
It sounds like the low voltage detector was connected via that terminal block, and acted as an N/C switch, opening when a fault happens. That's considered to be a "fall-safe" design, since a false trip is considered to be less dangerous than failing to trip on a hazard condition.
Ironic. *_Very_* ironic.
Oh, and i wonder if whoever wired that panel read the fine print about it taking stranded wire. You have to use a ferrule on the wire or hold the leaf spring open with a small screwdriver while inserting the wire. (There's a slot on the connector for that purpose.) If you jam a stranded wire in there blindly without a ferrule, it will just bunch up in the hole.
While getting to the exact source of he problem is important, what is more important to regulators and insurance people is did similar failures occur when the ship was in an open area and such a failure would have very little consequence. IF there were previous situations and the cause was not identified, that speaks to neglect. If this was the first time, it was at a most unfortunate time.
Ahh Wago I've used a few of them instead of wire nuts. Halo high hats had similar quick connect push in connectors. Much faster good for hot attics.
I did not know you were an electrical egineer too. Wow.
I work with similar equipment on shore. Under voltage relays are all electronic.
a one line diagram is just the beginning, where is the control schematic and all the protective devices, CT's, PT's used?
next question....Wago connector or chineese wago look connecter?????
Push in connectors generally are not very solid, although people are speaking highly of Wago. The connecting wire could have been undersized. The voltage is actually very high. I think the diagram said 6,000 volts.
I do not trust any form of push terminal. While they're great for the installer, they are terrible for reliability. And undersized wire makes them even less reliable. Stranded wire can also cause problems.
These types of connectors can also have insufficient contact area with the conductors and the spring tension can suffer from metal fatigue.
I avoid them.
THe havent bee using the path HR1 and LR1 for months. Did they experienced a similar incident in the past ?.
im not a fan of wago style connectors there is such a small point of contact even when correctly installed
I still feel like foul play is afoot, but I can't figure out to what end.
I am still laughing at Lara Logan claiming the GPS was spoofed. And her claim that she learned that from undisclosed sources. OY!!!
That is a passive terminal block, control wiring should have been copper. Wago has 40 years in the design of ghat block and i work with them a lot in new equipment. Most failures I see are installation induced, IE wrong wire size, no ferule, improper tools used in install or exceeding current spec of part.
WAGO connector???
There is a big push to use those things on line voltage wiring in your home....everything from light switches to outlets use those silly things...I am now replacing 10 year old outlets due to overheating from these connectors....
They likely were not wago but cheaper knockoffs or pushing that come with many light fixtures. Wago lever nuts are tried and true I use them
Yea, it is going to come out that it is just some sort of fault in some part and nobody is to blame.
The real problem is that ships nowadays shut down the main engine and the controls when there is a power fault. That was not the case on older ships.
Ships were designed to survive individual part failures. That one was operated incorrectly.
pretty sure that tiny DIn rail terminal/fuse isnt the part......should be some pretty big switch gear and breakers involved.
Hi Jeff on the day of the accident we had an big influx of plasma from our sun so the ground currents were elevated,so I am thinking that this might of tripped the safety switches on the boat . Just a thought. Things we cannot see are sometimes to blame.
no likely they are all shielded by metal, even cell phone signals not likely to get out
This is 'life safety' so you could ask the question 'why have an under voltage coil at all?' Usually anything that prevent the function, is omitted. If the volts go down the current will go up and trip on over current, eventually. Synchronised Alternators, also look doggy to me. A bit Heath Robinson!
Phase sync on the two generators is certainly a requirement but voltage being exactly the same( is not required?)
When I was in the Navy, you could manually sync two AC power sources if the phase difference was 30° or less.
Please tell me they don't use aluminum wire on ships. (except for major high voltage lines) That combined with push-in connectors is a really bad combination. There is a reason they stopped using aluminum wire in homes.
breakers open because something is wrong
Sometimes there are faults in the fault-detecting circuitry too. AND connections under vibration suffer from physical wear that can result in faults too.
I see why, they not using screw down conection terminals which is absolutely necessary for any boat at sea due to vibration and salt corrosion .
30 years as a marine engineer, 10 of them as a chief engineer. NO ONE would ever use aluminum wire on a ship. Not even silly Naval architects with PhDs. The ship was 20 or so years old with constant vibration, heat, and salt air environments progressing something called entropy. Everything decays and goes back to nature. Given enough time (another 60 years), the bridge would have fallen on it's own. What I am getting at is that "man" cannot control everything all of the time. I would bet that the low voltage trip was because of a computer or microprocessor error.
Going after the terminal block and the wires? Sounds to me like they are trying to shoot the messenger. I would question the low voltage detector for a failure in the active components it may contain. Also, failure of this device, a loss of signal, should not result in an unsafe condition.
A failure with a component like this could easily have caused the low voltage detection system. Most likely, the investigators tested the low voltage detector and found the detector to be working correctly, but found the detector was being fed a bad signal, possibly due to this terminal block.
WHARE are Dali's Anchors. The stearboard side anchor was dropped before the collision and ws no doubt cut loose so that the Dali could be removed from Key bridge. The port anchor appears to have been removed at some point. The Dali will need these anchors to put to sea, whrer are they?
I can see them just taking that cheap and easily replaceable block with them (with the original wiring still attached) to study in the lab to evaluate its operation at their leisure rather than delay departure of the ship so they could study a three dollar part. (And maybe the coffee is better at the lab.)
You mention 3 things to sync the generators. 1. Voltage, 2. Phase, and then you again say Voltage for #3?
Undervolgae release its just a Coil (elektromagnet) inside breaker which trips(opens) breaker if the coil voltage is lover than 30-35% of nominal voltage. This coil is not responsible for monitoring bus voltage. To close and keep closed breaker this coil must be energised all time. So if there was loose wire in wago terminal which was supplying UV (undevlotage coil) then breaker will trip. Opening primary breaker will csause to trip a scondary breaker (also via UV) - UV coils are used very often in marine switchboards, simple way to ensure that power goes only from source 6600VAC to consumers 440VAC. Spring terminals are way better than screw terminals do not need of retaightenning after the time - but they have to be used properly. Most common mistake which i see is trying to insert 2 wires into single hole without proper double ferrule.