Sadly, everyone is. My sisters commission is gonna get delayed, parts cant get ordered. Also ALL info is stored in that system from customer emails and employee financials. Everyone needs to change passwords asap & banks need to be monitored.
I worked for CDK. The incompetence there is off the charts! Then again, the dealers are of criminal mentality so the irony of all of this is priceless.
@@ericoyen1704 Because like in many other industries, CDK has no real competitors. Build a competing product requires lots of money and time. Quite simply there is no incentive.
I'm a former employee as well. Can confirm it was horribly run. You could get angry messages for taking time to use the restroom. I also heard from an employee who stayed there longer than I that they laid off most of their support staff a few months to outsource it overseas.
Was it deliberate that they came up with an acronym that sounds (when spoken verbally) like "seedy-kay" ?? Because a lot of stealerships are already seedy enough, let alone with software to help them get better at it :)
@@Magusarcan Is CDK all cloud based? I wonder if this is why they are off line? Also does CDK programs take care of everything? Meaning that if a new state law happens CDK send out notices and changes to it program to comply? In the sense could someone clueless about dealerships run it very well with the use of CDK? CDK does all the leg work which is why dealers pay the cost to have CDK do everything including payroll?
I had an emissions recall done last week on a Cummins diesel and they dealership handed me a hand written invoice on carbon paper stock with a 1987 print date, the old Plymouth logo and telephone numbers with an area code that was changed back in 1991... Someone at that dealership packed those away 30 years ago with the idea "We may need these when those new fangled computers go down"... Maybe we should all start to learn Morse code and get our Ham Radio operators certification...
I'm a software engineer. This falls squarely on the shoulders of CDK. They chose to build the software that way. They should be sued for basically lumping all customer data together probably in one data center. This means that they didn't have proper disaster recovery in place. They also didn't have distinct tenants for each company. It's unreal that this level of incompetence gets the amount of money they do.
There are cybersecurity regulations when developing software in a classified or highly regulated (nuclear power) environment, this needs to be applied to the private sector.
As a retired IT Information Security Officer for large healthcare system. Funding for Technology Security is at the bottom of the list. So many times digital equipment with hard drives (X-ray, MRI, ultrasound and printer/copiers were disposed or sold full of unencrypted patient information. Patients credit cards information was stored in some financial departments in Word or Excel files. The top executives didn’t care about PCI credit card compliance.
CDK also provides dealer management software for small construction and equipment dealers. This is devastation to small businesses like ours. We’re on able to conduct business.
I was a Field Engineer for Sonic Automotive - CDK runs EVERYTHING! Contracts, Loans, Accounting, Sales, EVERYTHING runs off of that system. It is so outdated, it is like a 1980's database. You have to input menu numbers to go to certain screens - it is SO OLD. It never ceases to amaze me how these dealerships will try to save a buck - including buying outdated, inefficient software.
100% correct. Owned an IT company years ago (now retired) and had many dealers as clients. The system runs on a server that dumb terminals access. The frequent issues we saw were when printers would become "unaligned" or deletes and need to be reset/reinstalled. Slow and archaic software. Another one that dealers used, is out in the shops, is Reynolds. It's used by the mechanics to request the correct part and also gives a diagram (that one can zoom out or in) for when they'll pulling and replacing parts. The main issue here is that when it goes down, we (or any IT support), couldn't do anything because they control the server it runs on. It was a pain in the aS... Yet they wanted access to the network (our routers, switches, etc.) We told them to pound sand and installed a separate LAN just for their one server to isolate them from the dealer's network. Their access to the internet was on another VLAN. Their system, on average, would go down 1 a month and we had to contact, we billed the dealer, to get them to fix it.
I work in the service department of one of the dealerships that's affected by this. Without CDK, there's no work orders. Without work orders, technicians' flagged hours are difficult to track. If we can't track those hours properly, there's no way to tell if our paychecks will be correct. Just another thought for the people say that we've "gotten what we deserve".
I used to do bookkeeping and payroll for a dealership. We used ADP but if that went down it killed everything because parts, service, sales and accounting were all linked. This would be a nightmare. Edit I just found out that ADP is CDk. Shit.
@cmmosher8035 Yup. We are currently on the verge of having the techs riot. It might be different if my employer had some kind of plan and communicated that plan to us. Instead, we're getting silence. But that's a discussion for a different forum.
@@matthewalvarez5661 I had it go down but never more than a day so we just paid guaranteed hours but I can image what happens for an extended period of time. Btw it's been about a decade since I worked with it. Good luck.
Lol, my dealership is going to pay a round number to everyone, the deduct the difference between flagged and the fixed pay. A few guys in the shop have been sick since it started and most of the day is spent playing cards waiting for a ticker to get Witten for hours at a time. The first day we got 4 tickets in for a shop of 18 techs. 2nd day was up to 16 tickets. Fun times especially when you can't leave early
WTF, my guidance counselor in high school never told me I could be a computer hacker and make hundreds of millions of dollars working from home. Pisses me off.
I've been working in IT for more years than I care to admit, and we're quickly approaching the point of diminishing returns where the convenience of putting things online or using services hosted in the cloud isn't worth the security risks. No matter how smart you are or how much money you throw at the problem, there's always somebody out there smarter than you are, and if they want to hack into your online services, they'll find a way. Technology isn't inherently a bad thing but like most things you have to use it the right way, and increasingly that means the best option is returning to locally-hosted services/servers with little to no connectivity to the Internet.
Oh good, someone with the ability to do something about it is actually starting to figure out why it's happening. Shame that you whizzed right past the point and rushed to the conclusion of "we have to take things offline". Hint: hackers target systems that are likely to cause the greatest systemic failure. Why?
@@chernobyl169 They target a lot more than that. You only hear about the big ones. There are millions of little ones every day nobody bothers to mention because they're so common.
7:52 Yes, hackers can disable that, plus the heated seats will probably stop working permanently when the company managing the subscription service decides to end support (see Spotify's car thing), gets shutdown, or goes bankrupt. This is why people should be fighting this crap.
Seconded. Anything remotely managed can be remotely hacked. So many new questions I have to remember to ask when buying a car, to make sure I don't accidentally buy a car with any of this crap in it (I refuse to buy new, so it'll be research, and relying on the seller knowing what they have).
Subscriptions should be illegal unless the item needs periodic updating. Heated seats don't need updating. Neither does my checkbook program, yet they went to subscription even tho I don't use it online, don't ever update it, don't need or want any updates. The one time I did update due to their constant update notifications, the update messed up all of my saved reports and I had to redo them. Never updated since then.... no need to.
Remember, the US government is mandating speed limiters and remote kill switches in all new cars made after 2026. And it is in coldwar 2.0 against two big nations with lots of hackers.
Old dealership office gal here - that was the Nixdorf computer that handled the forms. You plugged the numbers in and choose other options and viola! All your forms with the print not quite in the boxes.
Sounds right from what I've heard from our IT. Hopefully the upper management shows some compassion and pays employees to make up for the loss of work. My company has said they are going to pay everyone a flat pay, then deduct how much extra you got paid vrs booked as they can't be bothered to issue correct paychecks.
@@UncleKennysPlace It's not really free if the car dealerships have lobbied so hard that it's impossible for the manufacturers to sell cars bypassing the dealerships. In fact, it's the opposite of free. The government is actually forcing you to buy from the dealerships whether you like it or not. Thankfully Tesla managed to break this monopoly.
It's because Cyber and IT are seen as different in many organizations, cyber is under IT in others but doesn't have funding until something like this happens. I was hired by the company I with for after an incident where we went from a security department that was about 10 people to a department that was almost 10x because of the incident.
@@peteranderson037 tech debt is caused by poor design and band-aid fixes and is indicative of poor solution management. Clicking on an email by a company that is hired to send out cyber security mail campaign for employee awareness then getting pwned because someone clicks a link in an email is pure irony
As someone that has worked in IT. Yes. I've seen far more secure systems working for a Bank. For a program that is dealing with financial information it should have been way better protected.
The fact that they're down for more than a few hours means either they don't have a proper disaster recovery plan or that the plan was inadequate or that they hadn't been regularly testing their DR system and it didn't work. As somebody who has spent several decades in the IT industry this honestly doesn't surprise me one bit. Most companies don't take security or disaster recovery seriously until it's far too late... and a lot of companies don't survive an outage like this. All one needs is proper security policies, regular patching, frequent backups(at least daily), frequent application consistent immutable storage based snapshots(at least hourly, possibly every 15 minutes or less), remote replication, etc... AND frequent testing of the DR system. But all of that costs money which is why these greedy bastards clearly skimped on most of it. Wonder how long until they file for bankruptcy......
Actually what they need is a Ransomware recovery plan, as a Ransomware event is different from DR in software effects, hardware effects, & decision making. While a DR event usually leaves management no option but to let IT put things back together as fast as possible (or cutover to a warm/hot site), a skilled Ransomware event usually locks you out of all of it (to include your alternate sites) - but leaving management with the enticing option to pay the ransom and have it all back within hours. A good Ransomware recovery plan is different than a DR plan and can have things back orders of magnitude faster than a DR plan if everything is really locked up.
Not necessarily. Steve mentioned something about CDK being in negotiations with the hackers which suggests a ransomware attack. If that's the case, this could drag out for some time. I was in IT back in the day. Then our databases (even our cloud databases) ran tape backups every six hours. Yes, it consumed an enormous amount of tapes, time and network resources but at least it was secure.
@@mikedelcaribe7422 That's the curse of succeeding. Because all the preparations for Y2K allowed us to transition with very little disruption, many people now believe it was unnecessary and just wasted money. They're the ones who refuse to invest money in IT security because the company has never been hacked before.
@@johnediger7820 Tape. Yup, being a tape monkey was part of my job when I started in operations back in the mid 80's. Cards, Tape, Removable Disk Packs, High Speed Chain Printers... - it was a whole other world.
Steve, I have a towing business in Hoover Alabama; none of my dealer-customers have been able to issue Purchase Orders for my towing invoices for over 2 weeks now. Obviously that is not the only problem they are having but it's going to be interesting for me @ the end of the month when I send these dealers their bills; HOPEFULLY they can still issue checks manually~!
That's the problem with single sourcing anything . Remember this lesson when you're thinking about your credit cards and your banking. Digital money can just disappear.
Digital currency is even more dangerous to the common man than fiat currency. Fiat currency is at least physical whereas digital is just 1s and 0s in the ether.
It’s affecting employees mechanics parts warehouse’s employees and other businesses and they employees since last week so it’s not just about dealership it’s about employees sitting home until they fix this mess
@@angelcabeza6464 Here's the problem though: it affects aftermarket repair facilities as well. People that have nothing to do with other people buying overpriced privacy nightmares. There are a lot of parts that should really be purchased from the manufacturer. It's more expensive up front don't get me wrong lol it's borderline extortion and I hate it too as an automotive technician; but by the time you've purchased 5 Dorman parts from NAPA and Autozone you were time and money ahead getting the dealer part.
Lucky, my dealership took a day to start handwriting tickets for us, NO you can't leave early. They have also already said we are not getting g anything to make up for no tickets, having to write everything on cabon paper, and a parts department that can't oder parts and takes almost an hour to bring you an oil filter for a 0.3 job.
My husband works in a very important, very sensitive industry and their biggest competitor just got hacked. They deal with so much personal information it’s scary that they’re not taking better care of people’s information.
Why should they? If no one provides better security, then you won't lose customers over it, and the cost of a data breach is 1 year of credit monitoring.
@@werefrogofassyria6609If you get offered credit monitoring as compensation, dot NOT accept it! These monitoring solutions will lock down your credit profile and make it near impossible to borrow money or do any business that requires a credit check, because you have to manually request an unlock at the credit bureau each time, and the credit bureau's system is poorly equipped to handle such requests. Just hire a lawyer to deal with the mess instead.
My daughter is a Service Advisor at a dealership. Its been a nightmare. CDK Global is not one of the best software programs available. But its affecting buying, selling, service, parts inventory, warranty systems and more.
CDK is who we use for our business software. We are a construction equipment dealer, all 26 dealers in our group use it. We have been deadlocked for a week, went back up an hour ago, and now its back down. Absolutely hilarious. Too bad they ditched ALL use of paper and no one is trained to do things manually 🤣😂🤣😂
The dealerships of bigger cities are often owned by a single corp. For example, one corp can own the dealership franchise for Ford _and_ a separate one for Dodge, and then Nissan and etc.... So one can go car shopping, just to find all the dealerships are disabled. A family member who is a salesperson at one of these dealerships, told me they are doing everything on paper- including the finance part as well. It is slow and tedious work, with smaller rewards. But it's something. They have never closed completely. The guys are still getting a small paycheck. It's a true test of resilience. I feel for the folks who are stressed out this month, over paying their rent or putting food on the table for their families!🙏🏼
Found out the Nissan dealership closest to me has common ownership with the Hyundia dealership next door. Tried to book an appointment for my Leaf: and the were asking about a Hyundia I had taken to the other dealership for recall work.
Usually, it is a few big ones that own most and a few smaller ones. You always get a better price at the smaller ones. They have fleet costs they can not spread to other locations.
This type of dealer computer support has been around since the 1970's. The two providers,who controlled 90+% of the industry, were ADP and Reynolds and Reynolds. They did business on their terms: sign a long term contract, and, if the task employs numbers( payroll/payables/taxes/ record keeping/ you name it. The gimmicks: 1) " We do all your work. Period. " 2) the two providers used deliberately incompatible programs...so, if you decided to switch to a different lousy provider, you had to hire teams of people. To manually enter all data to new system. With keyboards. Time estimate for task: medium size store was about 10, 000 in labor.
I'm old enough to remember when everything was done with pen and paper. When you went to a parts store they had books lined up across the counter to look up the part
@@kenmore01 The point is, our world worked once without everything "needing" a computer. Still can, if we want it to. And more and more, we want it to.
I'm old enough to remember calling my banker to let them know I was gonna buy a car and I'll stop by to sign the paperwork if I write a check for more than what's in my account.
I'm old enough too. I remember taking my son to the bank to open a checking account in his name as he was going off to college. On the way out, the banker said in a quiet tone, "Don't worry, if he overdraws we'll give you a call before we bounce anything." He never did. But those days are gone.
Yep, been there with local bank when I was arranging to buy a house at auction and move it. “I’ll have the gal write up the paperwork and you can come in and sign it Monday, but if you need to write a check now, we’ll cover it.” Handshake and done. I miss those days.
@@jeepien My dad used to play golf once a week with the manager of the bank we used. If my account was a little short I would get an earful later. Only did it ONCE.
This is one of the main vulnerabilities of SaaS and cloud-based software. Too many companies don't know how to secure their systems and all these hacking events prove it. How big a hit will the economy take if no one in America can buy a car for the next week or two?
I am a technician for a Chevrolet Buick GMC dealer in Oklahoma City, and this has crippled our business. Plus, we just had a storm blow through that took out the power for nearly a day. We could not order parts, write ROs, or sell cars for 3 days. Finally, we got a system in place to hand write everything. But it has slowed our processes to a crawl. Great video. We have had to do our own research because we aren't being told anything. Just they are working on a fix.
Because there are too many shady actors in the business. If the businesses were relying on off site storage for all the sales information entered into this system they were fools. Such important information should be stored locally as well as on remote servers for backup reasons. The bigger issue is if the program has mandatory online license checks that happen every time it's opened up with no offline recovery mode. I suspect that the worst of the hacking attacks are blocking any access to the servers for the remote check and data access.
Thing is it's possible for even the most secure company to be hacked. It's culture and their response to the hack that's the important bit. Unfortunately, that's very hard to enforce.
As an Info Tech professional, it boggles my mind that anyone is susceptible to these types of attacks. Even back in the 90's, I had backup rotations and system restoration media that would allow me to restore any and all of our systems going back as far as 5 years. I had a daily rotation for 7 days, a weekly rotation for 5 weeks, a monthly rotation for 13 months and a yearly rotation for 5 years. Basically, a rotation of 30 backups. In the best case scenario, I would lose 1 day of operations and data. In the worst case, I could "start over" from any one of 29 other points of reference, and one of them would certainly be from prior to the hack. I would lose 1 day of operation and as many days/weeks/months/years of data that it took for me to get to a point of reference from before the hack. But at least everyone would be back "up and running" on the software by the next day, even if prior data was lost.
Been there done that. The malware may have been in the system for months before it executed. The backup rotation cycle may be so old, or cycled out, so all backups might have the malware in them. This is the new way of organized criminals.
Having worked in dealerships for some time I can tell you that CDK, a spinoff from ADP, outages are not crippling to dealer business but are a major pain in the rear. Business goes back to pen and paper. The real hassle is eventually having to catch the computer up to what has occured during the outage. Parts sales, inventory and so on are all handled by the system. All retail transactions be it sales, service or parts use the system. Accounting is probably paralyzed so if a dealer owes you money prepare to wait. I don't envy the current employees.
Software as a service or SAAS applications are subject to the same problems any application is. The interesting thing about these latest attacks is they hit aggregation points and take out massive amounts of businesses. This has happened in healthcare as well.
Can you imagine? Gong into the warehouse to take out the old typewriter and fill the forms that way? Then have someone go to the DMV to file the paperwork in person instead of electronically? Back to the dark ages! 😹
About three summers ago our former provider was hit. When I came in for an appointment they informed their computers were down and needed to fill out paperwork. As the appointment went on it became clearer how bad it was. They had bits and pieces of my medical history with them. They did a routine EKG, but then told me they had nothing to compare it to as the one from a couple years back was not there. I took my mother in a couple months later for a routine appointment and while they had the system back up there was a lot of "missing information' which lead me to believe this was more than just "unauthorized access on our network" as the 'Cover Our A**es' letter they sent to me tried to downplay the events. This was more a ransomware attack. Plus, I still believe this incident was 'ground zero' for the identity theft/fraud using my late father's name (also a patient there). The timing of everything lines up. Even a couple years after the fact I am still discovering more fraud as a household Change of Address as exposed more. About two years ago they ended up merging with )or more likely being assimilated) into Dignity Health. Always seemed odd to me as they strived on being an independent provider, but I suspect this incident is what lead up to the merger.
@@LoneTiger Pain in the a**, but a computer work around is fillable PDFs with a folder for each type. That will teach those young whipper snappers a thing or two.
It's because these systems are vertical stacks of data. CDK application is just interfacing with data and providing basic CRUD and I/O. Even most modern node / kubernetes applications, which are decentralized back ends, still have the single point of failure of their database, which is centralized and vulnerable. Clearly CDK did not have proper offsite backups, third party backups or isolation processes for their data. Perhaps even their source code as well, which prevents them from firing up new infrastructure AND restoring the data. This is gross incompetency from the top down.
@@TheeGlocktopus It's not just the data backups. The sophisticated hackers clean you out entirely, AD/Network&Firewalls&LoadBalencers/Physical/Virtual Servers/Storage/DB configs. Putting all that back together so you can even begin to do actual Data Backup Recovery's is gonna be days unless you have a specific Ransomware Recovery Plan to quickly get all these pieces back running without the hackers finding this recovery data and "locking" it up.
I remember my father, an architect by profession, typing long technical letters, with his right hand index and second finger (because he had a cigarette in his left hand). On an old upright long throw manual typewriter, that must have been pre war. He said it was still quicker than sending documents back to the typists for correction five or six times, because typists often have some antipathy to technical words, and decided off their own bat, to spell and punctuate the way they thought it looked right ! I mean you've just written it down for them, so you can't possibly have meant that !
Ha!!! I'm so old I remember having to use a typewriter-like device to punch holes in cards to create decks that were fed into the computer's card reader.
well la dee freakin da with your fancy IBM selectric. when I bought my first Flinstones pushmobile, the information was recorded on a stone tablet with a hammer and chisel.
Working in the IT industry, this is pretty normal. Most small businesses don't have / want to have an IT person on staff and it is much easier to offload the server required for the server to the software vendor's "cloud". These same systems are in place in Retail and Medical as well. Epicor is a large retail software company that nearly all Ace Hardware's (independently owned) still license their software through. Epicor's cloud servers cost about the same as having a physical server on prem and replacing it every 5 years.
You know what irks me? When a company gets hacked and as part of their notification they send you they send how to not get hacked information. I started reminding them how it was THEY who got hacked, not me.
@@samholdsworth420 When you're given one plate of food and it's only got stale bread on it, that's not your fault. When consumers have no alternatives for cars, TVs, cell phones, you name it, that come with features we don't like, it's not the consumers' fault. Our free market is nowhere near being actually free because it's so heavily dominated by a few key players that what those few at the top choose become the only options for the overwhelming majority of people. I'd love a smart phone that doesn't require the use of two hands, but companies like Apple and Samsung are too focused on making smart phones bigger to have a viable option for me. I'd love pants with pockets, but the places I can afford to shop only offer pockets in the men's section (and I got too much junk in the trunk for those to fit me right). Rollbacks on anti-trust, anti-monopoly, and consumer protection laws have severely harmed us. And considering I wasn't even of voting age when all that rollback shit took place, it sure as hell ain't my fault. We weren't shot in the feet by ourselves as a whole, all-encompassing 'the public'- we were shot in the feet by anyone who voted to strip our economy of necessary protections. If that was you, then by all means, take responsibility, but don't lump me into that group.
@@samholdsworth420 Most people have accepted always-connected devices, but not everyone. For example, I'm setting up a NAS for my sister to use with her security cameras to monitor her kids. This NAS will also allow her to stop uploading data to the cloud with her phone storage. We've priced out a 32TB (24TB usable) system, with a case and board that can accommodate 8 more HDDs.
CDK is a DMS (dealer management system) which you can think of like an accounting system, inventory management, repair and parts order writer, forms printer, bank/loan applications integrations, and other automotive interfaces. DealerTrack is another huge DMS. If you need to run a dealership, the first software you get is a DMS. Yes, these are often run in the ‘cloud’, and sometimes have outages. Then you have to manually write everything and all the olds guys flex and reminisce about the old days of coke and babes.
Don't forget forget about the stick shifts you cant work while yer sucking your thumb because you entirely dependent on the cloud. This isn't just an outage this is a big glaring warning sign that central based systems are a bad idea. This system was hacked for profit . It just didnt not work because of hardware issues.
The real problem here is the dealerships don't have a disaster recovery plan when said service goes down. That's just negligent as a business to not identify single-points of failure and have a plan B worked out "just in case".
As I have said before, lately, convenience has become inconvenient for customers and sellers. This happens when you put all your eggs in one basket, unfortunately. 😢😢😊
I remember working at a community hospital in SC some forty years ago. We were in the process of taking the hospital's daily census from an arduous manual proceedure to a computerized program. The transition was fraught with problems but we finally accomplished that modernization. I remember looking our office manager in the eye and saying, "Keep a record of how to do this manually. Sooner or later, you're going to need it!
@@jamesphillips2285 My experience with manual backup processes is that even when you have a manual backup process, you dont have enough staff to run it.
The option is to have every dealer own, setup and maintain their own server and software system. And no passing along a copy to your uncle who runs the dealership down the street. See, we tried all that back in the '80s and '90s. It's just easier all around to have the software reside in a central server.
Exactly why I use Hubitat and power backups. I can run all week and then fire up the propane generator, if needed. I have two antenna for TV. I will be OK. 1 watt + 1 watt, etc. I did all the measurements and installed the backups and a generator. I would be upset and have to adjust but be functional unlike most.
This is why I dislike the concept of "Always online". Something like this sounds like it could be handled by a spreadsheet program such as excel. What I suspect happend is they had old hardware that was running an older version of windows that was compromised and they got in that way. Its scary how many institutions still rely on windows xp that are considered critical infrastructure.
It sucks. It affects service, parts and sales. I'm a vendor for dealerships and they're back to the stone ages. Really, it's worse for the service departments, because they're having to do manual repair orders, and the whole infrastructure to do them...charging parts, writing PO's, everything...just doesn't exist anymore. So they can write them, but all of the ones they're doing right now will eventually have to be entered into the system again once it's back up and running. And that will be an even bigger nightmare.
That's the Dangers of Always Online. People forget the old days. People forget how it used to be. It makes people stupid and ignorant. Any Dealership doing things by hand and not relying on this program? They pointing and laughing at your stupidity and ignorance while they continue on, gaining all the business that YOU have lost.
My father-in-law works in a dealership parts department, and he's been telling us about this the last couple days. He thinks CDK paid the hacker ransom and should be back online by early next month. He's been relating to us how all the young guys that work there have never done anything without a computer, and he's having to step in and remind them about all the pre-filled or automated information that they need to remember to collect to fill out a complete sales slip. I was jokingly suggesting, but it doesn't sound like they have any old sales slips lost in a corner, somewhere, that they could use and say "fill in all the fields". He used to be the parts manager, but due to internal politics (ageism), he's now just a frontline parts guy. But it would have been on him if there just "happened" to be a back stock of paper slips that weren't thrown away. Seems like, after this, they're gonna get a pack of paper slips to leave in a closet, "just in case".
After working a few days, and multiple hours, I don't think most people can. Every engineer I talk to says they couldn't quickly make change in their head after a day of work.
You're that guy that like to produce a penny after the sale is rung up so when the cashier stumbles you can say: "Hey everybody, I'm smarter than the checkout boy!"
Something most people aren't aware of is the impact this is having on 3rd party repair shops. Need an oil change? Sure, no problems. Need an oil filter housing from $manufacturer? Sorry, can't order it, can't get it, can't do your oil change. These repair places can perform the basic repairs, but if you're replacing your tire and it needs a new pressure sensor from the manufacturer? They can't get it. If it's a generic part? Ok, no problem. Manufacturer-specific part? Nope. Not happening. All I can tell you is that if you haven't kept your vehicle maintained and it breaks down right now, you're going to have a *really* bad time. The shop one of my kids works at has cars stacked up because they can't get manufacturer parts! It's killing the work the mechanics can do and their paychecks along with it.
Ben's on the truck in front of Steve's "The Boss" plate. This is why you do EVERYTHING by hand, then scan it into the computer in case you lose the paper copy through either a natural disaster or your own negligence. NEVER rely on tech to handle ANY tasks involving math.
I don't have firsthand knowlege, so I'm speculating. Yes, the dealers almost certainly pay a subscription to this outfit - but it's more like Facebook than Adobe. The vendor provides a "cloud" application. All of your data is stored on the vendor's servers. You might have an application on your computer to access it, or it might just be a website. But the underlying problem is that all of the data and all of the backend functionality runs on the vendor's servers somewhere only reachable over the internet. Without the vendor's services being available on the internet, you can't access or do anything. There are no issues with competitors using the same system because the vendor's clients are all (supposed to be) siloed. It's roughly the same as you and your opposing counsel shopping at the same Walmart.
25 years ago I was head of the billing department of a fair-sized medical office. My boss was about to sign a contract for a new computer system in which all of our patient and billing data would reside on the vendor's servers. I said no way, I have to have our data physically stored on OUR in-office server. The sales rep said he could dump a read-only backup copy to me every day (but exactly how we would access such a thing and/or use it for recovery was not entirely clear). The final straw happened when my staff and I were actually in training at the vendor's office. The youngster doing the training answered one of our questions by logging into another client's real data, right there on the screen for us to see (and anyone walking by on the sidewalk looking in the window). Talk about a HIPAA violation.... I ended the training session right then and there. My boss was obviously horrified when I reported what had happened, and he was able to get us out of whatever contracts he had already signed. (And I hope that trainer got fired.)
@@CrankyBeach I worked for a very large (#2 or #3 in the US?) healthcare IT provider, so I understand what you mean. I doubt it was my company (I hope it wasn’t) but regardless I sincerely appreciate that you stood up and said no to the shenanigans of that outfit to protect patients who care about their privacy and believe that HIPAA matters. If more IT people had your attitude - and more middle managers listened to us, we’d have way fewer “oh, another day another batch of stolen PII and PHI”
OMG, can you imagine the hundreds of thousands of notice letters for data breach, identity theft, personal financial and credit data, etc!! Extreme liabilities!!
As a flat rate employee in a ford dealership in MA that is affected by the CDK hack i can say that we are servicing customers, vehicles, using hand written repair orders we are unable to give customers exact price quote for what their oil change brake job windshield wiper changes alignments whatever is going to cost them. We can’t cash out the RO because the computer is down and as a flat rate mechanic it means I’m paid by the billable hours that I do in a week therefore they can’t track my billable hours. They have no idea what to pay me for the affected weeks fortunately I work in a family owned dealership and they are paying an average of what I had done in the last six weeks to include billable hour bonuses for hitting certain milestones commissions based on up sells and other performance bonuses.
I am in the middle of this mess.....my car is sitting at a dealership for the 3rd week because the dealership cannot order the repair parts needed. I had to pay for a rental car for several days, then my dealer has provided a loaner, BUT my insurance coverage is only good for 15 days on a loaner. I don't blame my dealer (which is outstanding and I have bought many cars from them). I am not sure where I will be in the 8 days I have left. We have become way too dependent on computers, and this was just the next in a long line of proof that we are getting screwed over by bad actors.
You took it to a dealer for a repair!! What the F!! Never use them!!! Unless you want to get scammed. Rule 1: never use the dealer if you can avoid it.
@@toriless I had no choice, this repair was to replace the Steering Box, the Rack and Pinon on the vehicle, and a complete realignment. The vehicle was not under warranty, BUT this is a dealer I have been going to for more than 25 years, and purchased nearly 10 cars from, and they have always treated me properly, and this disruption was not the dealer's fault, and they did provide the loaner, so I do not regret using the dealer. for this
Sadly, no. The way automotive laws are structured in America, dealerships are the only 'proper' way to sell massive amounts of cars. There's even some baked-in provisions regarding dealerships having 'territories' or whatnot so competitors can't set up shop too close to each other without running into legal issues. We're stuck with dealerships and automotive laws structured as they are for the foreseeable future.
Tech is here to stay. However relying on a single point of failure software application is foolish. Every IT course teaches you about redundancy and in this case there clearly wasn't any.
On Wednesday, June 19th, I had a neighbor with a leaking fuel pump on his Mercedes Benz. The plastic cap of the assembly had failed with multiple stress fractures, causing a leak - apparently a common issue with this model of Benz. After researching the part for replacement, I discovered 2 different fuel pumps available for this side of the fuel tank for this model - I called the stealership to confirm which was which (maybe one part number was superseded by the other, for example). The guy at the Benz parts counter said the whole system was down nationwide. A friend in Europe also was not able to log into his Benz portal and check for me. I figured it out - one part is for gas only, one part number is for E85/gas cars. I ordered the part from Benz the next day, who had to manually search their stock of fuel pumps to see if they had my part number, and then had to write me a paper receipt for it, lol. The Europe friend told me it was a system down for maintenance but I suggested it sounded an awful lot like a ransomware attack, esp since it has lasted for so long (3 days by that point). Then the next day this news came out. Part replaced though, no more fuel leak, neighbor is very happy.
Hey Steve. Love the content of knowledge that you provide! I have had a issue with my local Mini Cooper dealership in Orlando, Florida due to the Hacking of their software. I had to go back twice to make a purchase for a part to my R56 JCW. At first they couldn't make the sale of the part. But on the second trip they ended up writing a sales receipt for the sale. What a headache!
I get flack from our younger employees for not running the newest software. My servers are 12-20 years old, desktops are 0-12 years old, but everything is fast and reliable. About 1-2 hrs of downtime total per year across all systems. This is why I stay with the tried and true.
Dinosaur tech for the win, I suppose. I use an old phone (original OnePlus) and a patch cable to play music in my semi. I never have to fix the Bluetooth connection while driving. Sometimes I forget to charge the phone though.
@@SkylineFTW97 yep. Still have my first car too.. A manual 85 Prelude. Now have an 86 Si and my daughter loved it so now she has an 01. She is still working on the shifting skills though. And I have a bike but I have to admit bikes being so flipped around broke my brain, so I just have a little PCX 160. Also.. I was a Honda tech for 15 years. Funny the parallels...
Any business who's model includes, depending on another business to be able to operate, is not the best model. They should always have a backup, even if it is pen and paper. I remember a long time ago when your credit card didn't work, they would paper slip it and process it later when the system is working.
These programs are expensive, and not always practical unless you're in a large auto group where having multiple programs makes sense. For example, Mazda (from my memory) only uses RouteOne. DealerTrack and CDK are a bit more widely accepted.
Mechanic here, CDK is used throughout the entire dealership even the mechanic need it for administrative duties necessary for warranty repairs. There are other OS but CDK is the most common. This is just a glimpse into what is come. Im considering changing careers into cybersecurity!
I worked in the motor vehicle industry for 30 years. I retired a couple years ago after serving as Information Technology Administrator for one of the largest companies in our country. We changed our antiquated system over to CDK Global in 2019. CDK Global applications managed our New Vehicle Sales, Used Vehicle Sales, Aftersales (Service and Parts) and Accounting. However, we also maintained a manual system as an emergency backup! At least we did, up until I retired.
When our network went down at the auction we would go outside and write up the cars vins on the tickets that we usually just printed out.. vin, year, color, make n model, transmission type, mileage. It can be done & it's time consuming but you have to be prepared
There was a power failure at a local drug store. The cashiers were all stuck but then the old pharmacist (who probably also owned the store) came out and had everyone checked out manually in a few minutes. He said "This i how we used to do it!".
Welcome to the cloud. An old, circa 1985, large fold out ad in a popular computer magazine by AT&T chided IBM for "owning your data". AT&T at the time was marketing Unix systems on which you kept your data on your equipment. The ad feature a ransom note on an IBM blue background with letters torn out of many different paper publications. The ransom note read "We have your data and it going to cost you to get it back"; seems that we have come full circle.
Worked at a Ford Dealer back in the day and our software was from Reynolds and Reynolds and even back in the ninety's it printed all our forms from sales, service, parts and most importantly payroll. Every close of business we had to put everything from that day on these tape drives that looked like oversize eight tracks, It took about twenty minutes to do a back-up. Hopefully there is an air gapped back-up for these affected dealerships.
*Amazing video, you work for 40yrs to have $1M in your retirement, meanwhile some people are putting just $10K into trading from just few months ago and now they are multimillionaires*
Hello , I am very interested. As you know, there are tons of investments out there and without solid knowledge, I can't decide what is best. Can you explain further how you invest and earn?
I'm favoured, $90K every week! I can now give back to the locals in my community and also support God's work and the church. God bless America,, all thanks to Ms Louis Stella 😊🎉
Good day all👍🏻 from Australia 🇦🇺. I have read a lot of posts that people are very happy with the financial guidance she is giving them ! What way can I get to her exactly ?
From an IT perspective (in another industry), what I've seen happen often is where companies using one of the leading software platforms and the vendor discontinues the in-house licensed platform. To keep up with regulations and updating accounting forms, the vendor pushes clients into a online cloud platform; thereby, forcing monthly subscription fee based pricing. The client companies then downsize their in-house IT, accounting and logistics staff and switch to the vendor's newest all powerful super duper cloud platform and the savings from downsized staff go to the significantly higher costs of the monthly sub pricing model. The executives justify it by saying the cloud vendor is now responsible and it's not the client company's problem, but the clients are ultimately still responsible for business continuity or non-continuity (i.e. and take all the risk while function is out of their control/hands). If your cloud vendor didn't have at least multiple immutable backup strategies, a simple compromise of the cloud would yield a near total loss.
I’ve been around long enuf to remember that automation was going save the trees and time. 30 years or more later its done neither and in fact created all new problems.
Yeah, the joke at work is that we've actually _increased_ our paper usage since going "paperless", rather than decreased it, because now, instead of doing all your edits in pencil on a single draft, we print off multiple copies each time we make a revision so we can hand it around for review, again. (COVID actually did us a solid in that department, because it forced us into a much more digital footprint when nobody was sitting next to anybody to hand a paper copy to, anymore).
You believed that? People said the same thing with the invention of steam-powered machinery. So long as there's a greedy capitalist in charge, any innovation that saves time, money, effort, etc., will be used to benefit the fat cats. All the little guys? 'Sucks to suck, pull yourself up by your bootstraps, just be rich,' yada yada...
From the comments, it sounds like it's a Duopoly: one major competitor. It's possible there's a third option, but I highly doubt there's any more than that.
Use for their financials. They require you to sign expensive long contracts for each. They also are the clock in/out for paychecks for many dealers. So in a way for people that dislike the subscriptions services and dealer and documentation fees, it gets paid forward to dealers that choose these systems which are nearly all of them
When I did dealer finance back in the nineties, we used EDS and Reynolds and Reynolds. One or the other would shut down every now and then. Didn't seem to be a huge deal. I would use a handheld lease/finance calculator and hand print the contracts. As my GM would say, "the show must go on".
It has affected the whole chain top to bottom, left to right for the entire automotive world, including trucks, construction equipment, mining equipment and more. Independent mechanics, dealer mechanics, suppliers, distributors, truckers, tow trucks, long-haul truckers, industrial equipment, parts suppliers, taxi/uber/doordash drivers, buses, anyone and everyone who might rely on any sort of vehicle for transportation or work. Please don't direct your violin from shady dealers to the entirety of the industry.
This is one of the reasons I don't want to have any "smart" devices / appliances. If it can connect to the internet, it can be hacked and compromised / disabled.
You gotta ask the same question every day: "Why is the ONLY way to do business on the internet?" I have a 3d printer that DEMANDS I print using cloud-services...if I refuse, there is a lengthy process to manually transfer files one by one. Can't connect using USB, even though the printer has three ports on the front panel! "Not communications ports, only for use with memory-sticks or web-cam." It's a shame...damn good printer comes equipped with messed up interface.
A couple of years ago I was at a friend's house. We were in the kitchen. My phone was in my purse in another room. The next day I got a ton of ads on social media for a kitchen gadget the friend had shown me. Creepy. "Of course no one is spying on you. You're just paranoid," said the microwave oven.
@@CrankyBeach Yep. 2 months ago I was looking into getting a new credit card, and talked to my mom over the phone about it. Less than a week later I get 3 different letters in quick succession from Capital One, Citi, and Wells Fargo with "pre-approval" notices. God that made my skin crawl.
Exactly. I don't need a smart fridge, i need a fridge that keeps food cold. I don't need a smart dishwasher, i just wand a machine to wash my dishes. Same with laundry devices, microwaves, stoves, ovens, air conditioning systems, etc. Now, a slightly intelligent thermostat (that can automatically switch between heating and cooling), that's fine, or maybe even have a schedule i can program/profile in for a week.. But it does NOT need to be on the internet. And i don't need my data on a cloud somewhere. I want it on media i control, and no where else.
Last week Thursday I had an 8 am appointment at my local KIA dealer. I got there and it was surreal since no one seemed to be around. Luckily one tech let me into the garage and asked "How may I help you". I told him I had an appointment, and then he mentioned to me that they had been hacked. I was pissed that I was not called at least to say do not come in. But at the least, the tech had a pad and a pencil and took my name and phone number down. Then I realized that not only could they not do the work because their computers were down and could not find the parts, but they also lost all their customer data. How can a company not have a local backup to at least know vital information about their customers? Anyway, I hope to return tomorrow to have the work done, by them looking up my VIN number to find the needed parts for the brake job, since they still have not got their system back up and running. Good work CDK👍, and I hope a few of those 15,000 dealers wake up to how stupid their whole system is.
I used to be in the Dealer Management System (DMS) and a competitor of ADP which through merger became CDK. Our system put the server at the dealership which has its pluses and minuses. But communication with Manufacturer always go thrtough the DMS to the OEM. Therefore the systems are always at some sort of risk. CDK's system to my knowledge is similar to your description and a subscription is a good word for it. I believe that CDK nolonger puts servers in dealerships, so when their data center gets infected, it can affect many customers at once.
In the old days of computers, they were big, so multiuser setups came into being by having dozens of terminals scattered around the office building, cabled back (remember RS232) to that computer. Then the personal computer happened, and people could have their own detached computers, that didn't depend on a vulnerable centralized location. Now the internet age, and that beget "the cloud", that looks remarkably like the old "timeshare" mainframes of the 1970s ! Let's reinvent the failures of yesterday !
@@jean-marcknight8816what aspect of the thesis are you questioning? The thesis that computing was centralized, then somewhat decentralized, then back to more centralized computing is a valid statement. "cloud" just means sharing someone else's computer. That's conceptually similar to the TimeSharing Option on mainframes that has been around many decades.
@@handleit2000 We're questioning when exactly corporate office computers were decentralized, because that never truly happened. The "clouds" were always there for large interconnected data structures.
@@MK_ULTRA420true, the big iron and centralized computing never went away. There was a move towards decentralizing, then a move back to centralization, at least from the aspect of what articles popular computing news media published. 😂 😂😂
@@handleit2000 I question everything, It is just a word salad. I don’t know how you can find that "conceptually" mainframes and cloud have anything in common.
The primary reason for the central system design is that dealerships want to avoid paying for the computer hardware and IT staff to run the software locally. Dealerships want to outsource these costs to the vendor as the vendor can do it cheaper. Plus, performing software upgrades on a complex system in a local platform is painful. Most banks and mortgage companies follow the same model, where the software runs in a central location.
Redundancy is expensive. Car dealerships won't buy *pens* for their salesweasels. You think they're going to want to hear about backups? They'll never get hac... oh wait.
@@zburnham I suspect the bad guys got malware on the server(s) which encrypted the customer database(s) using a very large random key. A ransomware attack. They can pay up or rebuild the systems from scratch and use backups of the database files. If they have backups of them......
As a retired automotive and financial industry IT specialist, I saw several ransomware attacks on similar systems. The outages caused were measured in hours. We’ve also seem similar outages of government and business system reported almost daily. Those systems were segregated, fire walled and backed up. Somebody failed here, severely impacting an entire industry.
So glad my company does daily back-ups. Got hit with ransomware a few years ago. IT team laughed at the hackers and loaded a back-up. One of our competitors got hit and they were down for almost 2 months. I agree someone shit the bed on this one.
@@God__Emperor_that youtube handle lol. Backups only work if you fix how the hackers got in though. Load everything back for them to just use the same exploit isn't really a solution Yes/No?
@@God__Emperor_ These criminal organizations are getting more clever and patient. They may have gotten in months ago, waited for all of the backups to cycle through before they executed the attack. Thus, the backups don't help.
I did IT for a Chevy dealer during covid when their IT guy died. They used a similar system. Totally SaaS (software as a service) for sales, parts and service. No local server to maintain. Just a relatively simple network with gigabit fiber and really squirrelly printer configurations.
I used to support this DMS before ADP spun it off to CDK. Back then, the software only ran on a server at ADP. Dealerships would actually Remote Desktop into the server to access the DMS as if they were local. They could only see their dealership, but they were all logged into the same system. There were lots of bugs, so ADP didn’t want to deploy it to clients as a standalone software license. It wasn’t very secure back then. Sounds like it still isn’t
Better start growing your own food and dig your own well. Its easy to hate on tech but we litterally could not sustain the number of humans that currently live on earth without tech.
If you kept your entire budget on a single sheet of paper, and that paper blew into the fireplace, would you blame your dependence on paper or would you blame the lack of redundancy? It's not about reliance on technology. It's about reliance on a single point of failure.
Oh no, the forcefully required middlemen can’t do business marking up cars, what a shame. Would be a good excuse to just start doing direct manufacturer to consumer sales.
And -ADITIONALLY: Just a few years before the infamous potato famine, Ireland was the site of a massively successful temperance campaign led by the noted Catholic priest, Theobald Mathew. So successful was this campaign that between 1838 and 1841, their national alcohol consumption was cut in half.
"... Within Ireland itself there were substantial resources of food that, had the political will existed, could have been diverted, even as a short-term measure, to feed the starving people. The policy of closing ports during periods of shortages in order to keep home-grown food for domestic consumption had on earlier occasions proved to be effective in staving off famine within Ireland. During the subsistence crisis of 1782-84, an embargo was placed on the export of foodstuffs from the country. The outcome of this humanitarian and imaginative policy was successful. The years 1782-84 are barely remembered as years of distress. By refusing to allow a similar policy to be adopted in 1846-47, the British government ensured that Black ’47 was indelibly associated with suffering, famine, mortality, emigration, and to some, misrule." "Almost 4,000 vessels carried food from Ireland to the ports of Bristol, Glasgow, Liverpool and London during 1847, when 400,000 Irish men, women and children died of starvation and related diseases. The food was shipped under military guard from the most famine-stricken parts of Ireland; Ballina, Ballyshannon, Bantry, Dingle, Killala, Kilrush, Limerick, Sligo, Tralee and Westport. A wide variety of commodities left Ireland during 1847, including peas, beans, onions, rabbits, salmon, oysters, herring, lard, honey, tongues, animal skins, rags, shoes, soap, glue and seed. The most shocking export figures concern butter. Butter was shipped in firkins, each one holding 9 gallons. In the first nine months of 1847, 56,557 firkins were exported from Ireland to Bristol, and 34,852 firkins were shipped to Liverpool. That works out to be 822,681 gallons of butter exported to England from Ireland during nine months of the worst year of the Famine.” - Learn About the Great Hunger, Ireland’s Great Hunger Museum
I work at a dealership affected, this is a "all inclusive" system: sales, service, parts, book keeping and Finance. employees aren't sure about pay, retail buyers are personal info compromised.
Steve... remember, anything "cloud" based is just using someone else's computer remotely.. I'm sure the dealers were sold this on the promise that they don't need an IT department to maintain software and the service provider will provide "everything" they need . I always advise that any cloud service should be able to provide you (or a way to obtain) a backup of your data on a regular basis for you to store and use, should the cloud service cease to exist, or there are disputes, etc.
This happens when companies like CDK are allowed to get large enough to squash competition and become sole source providers. Many typewriters, pens and sales invoice forms come out of storage closets just like the good old days.
Screw car dealerships… I bought a new Nissan a number of years ago and was promised Lifetime oil changes complete BS. They tell me they only do synthetic oil changes and my oil free oil changes is now void.
They can’t discontinue regular oil changes if they promised to provide regular oil changes. Small claims court? Of course I wouldn’t want to go through this, but maybe one of the scores of customers they arbitrarily cut off will.
It's amazing how bad things can get when you put all your eggs in one basket. The reason why this company didn't make it known how deeply entwined in all aspects of the dealership business they were is because then people would be upset that they were a MONOPOLY
Most commercial software, from small to large process remotely. It’s good for wide distribution of updates, security (backup) and subscription control. Obviously, when hacks or crashes occur, this model causes a lot of pain.
So a group of criminals went after another group of criminals.
And not a single tear was shed.
People needing to buy or sell a car RIGHT now and can't probably are along with a few curse words.
@@arthurmarshall6825 if only Craigslist existed
yeah, and us mechanics are getting screwed too.
@@Jodah175 please keep telling society these things. We have had this line of reasoning all the way up to major political frameworks.
Sadly, everyone is. My sisters commission is gonna get delayed, parts cant get ordered. Also ALL info is stored in that system from customer emails and employee financials. Everyone needs to change passwords asap & banks need to be monitored.
I worked for CDK. The incompetence there is off the charts! Then again, the dealers are of criminal mentality so the irony of all of this is priceless.
Yikes! I would just simply call that karma. If the management at that company is as bad as you claim, why are they still even in business?
@@ericoyen1704 Because like in many other industries, CDK has no real competitors. Build a competing product requires lots of money and time. Quite simply there is no incentive.
I'm a former employee as well. Can confirm it was horribly run. You could get angry messages for taking time to use the restroom. I also heard from an employee who stayed there longer than I that they laid off most of their support staff a few months to outsource it overseas.
Was it deliberate that they came up with an acronym that sounds (when spoken verbally) like "seedy-kay" ?? Because a lot of stealerships are already seedy enough, let alone with software to help them get better at it :)
@@Magusarcan Is CDK all cloud based? I wonder if this is why they are off line? Also does CDK programs take care of everything? Meaning that if a new state law happens CDK send out notices and changes to it program to comply? In the sense could someone clueless about dealerships run it very well with the use of CDK? CDK does all the leg work which is why dealers pay the cost to have CDK do everything including payroll?
I had an emissions recall done last week on a Cummins diesel and they dealership handed me a hand written invoice on carbon paper stock with a 1987 print date, the old Plymouth logo and telephone numbers with an area code that was changed back in 1991... Someone at that dealership packed those away 30 years ago with the idea "We may need these when those new fangled computers go down"... Maybe we should all start to learn Morse code and get our Ham Radio operators certification...
They actually removed the Morse code requirement for ham radio operators certificates, you don't need to know it anymore.
Steve *is* wearing an ARRL Field Day shirt! 🙂
I don't have any certifications, but I own three CBs...
@@shawnerz98 I was wondering if anyone would read the entire comment and make the connection! ✌
@@shawnerz98 i heard his fingers are on fire when he taps the paddles.
I'm a software engineer. This falls squarely on the shoulders of CDK. They chose to build the software that way. They should be sued for basically lumping all customer data together probably in one data center. This means that they didn't have proper disaster recovery in place. They also didn't have distinct tenants for each company. It's unreal that this level of incompetence gets the amount of money they do.
There are cybersecurity regulations when developing software in a classified or highly regulated (nuclear power) environment, this needs to be applied to the private sector.
@@kathleenkrug-byle1199 absolutely
It's also clear that they didn't apply some patch that's been around for months most likely, and got ransomware'd
As a retired IT Information Security Officer for large healthcare system. Funding for Technology Security is at the bottom of the list. So many times digital equipment with hard drives (X-ray, MRI, ultrasound and printer/copiers were disposed or sold full of unencrypted patient information. Patients credit cards information was stored in some financial departments in Word or Excel files. The top executives didn’t care about PCI credit card compliance.
Saving this
So the dealerships are getting scammed. The irony
and guess who is going to end up paying for it?
CDK also provides dealer management software for small construction and equipment dealers. This is devastation to small businesses like ours. We’re on able to conduct business.
@@mechanoid5739 Sue the asses off of CDK for not having a proper isolated backup system.
@@mechanoid5739
@@mechanoid5739not me, never bought from a stealership probably never will
I was a Field Engineer for Sonic Automotive - CDK runs EVERYTHING! Contracts, Loans, Accounting, Sales, EVERYTHING runs off of that system. It is so outdated, it is like a 1980's database. You have to input menu numbers to go to certain screens - it is SO OLD. It never ceases to amaze me how these dealerships will try to save a buck - including buying outdated, inefficient software.
Not for much longer, if they are that vulnerable, and that slow to get back up from their faceplant into the mud.
100% correct. Owned an IT company years ago (now retired) and had many dealers as clients. The system runs on a server that dumb terminals access. The frequent issues we saw were when printers would become "unaligned" or deletes and need to be reset/reinstalled. Slow and archaic software.
Another one that dealers used, is out in the shops, is Reynolds. It's used by the mechanics to request the correct part and also gives a diagram (that one can zoom out or in) for when they'll pulling and replacing parts. The main issue here is that when it goes down, we (or any IT support), couldn't do anything because they control the server it runs on. It was a pain in the aS... Yet they wanted access to the network (our routers, switches, etc.) We told them to pound sand and installed a separate LAN just for their one server to isolate them from the dealer's network. Their access to the internet was on another VLAN. Their system, on average, would go down 1 a month and we had to contact, we billed the dealer, to get them to fix it.
Sears was like that as late as 2010. Black screens with yellow characters, 15 selection deep options LOL.
So the problem is nobody has the DOS 5.0 disks and NetWare disks for their server install? 😮😮😮
@@winstonsmiths2449 I did a bit of temp work at a Sears regional office circa 1990. Their systems were decades behind the times even then.
I work in the service department of one of the dealerships that's affected by this. Without CDK, there's no work orders. Without work orders, technicians' flagged hours are difficult to track. If we can't track those hours properly, there's no way to tell if our paychecks will be correct. Just another thought for the people say that we've "gotten what we deserve".
I used to do bookkeeping and payroll for a dealership. We used ADP but if that went down it killed everything because parts, service, sales and accounting were all linked.
This would be a nightmare.
Edit I just found out that ADP is CDk. Shit.
@cmmosher8035 Yup. We are currently on the verge of having the techs riot. It might be different if my employer had some kind of plan and communicated that plan to us. Instead, we're getting silence. But that's a discussion for a different forum.
@@matthewalvarez5661 I had it go down but never more than a day so we just paid guaranteed hours but I can image what happens for an extended period of time. Btw it's been about a decade since I worked with it. Good luck.
Lol, my dealership is going to pay a round number to everyone, the deduct the difference between flagged and the fixed pay. A few guys in the shop have been sick since it started and most of the day is spent playing cards waiting for a ticker to get Witten for hours at a time. The first day we got 4 tickets in for a shop of 18 techs. 2nd day was up to 16 tickets. Fun times especially when you can't leave early
WTF, my guidance counselor in high school never told me I could be a computer hacker and make hundreds of millions of dollars working from home. Pisses me off.
I've been working in IT for more years than I care to admit, and we're quickly approaching the point of diminishing returns where the convenience of putting things online or using services hosted in the cloud isn't worth the security risks. No matter how smart you are or how much money you throw at the problem, there's always somebody out there smarter than you are, and if they want to hack into your online services, they'll find a way.
Technology isn't inherently a bad thing but like most things you have to use it the right way, and increasingly that means the best option is returning to locally-hosted services/servers with little to no connectivity to the Internet.
LOL ... HEY always make ... backups! smFh
@@mos8541 Some do have backups, which the hackers delete as part of their attack
Oh good, someone with the ability to do something about it is actually starting to figure out why it's happening.
Shame that you whizzed right past the point and rushed to the conclusion of "we have to take things offline".
Hint: hackers target systems that are likely to cause the greatest systemic failure. Why?
@@mos8541 Good luck backing up your bank account and the deed to your home. 😂
@@chernobyl169 They target a lot more than that. You only hear about the big ones. There are millions of little ones every day nobody bothers to mention because they're so common.
7:52 Yes, hackers can disable that, plus the heated seats will probably stop working permanently when the company managing the subscription service decides to end support (see Spotify's car thing), gets shutdown, or goes bankrupt. This is why people should be fighting this crap.
Seconded. Anything remotely managed can be remotely hacked. So many new questions I have to remember to ask when buying a car, to make sure I don't accidentally buy a car with any of this crap in it (I refuse to buy new, so it'll be research, and relying on the seller knowing what they have).
Subscriptions should be illegal unless the item needs periodic updating. Heated seats don't need updating. Neither does my checkbook program, yet they went to subscription even tho I don't use it online, don't ever update it, don't need or want any updates. The one time I did update due to their constant update notifications, the update messed up all of my saved reports and I had to redo them. Never updated since then.... no need to.
That's why I'm only gonna buy classics, not new cars.
Remember, the US government is mandating speed limiters and remote kill switches in all new cars made after 2026. And it is in coldwar 2.0 against two big nations with lots of hackers.
I'm gonna need to find someone to jailbreak my new car so the features behind the paywall can be unlocked for free.
😆
Old dealership office gal here - that was the Nixdorf computer that handled the forms. You plugged the numbers in and choose other options and viola! All your forms with the print not quite in the boxes.
You would be broke as a realtor if you did that crap.
@@torilessyeah so you understand the manipulation then how is this all a believable narrative to you overall?
My son works at a dealership, he's been told they don't expect the CDK system to be back up until mid-July.
Sounds right from what I've heard from our IT. Hopefully the upper management shows some compassion and pays employees to make up for the loss of work. My company has said they are going to pay everyone a flat pay, then deduct how much extra you got paid vrs booked as they can't be bothered to issue correct paychecks.
Ha!
It’s cute you think they’ll actually do the morally correct and ethical thing.
The hacker group should really demand dealers stop charging a "dealer markup" or "market adjustment".
Sadly people with wealth or ignorance would still buy those vehicles.
Go get a degree, learn how to do it yourself and buy your own parts. It really is simple…
Dealer markdown
@@ronhall3686 It's the _free market._ The last thing you want is the government controlling the market.
@@UncleKennysPlace It's not really free if the car dealerships have lobbied so hard that it's impossible for the manufacturers to sell cars bypassing the dealerships. In fact, it's the opposite of free. The government is actually forcing you to buy from the dealerships whether you like it or not. Thankfully Tesla managed to break this monopoly.
The big businesses saved a few bucks, now theyre really paying for it. Low cost IT is basically no IT
The Cloud is totally totally secure. It's web scale.
It's because Cyber and IT are seen as different in many organizations, cyber is under IT in others but doesn't have funding until something like this happens. I was hired by the company I with for after an incident where we went from a security department that was about 10 people to a department that was almost 10x because of the incident.
Technology debt is a thing, but don't tell that to a board of directors.
@@peteranderson037 tech debt is caused by poor design and band-aid fixes and is indicative of poor solution management. Clicking on an email by a company that is hired to send out cyber security mail campaign for employee awareness then getting pwned because someone clicks a link in an email is pure irony
As someone that has worked in IT. Yes. I've seen far more secure systems working for a Bank. For a program that is dealing with financial information it should have been way better protected.
The fact that they're down for more than a few hours means either they don't have a proper disaster recovery plan or that the plan was inadequate or that they hadn't been regularly testing their DR system and it didn't work.
As somebody who has spent several decades in the IT industry this honestly doesn't surprise me one bit. Most companies don't take security or disaster recovery seriously until it's far too late... and a lot of companies don't survive an outage like this.
All one needs is proper security policies, regular patching, frequent backups(at least daily), frequent application consistent immutable storage based snapshots(at least hourly, possibly every 15 minutes or less), remote replication, etc... AND frequent testing of the DR system. But all of that costs money which is why these greedy bastards clearly skimped on most of it.
Wonder how long until they file for bankruptcy......
Didn't we learn about business continuity planning with the Y2K drills? Apparently not.
Actually what they need is a Ransomware recovery plan, as a Ransomware event is different from DR in software effects, hardware effects, & decision making. While a DR event usually leaves management no option but to let IT put things back together as fast as possible (or cutover to a warm/hot site), a skilled Ransomware event usually locks you out of all of it (to include your alternate sites) - but leaving management with the enticing option to pay the ransom and have it all back within hours.
A good Ransomware recovery plan is different than a DR plan and can have things back orders of magnitude faster than a DR plan if everything is really locked up.
Not necessarily. Steve mentioned something about CDK being in negotiations with the hackers which suggests a ransomware attack. If that's the case, this could drag out for some time. I was in IT back in the day. Then our databases (even our cloud databases) ran tape backups every six hours. Yes, it consumed an enormous amount of tapes, time and network resources but at least it was secure.
@@mikedelcaribe7422 That's the curse of succeeding. Because all the preparations for Y2K allowed us to transition with very little disruption, many people now believe it was unnecessary and just wasted money. They're the ones who refuse to invest money in IT security because the company has never been hacked before.
@@johnediger7820 Tape. Yup, being a tape monkey was part of my job when I started in operations back in the mid 80's. Cards, Tape, Removable Disk Packs, High Speed Chain Printers... - it was a whole other world.
Steve, I have a towing business in Hoover Alabama; none of my dealer-customers have been able to issue Purchase Orders for my towing invoices for over 2 weeks now. Obviously that is not the only problem they are having but it's going to be interesting for me @ the end of the month when I send these dealers their bills; HOPEFULLY they can still issue checks manually~!
That's the problem with single sourcing anything .
Remember this lesson when you're thinking about your credit cards and your banking. Digital money can just disappear.
Digital currency is even more dangerous to the common man than fiat currency. Fiat currency is at least physical whereas digital is just 1s and 0s in the ether.
@@KittyDad All currency is now "1s and 0s."
@@joesterling4299 Not yet
@@joesterling4299wait, I just checked my wallet & saw some presidents printed on linen & cotton... I guess there were 1s & 0s on some of it though. 😂
cash can just disappear with no recourse to recoup it. With digital transactions you have an option to dispute and get it back.
I suspect absolutely nobody that is not connected to a dealer is more than ok with them being hit after all of the BS reported the last few years.
It’s affecting employees mechanics parts warehouse’s employees and other businesses and they employees since last week so it’s not just about dealership it’s about employees sitting home until they fix this mess
@@Steel387 oh well dont scam people and call it a job simple
@@angelcabeza6464 Here's the problem though: it affects aftermarket repair facilities as well. People that have nothing to do with other people buying overpriced privacy nightmares. There are a lot of parts that should really be purchased from the manufacturer. It's more expensive up front don't get me wrong lol it's borderline extortion and I hate it too as an automotive technician; but by the time you've purchased 5 Dorman parts from NAPA and Autozone you were time and money ahead getting the dealer part.
It's affecting service, too, so there are people who are very concerned about it. My friend's car is being held hostage.
@@angelcabeza6464not all mechanics are scammers my dude.
They told us they were going to pay us based on our average for the last few weeks... I smell two lawsuits
Lucky, my dealership took a day to start handwriting tickets for us, NO you can't leave early. They have also already said we are not getting g anything to make up for no tickets, having to write everything on cabon paper, and a parts department that can't oder parts and takes almost an hour to bring you an oil filter for a 0.3 job.
My husband works in a very important, very sensitive industry and their biggest competitor just got hacked.
They deal with so much personal information it’s scary that they’re not taking better care of people’s information.
Why should they, 'we'll never get hacked'. Oops....
people are reduced to a number no one cares. we're just worker ants
Why should they? If no one provides better security, then you won't lose customers over it, and the cost of a data breach is 1 year of credit monitoring.
@@werefrogofassyria6609If you get offered credit monitoring as compensation, dot NOT accept it! These monitoring solutions will lock down your credit profile and make it near impossible to borrow money or do any business that requires a credit check, because you have to manually request an unlock at the credit bureau each time, and the credit bureau's system is poorly equipped to handle such requests. Just hire a lawyer to deal with the mess instead.
Russians hackers are really good, but we have to be better.
My daughter is a Service Advisor at a dealership. Its been a nightmare. CDK Global is not one of the best software programs available. But its affecting buying, selling, service, parts inventory, warranty systems and more.
CDK is who we use for our business software. We are a construction equipment dealer, all 26 dealers in our group use it. We have been deadlocked for a week, went back up an hour ago, and now its back down.
Absolutely hilarious. Too bad they ditched ALL use of paper and no one is trained to do things manually 🤣😂🤣😂
take a well deserved break, we do when the VDI is down
Ditching all paper is not a bad thing, not having an independent back-up is tragedy.
The dealerships of bigger cities are often owned by a single corp.
For example, one corp can own the dealership franchise for Ford _and_ a separate one for Dodge, and then Nissan and etc....
So one can go car shopping, just to find all the dealerships are disabled.
A family member who is a salesperson at one of these dealerships, told me they are doing everything on paper- including the finance part as well. It is slow and tedious work, with smaller rewards. But it's something. They have never closed completely. The guys are still getting a small paycheck. It's a true test of resilience.
I feel for the folks who are stressed out this month, over paying their rent or putting food on the table for their families!🙏🏼
Found out the Nissan dealership closest to me has common ownership with the Hyundia dealership next door.
Tried to book an appointment for my Leaf: and the were asking about a Hyundia I had taken to the other dealership for recall work.
Usually, it is a few big ones that own most and a few smaller ones. You always get a better price at the smaller ones. They have fleet costs they can not spread to other locations.
This type of dealer computer support has been around since the 1970's. The two providers,who controlled 90+% of the industry, were ADP and Reynolds and Reynolds. They did business on their terms: sign a long term contract, and, if the task employs numbers( payroll/payables/taxes/ record keeping/
you name it. The gimmicks:
1) " We do all your work. Period. "
2) the two providers used deliberately incompatible programs...so, if you decided to switch to a different lousy provider, you had to hire teams of people. To manually enter all data to new system. With keyboards. Time estimate for task: medium size store was about 10, 000 in labor.
Warren Buffet would call that a moat to retain customers.
Sounds like a monopoly even if it's two companies setting it up to be no competition.
CDK used to be ADP. They split off years ago.
@@angelachouinard4581Technically it's a duopoly (2) or oligopoly (3+).
Retired a few years ago myself...
So please fact check!!!!
I'm old enough to remember when everything was done with pen and paper. When you went to a parts store they had books lined up across the counter to look up the part
MY napa still have their books when the computer can't find the part the book always seems to come up with the solution to the problem
Pretty hard to hack a book!
There are a LOT more parts now.
@@kenmore01 The point is, our world worked once without everything "needing" a computer. Still can, if we want it to. And more and more, we want it to.
@@bastardferret869Exactly, and you can't even buy groceries if the power goes off now because clerks can't even do simple math now.
I'm old enough to remember calling my banker to let them know I was gonna buy a car and I'll stop by to sign the paperwork if I write a check for more than what's in my account.
I'm old enough too. I remember taking my son to the bank to open a checking account in his name as he was going off to college. On the way out, the banker said in a quiet tone, "Don't worry, if he overdraws we'll give you a call before we bounce anything." He never did. But those days are gone.
@@jeepienI now live in a bigger city but back only 20 years ago that was the way it was in my hometown of 25,000 people.
Yep, been there with local bank when I was arranging to buy a house at auction and move it. “I’ll have the gal write up the paperwork and you can come in and sign it Monday, but if you need to write a check now, we’ll cover it.” Handshake and done. I miss those days.
@@jeepien My dad used to play golf once a week with the manager of the bank we used. If my account was a little short I would get an earful later. Only did it ONCE.
Pepperidge farms remembers a time before late stage capitalism too. Good ol days.
This is one of the main vulnerabilities of SaaS and cloud-based software. Too many companies don't know how to secure their systems and all these hacking events prove it. How big a hit will the economy take if no one in America can buy a car for the next week or two?
I am a technician for a Chevrolet Buick GMC dealer in Oklahoma City, and this has crippled our business. Plus, we just had a storm blow through that took out the power for nearly a day. We could not order parts, write ROs, or sell cars for 3 days. Finally, we got a system in place to hand write everything. But it has slowed our processes to a crawl. Great video. We have had to do our own research because we aren't being told anything. Just they are working on a fix.
Why do I have zero sympathy for car dealers?
Because you are human and have some sense?
Because there are too many shady actors in the business. If the businesses were relying on off site storage for all the sales information entered into this system they were fools. Such important information should be stored locally as well as on remote servers for backup reasons. The bigger issue is if the program has mandatory online license checks that happen every time it's opened up with no offline recovery mode. I suspect that the worst of the hacking attacks are blocking any access to the servers for the remote check and data access.
I feel sorry for the customers who are stuck mid- purchase.
You shouldn't, but with recent law passed that all cars will have imobilers mandated this is just a taste of what's to come.
Dealerships? Correction: Stealerships.
Any business or government agency that has been hacked should be required to disclose that information before the customer makes a purchase.
Ah, so you want there to be a checkbox before you do anything that the company has been hacked. It's a worthless statement.
@@werefrogofassyria6609
Put it in the ads
they will definitely make you sign away this right in the terms and conditions....
Thing is it's possible for even the most secure company to be hacked. It's culture and their response to the hack that's the important bit. Unfortunately, that's very hard to enforce.
As an Info Tech professional, it boggles my mind that anyone is susceptible to these types of attacks. Even back in the 90's, I had backup rotations and system restoration media that would allow me to restore any and all of our systems going back as far as 5 years. I had a daily rotation for 7 days, a weekly rotation for 5 weeks, a monthly rotation for 13 months and a yearly rotation for 5 years. Basically, a rotation of 30 backups.
In the best case scenario, I would lose 1 day of operations and data. In the worst case, I could "start over" from any one of 29 other points of reference, and one of them would certainly be from prior to the hack. I would lose 1 day of operation and as many days/weeks/months/years of data that it took for me to get to a point of reference from before the hack.
But at least everyone would be back "up and running" on the software by the next day, even if prior data was lost.
With Oracle you have every second backed up, why do you think Amazon uses it.
Been there done that. The malware may have been in the system for months before it executed. The backup rotation cycle may be so old, or cycled out, so all backups might have the malware in them. This is the new way of organized criminals.
Having worked in dealerships for some time I can tell you that CDK, a spinoff from ADP, outages are not crippling to dealer business but are a major pain in the rear. Business goes back to pen and paper. The real hassle is eventually having to catch the computer up to what has occured during the outage. Parts sales, inventory and so on are all handled by the system. All retail transactions be it sales, service or parts use the system. Accounting is probably paralyzed so if a dealer owes you money prepare to wait. I don't envy the current employees.
Whereas if you operated on a secured and backed up internal server and uploaded as needed, there would be no issues. Incompetence and laziness.
Software as a service or SAAS applications are subject to the same problems any application is. The interesting thing about these latest attacks is they hit aggregation points and take out massive amounts of businesses. This has happened in healthcare as well.
Can you imagine? Gong into the warehouse to take out the old typewriter and fill the forms that way? Then have someone go to the DMV to file the paperwork in person instead of electronically? Back to the dark ages! 😹
About three summers ago our former provider was hit. When I came in for an appointment they informed their computers were down and needed to fill out paperwork. As the appointment went on it became clearer how bad it was. They had bits and pieces of my medical history with them. They did a routine EKG, but then told me they had nothing to compare it to as the one from a couple years back was not there. I took my mother in a couple months later for a routine appointment and while they had the system back up there was a lot of "missing information' which lead me to believe this was more than just "unauthorized access on our network" as the 'Cover Our A**es' letter they sent to me tried to downplay the events. This was more a ransomware attack. Plus, I still believe this incident was 'ground zero' for the identity theft/fraud using my late father's name (also a patient there). The timing of everything lines up. Even a couple years after the fact I am still discovering more fraud as a household Change of Address as exposed more. About two years ago they ended up merging with )or more likely being assimilated) into Dignity Health. Always seemed odd to me as they strived on being an independent provider, but I suspect this incident is what lead up to the merger.
@@LoneTiger Pain in the a**, but a computer work around is fillable PDFs with a folder for each type. That will teach those young whipper snappers a thing or two.
It's because these systems are vertical stacks of data. CDK application is just interfacing with data and providing basic CRUD and I/O. Even most modern node / kubernetes applications, which are decentralized back ends, still have the single point of failure of their database, which is centralized and vulnerable. Clearly CDK did not have proper offsite backups, third party backups or isolation processes for their data. Perhaps even their source code as well, which prevents them from firing up new infrastructure AND restoring the data. This is gross incompetency from the top down.
@@TheeGlocktopus It's not just the data backups. The sophisticated hackers clean you out entirely, AD/Network&Firewalls&LoadBalencers/Physical/Virtual Servers/Storage/DB configs. Putting all that back together so you can even begin to do actual Data Backup Recovery's is gonna be days unless you have a specific Ransomware Recovery Plan to quickly get all these pieces back running without the hackers finding this recovery data and "locking" it up.
I'm old enough to remember typing documents on an IBM selectric using an OCR ball.
No ransomware there lol.
I remember my father, an architect by profession, typing long technical letters, with his right hand index and second finger (because he had a cigarette in his left hand). On an old upright long throw manual typewriter, that must have been pre war. He said it was still quicker than sending documents back to the typists for correction five or six times, because typists often have some antipathy to technical words, and decided off their own bat, to spell and punctuate the way they thought it looked right ! I mean you've just written it down for them, so you can't possibly have meant that !
The IBM typewriter was the best.
Ha!!! I'm so old I remember having to use a typewriter-like device to punch holes in cards to create decks that were fed into the computer's card reader.
well la dee freakin da with your fancy IBM selectric. when I bought my first Flinstones pushmobile, the information was recorded on a stone tablet with a hammer and chisel.
@@connor_flanigan You don't fool me. You had a small mastodon for a dishwasher and a warthog for a garbage disposal.
Working in the IT industry, this is pretty normal. Most small businesses don't have / want to have an IT person on staff and it is much easier to offload the server required for the server to the software vendor's "cloud". These same systems are in place in Retail and Medical as well. Epicor is a large retail software company that nearly all Ace Hardware's (independently owned) still license their software through. Epicor's cloud servers cost about the same as having a physical server on prem and replacing it every 5 years.
You know what irks me? When a company gets hacked and as part of their notification they send you they send how to not get hacked information. I started reminding them how it was THEY who got hacked, not me.
Them getting hacked is helping show the problems with always connected devices.
The problem with always connected devices is that the public accepted it...
We shot ourselves in the feet
Ya our voting machines are another
@@samholdsworth420 When you're given one plate of food and it's only got stale bread on it, that's not your fault. When consumers have no alternatives for cars, TVs, cell phones, you name it, that come with features we don't like, it's not the consumers' fault. Our free market is nowhere near being actually free because it's so heavily dominated by a few key players that what those few at the top choose become the only options for the overwhelming majority of people. I'd love a smart phone that doesn't require the use of two hands, but companies like Apple and Samsung are too focused on making smart phones bigger to have a viable option for me. I'd love pants with pockets, but the places I can afford to shop only offer pockets in the men's section (and I got too much junk in the trunk for those to fit me right).
Rollbacks on anti-trust, anti-monopoly, and consumer protection laws have severely harmed us. And considering I wasn't even of voting age when all that rollback shit took place, it sure as hell ain't my fault. We weren't shot in the feet by ourselves as a whole, all-encompassing 'the public'- we were shot in the feet by anyone who voted to strip our economy of necessary protections. If that was you, then by all means, take responsibility, but don't lump me into that group.
@@samholdsworth420 Most people have accepted always-connected devices, but not everyone. For example, I'm setting up a NAS for my sister to use with her security cameras to monitor her kids. This NAS will also allow her to stop uploading data to the cloud with her phone storage. We've priced out a 32TB (24TB usable) system, with a case and board that can accommodate 8 more HDDs.
CDK is a DMS (dealer management system) which you can think of like an accounting system, inventory management, repair and parts order writer, forms printer, bank/loan applications integrations, and other automotive interfaces. DealerTrack is another huge DMS. If you need to run a dealership, the first software you get is a DMS. Yes, these are often run in the ‘cloud’, and sometimes have outages. Then you have to manually write everything and all the olds guys flex and reminisce about the old days of coke and babes.
Don't forget forget about the stick shifts you cant work while yer sucking your thumb because you entirely dependent on the cloud.
This isn't just an outage this is a big glaring warning sign that central based systems are a bad idea. This system was hacked for profit . It just didnt not work because of hardware issues.
The real problem here is the dealerships don't have a disaster recovery plan when said service goes down. That's just negligent as a business to not identify single-points of failure and have a plan B worked out "just in case".
The biggest mistake businesses made when they migrated to computers was NOT keeping manual processes as a backup for situations like this.
As I have said before, lately, convenience has become inconvenient for customers and sellers. This happens when you put all your eggs in one basket, unfortunately. 😢😢😊
I remember working at a community hospital in SC some forty years ago. We were in the process of taking the hospital's daily census from an arduous manual proceedure to a computerized program. The transition was fraught with problems but we finally accomplished that modernization. I remember looking our office manager in the eye and saying, "Keep a record of how to do this manually. Sooner or later, you're going to need it!
Is that even possible anymore without all the paper filing cabinets that were replaced by computerization?
@@jamesphillips2285 My experience with manual backup processes is that even when you have a manual backup process, you dont have enough staff to run it.
This is why having so much dependant on an internet connection is dangerous
The option is to have every dealer own, setup and maintain their own server and software system. And no passing along a copy to your uncle who runs the dealership down the street. See, we tried all that back in the '80s and '90s. It's just easier all around to have the software reside in a central server.
Exactly why I use Hubitat and power backups. I can run all week and then fire up the propane generator, if needed. I have two antenna for TV. I will be OK. 1 watt + 1 watt, etc. I did all the measurements and installed the backups and a generator. I would be upset and have to adjust but be functional unlike most.
This is why I dislike the concept of "Always online". Something like this sounds like it could be handled by a spreadsheet program such as excel. What I suspect happend is they had old hardware that was running an older version of windows that was compromised and they got in that way. Its scary how many institutions still rely on windows xp that are considered critical infrastructure.
XP went out after the square wheel.
It sucks. It affects service, parts and sales. I'm a vendor for dealerships and they're back to the stone ages. Really, it's worse for the service departments, because they're having to do manual repair orders, and the whole infrastructure to do them...charging parts, writing PO's, everything...just doesn't exist anymore. So they can write them, but all of the ones they're doing right now will eventually have to be entered into the system again once it's back up and running. And that will be an even bigger nightmare.
That's the Dangers of Always Online. People forget the old days. People forget how it used to be. It makes people stupid and ignorant. Any Dealership doing things by hand and not relying on this program? They pointing and laughing at your stupidity and ignorance while they continue on, gaining all the business that YOU have lost.
Boohoo. Im sure theyllmanage considering how much they charge.
My father-in-law works in a dealership parts department, and he's been telling us about this the last couple days. He thinks CDK paid the hacker ransom and should be back online by early next month. He's been relating to us how all the young guys that work there have never done anything without a computer, and he's having to step in and remind them about all the pre-filled or automated information that they need to remember to collect to fill out a complete sales slip. I was jokingly suggesting, but it doesn't sound like they have any old sales slips lost in a corner, somewhere, that they could use and say "fill in all the fields". He used to be the parts manager, but due to internal politics (ageism), he's now just a frontline parts guy. But it would have been on him if there just "happened" to be a back stock of paper slips that weren't thrown away. Seems like, after this, they're gonna get a pack of paper slips to leave in a closet, "just in case".
Don't forget the payroll part for the technicians
@@c182SkylaneRGI heard the same thing from my service manager about 2 hours ago
This problem works its way down to cashiers at a convenience store who can't make change in their head if the register is down.
After working a few days, and multiple hours, I don't think most people can. Every engineer I talk to says they couldn't quickly make change in their head after a day of work.
You're that guy that like to produce a penny after the sale is rung up so when the cashier stumbles you can say: "Hey everybody, I'm smarter than the checkout boy!"
"cybersecurity incident" that's one word for gross negligence
Hacked due to negligence and laziness.
Something most people aren't aware of is the impact this is having on 3rd party repair shops. Need an oil change? Sure, no problems. Need an oil filter housing from $manufacturer? Sorry, can't order it, can't get it, can't do your oil change. These repair places can perform the basic repairs, but if you're replacing your tire and it needs a new pressure sensor from the manufacturer? They can't get it. If it's a generic part? Ok, no problem. Manufacturer-specific part? Nope. Not happening. All I can tell you is that if you haven't kept your vehicle maintained and it breaks down right now, you're going to have a *really* bad time.
The shop one of my kids works at has cars stacked up because they can't get manufacturer parts! It's killing the work the mechanics can do and their paychecks along with it.
Ben's on the truck in front of Steve's "The Boss" plate. This is why you do EVERYTHING by hand, then scan it into the computer in case you lose the paper copy through either a natural disaster or your own negligence. NEVER rely on tech to handle ANY tasks involving math.
I don't have firsthand knowlege, so I'm speculating. Yes, the dealers almost certainly pay a subscription to this outfit - but it's more like Facebook than Adobe. The vendor provides a "cloud" application. All of your data is stored on the vendor's servers. You might have an application on your computer to access it, or it might just be a website. But the underlying problem is that all of the data and all of the backend functionality runs on the vendor's servers somewhere only reachable over the internet. Without the vendor's services being available on the internet, you can't access or do anything.
There are no issues with competitors using the same system because the vendor's clients are all (supposed to be) siloed. It's roughly the same as you and your opposing counsel shopping at the same Walmart.
25 years ago I was head of the billing department of a fair-sized medical office. My boss was about to sign a contract for a new computer system in which all of our patient and billing data would reside on the vendor's servers. I said no way, I have to have our data physically stored on OUR in-office server. The sales rep said he could dump a read-only backup copy to me every day (but exactly how we would access such a thing and/or use it for recovery was not entirely clear). The final straw happened when my staff and I were actually in training at the vendor's office. The youngster doing the training answered one of our questions by logging into another client's real data, right there on the screen for us to see (and anyone walking by on the sidewalk looking in the window). Talk about a HIPAA violation.... I ended the training session right then and there. My boss was obviously horrified when I reported what had happened, and he was able to get us out of whatever contracts he had already signed. (And I hope that trainer got fired.)
@@CrankyBeach I worked for a very large (#2 or #3 in the US?) healthcare IT provider, so I understand what you mean. I doubt it was my company (I hope it wasn’t) but regardless I sincerely appreciate that you stood up and said no to the shenanigans of that outfit to protect patients who care about their privacy and believe that HIPAA matters. If more IT people had your attitude - and more middle managers listened to us, we’d have way fewer “oh, another day another batch of stolen PII and PHI”
OMG, can you imagine the hundreds of thousands of notice letters for data breach, identity theft, personal financial and credit data, etc!! Extreme liabilities!!
As a flat rate employee in a ford dealership in MA that is affected by the CDK hack i can say that we are servicing customers, vehicles, using hand written repair orders we are unable to give customers exact price quote for what their oil change brake job windshield wiper changes alignments whatever is going to cost them. We can’t cash out the RO because the computer is down and as a flat rate mechanic it means I’m paid by the billable hours that I do in a week therefore they can’t track my billable hours. They have no idea what to pay me for the affected weeks fortunately I work in a family owned dealership and they are paying an average of what I had done in the last six weeks to include billable hour bonuses for hitting certain milestones commissions based on up sells and other performance bonuses.
I am in the middle of this mess.....my car is sitting at a dealership for the 3rd week because the dealership cannot order the repair parts needed. I had to pay for a rental car for several days, then my dealer has provided a loaner, BUT my insurance coverage is only good for 15 days on a loaner. I don't blame my dealer (which is outstanding and I have bought many cars from them). I am not sure where I will be in the 8 days I have left. We have become way too dependent on computers, and this was just the next in a long line of proof that we are getting screwed over by bad actors.
They should let you drive your trade in in the meantime. It's not like they can sell it!
You took it to a dealer for a repair!! What the F!! Never use them!!! Unless you want to get scammed. Rule 1: never use the dealer if you can avoid it.
@@toriless I had no choice, this repair was to replace the Steering Box, the Rack and Pinon on the vehicle, and a complete realignment. The vehicle was not under warranty, BUT this is a dealer I have been going to for more than 25 years, and purchased nearly 10 cars from, and they have always treated me properly, and this disruption was not the dealer's fault, and they did provide the loaner, so I do not regret using the dealer. for this
Is this the final nail in the coffin for auto dealers? It is ironic that they are dependant on (a) software package(s) to do their paperwork!
Sadly, no. The way automotive laws are structured in America, dealerships are the only 'proper' way to sell massive amounts of cars. There's even some baked-in provisions regarding dealerships having 'territories' or whatnot so competitors can't set up shop too close to each other without running into legal issues. We're stuck with dealerships and automotive laws structured as they are for the foreseeable future.
We rely too much on tech and this is yet more proof of that.
Tech is here to stay.
However relying on a single point of failure software application is foolish.
Every IT course teaches you about redundancy and in this case there clearly wasn't any.
Yeah, like F'g CZcams, how would I get today's Lehto story, eeek, panic time.
On Wednesday, June 19th, I had a neighbor with a leaking fuel pump on his Mercedes Benz. The plastic cap of the assembly had failed with multiple stress fractures, causing a leak - apparently a common issue with this model of Benz. After researching the part for replacement, I discovered 2 different fuel pumps available for this side of the fuel tank for this model - I called the stealership to confirm which was which (maybe one part number was superseded by the other, for example). The guy at the Benz parts counter said the whole system was down nationwide. A friend in Europe also was not able to log into his Benz portal and check for me. I figured it out - one part is for gas only, one part number is for E85/gas cars. I ordered the part from Benz the next day, who had to manually search their stock of fuel pumps to see if they had my part number, and then had to write me a paper receipt for it, lol.
The Europe friend told me it was a system down for maintenance but I suggested it sounded an awful lot like a ransomware attack, esp since it has lasted for so long (3 days by that point). Then the next day this news came out.
Part replaced though, no more fuel leak, neighbor is very happy.
Hey Steve. Love the content of knowledge that you provide! I have had a issue with my local Mini Cooper dealership in Orlando, Florida due to the Hacking of their software. I had to go back twice to make a purchase for a part to my R56 JCW. At first they couldn't make the sale of the part. But on the second trip they ended up writing a sales receipt for the sale. What a headache!
I work at a Honda dealer as a tech, I heard about this from other dealers. We still use an older system, so we weren't affected.
I get flack from our younger employees for not running the newest software. My servers are 12-20 years old, desktops are 0-12 years old, but everything is fast and reliable. About 1-2 hrs of downtime total per year across all systems. This is why I stay with the tried and true.
@@grayrabbit2211 That's like driving a standard now. It is built in theft protection.
Dinosaur tech for the win, I suppose. I use an old phone (original OnePlus) and a patch cable to play music in my semi. I never have to fix the Bluetooth connection while driving. Sometimes I forget to charge the phone though.
@@niqhtt All my personal cars are stickshifts. My daily, my truck, and my bike.
@@SkylineFTW97 yep. Still have my first car too.. A manual 85 Prelude. Now have an 86 Si and my daughter loved it so now she has an 01. She is still working on the shifting skills though. And I have a bike but I have to admit bikes being so flipped around broke my brain, so I just have a little PCX 160. Also.. I was a Honda tech for 15 years. Funny the parallels...
Any business who's model includes, depending on another business to be able to operate, is not the best model. They should always have a backup, even if it is pen and paper. I remember a long time ago when your credit card didn't work, they would paper slip it and process it later when the system is working.
These programs are expensive, and not always practical unless you're in a large auto group where having multiple programs makes sense. For example, Mazda (from my memory) only uses RouteOne. DealerTrack and CDK are a bit more widely accepted.
Now, if your card or their machine doesn't work, they cancel the whole transaction and tell you to come back later.
Mechanic here, CDK is used throughout the entire dealership even the mechanic need it for administrative duties necessary for warranty repairs. There are other OS but CDK is the most common. This is just a glimpse into what is come. Im considering changing careers into cybersecurity!
I worked in the motor vehicle industry for 30 years. I retired a couple years ago after serving as Information Technology Administrator for one of the largest companies in our country. We changed our antiquated system over to CDK Global in 2019. CDK Global applications managed our New Vehicle Sales, Used Vehicle Sales, Aftersales (Service and Parts) and Accounting. However, we also maintained a manual system as an emergency backup! At least we did, up until I retired.
When our network went down at the auction we would go outside and write up the cars vins on the tickets that we usually just printed out.. vin, year, color, make n model, transmission type, mileage.
It can be done & it's time consuming but you have to be prepared
It was done for decades without a problem
There was a power failure at a local drug store. The cashiers were all stuck but then the old pharmacist (who probably also owned the store) came out and had everyone checked out manually in a few minutes.
He said "This i how we used to do it!".
Think about this when all those companies you deal with want to do everything in the "cloud"
I hope the hackers take it all down and make cloud utterly unusable.
People have no idea what the "Cloud" really is.
I never have and never will have any of my stuff there, even though they keep pushing it.
@@DKNguyen3.1415 To what end? Do you not like using the internet?
@@tedhardulak7698 What are you typing this comment on...
@@josephguerassio6680To what end? Is that not obvious to you? To make scammy cloud services unprofitable. Internet != Scammy cloud services.
Welcome to the cloud. An old, circa 1985, large fold out ad in a popular computer magazine by AT&T chided IBM for "owning your data". AT&T at the time was marketing Unix systems on which you kept your data on your equipment. The ad feature a ransom note on an IBM blue background with letters torn out of many different paper publications. The ransom note read "We have your data and it going to cost you to get it back"; seems that we have come full circle.
Worked at a Ford Dealer back in the day and our software was from Reynolds and Reynolds and even back in the ninety's it printed all our forms from sales, service, parts and most importantly payroll. Every close of business we had to put everything from that day on these tape drives that looked like oversize eight tracks, It took about twenty minutes to do a back-up. Hopefully there is an air gapped back-up for these affected dealerships.
I highly doubt their backups are viable if they are as mentioned, allegedly negotiating with the attackers.
*Amazing video, you work for 40yrs to have $1M in your retirement, meanwhile some people are putting just $10K into trading from just few months ago and now they are multimillionaires*
Hello , I am very interested. As you know, there are tons of investments out there and without solid knowledge, I can't decide what is best. Can you explain further how you invest and earn?
Hello how do you make such monthly?? I'm a born Christian and sometimes I feel so down 🤦♀️of myself because of low finance but I still
believe in God
I'm favoured, $90K every week! I can now give back to the locals in my community and also support God's work and the church. God bless America,, all thanks to Ms Louis Stella 😊🎉
Good day all👍🏻 from Australia 🇦🇺. I have read a lot of posts that people are very happy with the financial guidance she is giving them ! What way can I get to her exactly ?
She's always active on Whats~App... 🎉
Ben $ is providing shade for the white van, underneath "The BOSS" plate.
From an IT perspective (in another industry), what I've seen happen often is where companies using one of the leading software platforms and the vendor discontinues the in-house licensed platform. To keep up with regulations and updating accounting forms, the vendor pushes clients into a online cloud platform; thereby, forcing monthly subscription fee based pricing. The client companies then downsize their in-house IT, accounting and logistics staff and switch to the vendor's newest all powerful super duper cloud platform and the savings from downsized staff go to the significantly higher costs of the monthly sub pricing model. The executives justify it by saying the cloud vendor is now responsible and it's not the client company's problem, but the clients are ultimately still responsible for business continuity or non-continuity (i.e. and take all the risk while function is out of their control/hands). If your cloud vendor didn't have at least multiple immutable backup strategies, a simple compromise of the cloud would yield a near total loss.
This is why competition is important. If there is one company dominating a market, it only takes 1 hack to get everyone's info
There is competition, and cdk is not the largest player. Reynolds and Reynolds is in over half of all dealerships.
I’ve been around long enuf to remember that automation was going save the trees and time. 30 years or more later its done neither and in fact created all new problems.
Yeah, the joke at work is that we've actually _increased_ our paper usage since going "paperless", rather than decreased it, because now, instead of doing all your edits in pencil on a single draft, we print off multiple copies each time we make a revision so we can hand it around for review, again. (COVID actually did us a solid in that department, because it forced us into a much more digital footprint when nobody was sitting next to anybody to hand a paper copy to, anymore).
You believed that? People said the same thing with the invention of steam-powered machinery. So long as there's a greedy capitalist in charge, any innovation that saves time, money, effort, etc., will be used to benefit the fat cats. All the little guys? 'Sucks to suck, pull yourself up by your bootstraps, just be rich,' yada yada...
So what I’m taking away from this story is that there is one software company which has a monopoly over the auto industry.
From the comments, it sounds like it's a Duopoly: one major competitor. It's possible there's a third option, but I highly doubt there's any more than that.
And one hacker group that's smarter than all of them put together
@@gorak9000 Blacksuits - Russian
Use for their financials. They require you to sign expensive long contracts for each. They also are the clock in/out for paychecks for many dealers. So in a way for people that dislike the subscriptions services and dealer and documentation fees, it gets paid forward to dealers that choose these systems which are nearly all of them
When I did dealer finance back in the nineties, we used EDS and Reynolds and Reynolds. One or the other would shut down every now and then. Didn't seem to be a huge deal. I would use a handheld lease/finance calculator and hand print the contracts. As my GM would say, "the show must go on".
I have the world's smallest violin playing just for car stealerships
I suppose you don't mind working for minimum wage, right?
Lol
It has affected the whole chain top to bottom, left to right for the entire automotive world, including trucks, construction equipment, mining equipment and more.
Independent mechanics, dealer mechanics, suppliers, distributors, truckers, tow trucks, long-haul truckers, industrial equipment, parts suppliers, taxi/uber/doordash drivers, buses, anyone and everyone who might rely on any sort of vehicle for transportation or work.
Please don't direct your violin from shady dealers to the entirety of the industry.
This is one of the reasons I don't want to have any "smart" devices / appliances. If it can connect to the internet, it can be hacked and compromised / disabled.
We have WiFi light bulbs and a Ring doorbell. That’s the extent of our IoT (Internet of things).
You gotta ask the same question every day: "Why is the ONLY way to do business on the internet?"
I have a 3d printer that DEMANDS I print using cloud-services...if I refuse, there is a lengthy process to manually transfer files one by one.
Can't connect using USB, even though the printer has three ports on the front panel!
"Not communications ports, only for use with memory-sticks or web-cam."
It's a shame...damn good printer comes equipped with messed up interface.
A couple of years ago I was at a friend's house. We were in the kitchen. My phone was in my purse in another room. The next day I got a ton of ads on social media for a kitchen gadget the friend had shown me. Creepy. "Of course no one is spying on you. You're just paranoid," said the microwave oven.
@@CrankyBeach Yep. 2 months ago I was looking into getting a new credit card, and talked to my mom over the phone about it. Less than a week later I get 3 different letters in quick succession from Capital One, Citi, and Wells Fargo with "pre-approval" notices. God that made my skin crawl.
Exactly. I don't need a smart fridge, i need a fridge that keeps food cold. I don't need a smart dishwasher, i just wand a machine to wash my dishes. Same with laundry devices, microwaves, stoves, ovens, air conditioning systems, etc.
Now, a slightly intelligent thermostat (that can automatically switch between heating and cooling), that's fine, or maybe even have a schedule i can program/profile in for a week.. But it does NOT need to be on the internet.
And i don't need my data on a cloud somewhere. I want it on media i control, and no where else.
Last week Thursday I had an 8 am appointment at my local KIA dealer. I got there and it was surreal since no one seemed to be around. Luckily one tech let me into the garage and asked "How may I help you". I told him I had an appointment, and then he mentioned to me that they had been hacked. I was pissed that I was not called at least to say do not come in. But at the least, the tech had a pad and a pencil and took my name and phone number down. Then I realized that not only could they not do the work because their computers were down and could not find the parts, but they also lost all their customer data. How can a company not have a local backup to at least know vital information about their customers? Anyway, I hope to return tomorrow to have the work done, by them looking up my VIN number to find the needed parts for the brake job, since they still have not got their system back up and running. Good work CDK👍, and I hope a few of those 15,000 dealers wake up to how stupid their whole system is.
I used to be in the Dealer Management System (DMS) and a competitor of ADP which through merger became CDK. Our system put the server at the dealership which has its pluses and minuses. But communication with Manufacturer always go thrtough the DMS to the OEM. Therefore the systems are always at some sort of risk. CDK's system to my knowledge is similar to your description and a subscription is a good word for it. I believe that CDK nolonger puts servers in dealerships, so when their data center gets infected, it can affect many customers at once.
So dealerships are being taken for a ride....Oh, the irony!
In the old days of computers, they were big, so multiuser setups came into being by having dozens of terminals scattered around the office building, cabled back (remember RS232) to that computer.
Then the personal computer happened, and people could have their own detached computers, that didn't depend on a vulnerable centralized location.
Now the internet age, and that beget "the cloud", that looks remarkably like the old "timeshare" mainframes of the 1970s ! Let's reinvent the failures of yesterday !
You don't know what you're talking about do you ?
@@jean-marcknight8816what aspect of the thesis are you questioning?
The thesis that computing was centralized, then somewhat decentralized, then back to more centralized computing is a valid statement.
"cloud" just means sharing someone else's computer. That's conceptually similar to the TimeSharing Option on mainframes that has been around many decades.
@@handleit2000 We're questioning when exactly corporate office computers were decentralized, because that never truly happened. The "clouds" were always there for large interconnected data structures.
@@MK_ULTRA420true, the big iron and centralized computing never went away.
There was a move towards decentralizing, then a move back to centralization, at least from the aspect of what articles popular computing news media published. 😂 😂😂
@@handleit2000 I question everything, It is just a word salad. I don’t know how you can find that "conceptually" mainframes and cloud have anything in common.
The primary reason for the central system design is that dealerships want to avoid paying for the computer hardware and IT staff to run the software locally. Dealerships want to outsource these costs to the vendor as the vendor can do it cheaper. Plus, performing software upgrades on a complex system in a local platform is painful. Most banks and mortgage companies follow the same model, where the software runs in a central location.
Steve, along with solid explanations I appreciate the small changes in the background.
Stupid to have a single point of failure without a local or >paper< backup. That is also what a monopoly gets you. I do hope the FBI is involved.
Redundancy is expensive. Car dealerships won't buy *pens* for their salesweasels. You think they're going to want to hear about backups? They'll never get hac... oh wait.
@@zburnham I suspect the bad guys got malware on the server(s) which encrypted the customer database(s) using a very large random key. A ransomware attack. They can pay up or rebuild the systems from scratch and use backups of the database files. If they have backups of them......
If only we could buy cars from the manufacturers...
Bought my first car from a coworker in 1981. I paid 500 dollars for a 1974 green dodge dart. It was beautiful.
We were affected by this . Our southern Oregon Subaru dealer was affected. They held our car for 3 days before they figured out it was finished.
My sister works for their main competitor. The morning it broke I called and woke her up to let her know. She was dancing over the phone.
As a retired automotive and financial industry IT specialist, I saw several ransomware attacks on similar systems. The outages caused were measured in hours. We’ve also seem similar outages of government and business system reported almost daily. Those systems were segregated, fire walled and backed up. Somebody failed here, severely impacting an entire industry.
So glad my company does daily back-ups. Got hit with ransomware a few years ago. IT team laughed at the hackers and loaded a back-up. One of our competitors got hit and they were down for almost 2 months. I agree someone shit the bed on this one.
@@God__Emperor_that youtube handle lol. Backups only work if you fix how the hackers got in though. Load everything back for them to just use the same exploit isn't really a solution Yes/No?
@@MathiasYmagnusHackers and the Chaos have much in common.
@@God__Emperor_ These criminal organizations are getting more clever and patient. They may have gotten in months ago, waited for all of the backups to cycle through before they executed the attack. Thus, the backups don't help.
@boomergames8094 this happened 5 years ago, I think we are in the clear. But I get what you're saying.
I did IT for a Chevy dealer during covid when their IT guy died. They used a similar system. Totally SaaS (software as a service) for sales, parts and service. No local server to maintain. Just a relatively simple network with gigabit fiber and really squirrelly printer configurations.
I used to support this DMS before ADP spun it off to CDK.
Back then, the software only ran on a server at ADP. Dealerships would actually Remote Desktop into the server to access the DMS as if they were local. They could only see their dealership, but they were all logged into the same system. There were lots of bugs, so ADP didn’t want to deploy it to clients as a standalone software license.
It wasn’t very secure back then.
Sounds like it still isn’t
We are all overly dependent on technology.
Cell phones, computers, scanners, internet ...
Keep you pen and paper handy.
Exactly. They don’t teach writing in schools anymore from what I’ve been told so those kids will go into full panic mode when it happens to them.
Better start growing your own food and dig your own well. Its easy to hate on tech but we litterally could not sustain the number of humans that currently live on earth without tech.
That’s probably why I hate technology so much, it sustains all these people that I can’t stand!
My thoughts exactly….
If you kept your entire budget on a single sheet of paper, and that paper blew into the fireplace, would you blame your dependence on paper or would you blame the lack of redundancy?
It's not about reliance on technology. It's about reliance on a single point of failure.
Oh no, the forcefully required middlemen can’t do business marking up cars, what a shame. Would be a good excuse to just start doing direct manufacturer to consumer sales.
Dealers should just go away.
Happening in BC, Canada as well. Our local Honda Dealer can't sell cars, service cars, or sell parts in the parts department !
Ive used CDK at a tech at a dealership since 2006 and the software hasn’t change at all.
The Irish potato famine was from planting ONE crop.
And -ADITIONALLY: Just a few years before the infamous potato famine, Ireland was the site of a massively successful temperance campaign led by the noted Catholic priest, Theobald Mathew. So successful was this campaign that between 1838 and 1841, their national alcohol consumption was cut in half.
"...
Within Ireland itself there were substantial resources of food that, had the political will existed, could have been diverted, even as a short-term measure, to feed the starving people. The policy of closing ports during periods of shortages in order to keep home-grown food for domestic consumption had on earlier occasions proved to be effective in staving off famine within Ireland. During the subsistence crisis of 1782-84, an embargo was placed on the export of foodstuffs from the country. The outcome of this humanitarian and imaginative policy was successful. The years 1782-84 are barely remembered as years of distress. By refusing to allow a similar policy to be adopted in 1846-47, the British government ensured that Black ’47 was indelibly associated with suffering, famine, mortality, emigration, and to some, misrule."
"Almost 4,000 vessels carried food from Ireland to the ports of Bristol, Glasgow, Liverpool and London during 1847, when 400,000 Irish men, women and children died of starvation and related diseases. The food was shipped under military guard from the most famine-stricken parts of Ireland; Ballina, Ballyshannon, Bantry, Dingle, Killala, Kilrush, Limerick, Sligo, Tralee and Westport. A wide variety of commodities left Ireland during 1847, including peas, beans, onions, rabbits, salmon, oysters, herring, lard, honey, tongues, animal skins, rags, shoes, soap, glue and seed. The most shocking export figures concern butter. Butter was shipped in firkins, each one holding 9 gallons. In the first nine months of 1847, 56,557 firkins were exported from Ireland to Bristol, and 34,852 firkins were shipped to Liverpool. That works out to be 822,681 gallons of butter exported to England from Ireland during nine months of the worst year of the Famine.”
- Learn About the Great Hunger, Ireland’s Great Hunger Museum
I work at a dealership affected, this is a "all inclusive" system: sales, service, parts, book keeping and Finance. employees aren't sure about pay, retail buyers are personal info compromised.
F em
Steve... remember, anything "cloud" based is just using someone else's computer remotely..
I'm sure the dealers were sold this on the promise that they don't need an IT department to maintain software and the service provider will provide "everything" they need .
I always advise that any cloud service should be able to provide you (or a way to obtain) a backup of your data on a regular basis for you to store and use, should the cloud service cease to exist, or there are disputes, etc.
This happens when companies like CDK are allowed to get large enough to squash competition and become sole source providers. Many typewriters, pens and sales invoice forms come out of storage closets just like the good old days.
if only theres was some kind of law against that, oh well..
@@ryderlemon8908 It's sad the feds don't enforce anti trust laws.
Screw car dealerships… I bought a new Nissan a number of years ago and was promised Lifetime oil changes complete BS. They tell me they only do synthetic oil changes and my oil free oil changes is now void.
They can’t discontinue regular oil changes if they promised to provide regular oil changes. Small claims court? Of course I wouldn’t want to go through this, but maybe one of the scores of customers they arbitrarily cut off will.
Centralized power in any form is a horrible model unless you’re the one profiting.
It's amazing how bad things can get when you put all your eggs in one basket.
The reason why this company didn't make it known how deeply entwined in all aspects of the dealership business they were is because then people would be upset that they were a MONOPOLY
Most commercial software, from small to large process remotely. It’s good for wide distribution of updates, security (backup) and subscription control. Obviously, when hacks or crashes occur, this model causes a lot of pain.