What is the Log4J vulnerability?
Vložit
- čas přidán 12. 12. 2021
- The Log4J logging library for Java is used in thousands of applications, and on servers all across the Internet. There is a 10/10 critical vulnerability that can allow remote code execution on any affected server, if it has any connection to the Internet.
How will you be affected? Well, hopefully not that much, if you don't run your own servers. Regardless, always make sure to update and patch your servers and applications, and hopefully we'll get through the next few weeks okay... this vulnerability's already being exploited a lot!
Support me on Patreon: / geerlingguy
Sponsor me on GitHub: github.com/sponsors/geerlingguy
Merch: redshirtjeff.com
#Log4J #Log4Shell #DevSecOps - Věda a technologie
It's easy to forget that some kids who started playing Minecraft back in the day already grew up and are now infosec specialists.
This is true... I wasn't quite a kid the first time I played it, but it was _very_ early in my career!
It is also equally easy to gloss over the fact that many more kids who started playing Minecraft back in the day are now just adults playing Minecraft. Some of them actually admit this and stream this online.
@@Diggnuts It's quite hard to gloss over that when some of them stream it, tbh
@@jablue4329 That is correct, albeit still a bit puzzling to me since I never search for Mindcruft or other new game related stuff, but still their they pop up in some suggestion feed. Suggesting I should watch some middle-aged basement dwelling loser play a game for children with medium attention spans. I find it rather pathetic.
@@Diggnuts And you only do things for serious grown up adults, like complaining on the internet about the way people you don't even know have fun.
yea... as a software engineer, my Friday was loads of fun - i.e drop everything and rebuild external facing systems.
Any security-conscious company basically sucked up at least Friday and likely some of the weekend because of this :(
Why doesn't it ever happen on a Monday???
@@JeffGeerling indeed -- brady bunch style MS team video calls, fun stuff
We had the double header! Fri-Sat to update, Sat-Sun to fix what the update unintentionally broke. SMH
@@JeffGeerling The whole IT department at our company just noticed it today... doesn't speak for them... but they had a pretty busy monday today haha :D
@@thommy270 This was actually announced Friday to ruin anybody's plans for the weekend. Kinda surprised the entire IT dept "found out" about it on Monday. Hopefully they don't have any internet facing servers with this log4j vulnerability. I have seen several attempts on my Linux servers and are showing up in the access.log logs. Lucky they're don't have log4j on them. In fact I added a rule in fail2ban to keep track of the attempts.
I would like to request a minute of silence for all the IoT device that aren't patched and never will be that will flood the internet in like 12 days X)
Only time will tell if it will be mirai all over again
The majority of IoT devices are clients and not servers. And Java eats RAM so lots of the server-type IoT devices stays away from Java.
So it shouldn't be that high percent IoT devices that are suffering from this. And sane routers/firewalls shouldn't expose server-side functionality to the net. So it would be the server functions hiding behind that are vulnerable.
@@perwestermark8920 there are also firewall/waf/load balancer rulesets out now to block this.
@@perwestermark8920okay, so what about Java embedded?
Can make use of the same Log4J library.
Besides, it depends on the IoT device whether or not it is feasible.
@@perwestermark8920 5% of servers run on Java and thats a ton. Also it's not the 90s anymore. Java barely uses any extra ram than any other language. Your not gonna notice issues from ram, the biggest issue is java being stupid for when it should handle garbage collection.
@@themultigamer5682 I still haven't seen any Java application be even close to a C++ application when it comes to RAM use. For IoT, RAM matters a lot for cost. And for servers it also quickly adds up - 256 GB instead of 128 GB or 64 GB does matter. The ability to have complex data types side-by-side in a memory block matters quite a bit, instead of requiring it to be pointers to new memory blocks.
I dont get why things like fridges, Nests, RIngs, etc need a remote internet connection for you to use them while you are at home on your own local network. What is even worse is that nobody questions it.
Really wish there was a community effort to produce open-source IoT hardware that rivals the closed-source stuff.
@@Megatog615 whilst not hardware-only, this is one of the mantras/benefits of the Home Assistant community. Local first, and then controls around cloud if necessary.
java want to be backward compatible, that idea was before rest API making JNDI now obsolete but java like to keep its backward compatibility not sude if they would remove it.
They sell you cheaper electronics by subsidizing it with your personal data. They gain insights on users while offering competitive pricing.
@@benargee If it were just about money, they would offer both versions. The spyware version at a discount, and the non-spyware version at full price. All the evidence points to it not simply being "business". Major influential companies often make decisions that are against the population's interest, even when they know they will take a loss for it, for whatever reasons, and then they lie about the motives (ie giving explanations that dont make sense to people that are knowledgeable) or say things like "its because they care, and its for your own good" etc.
And that is why I had to do production testing after an all hands update and deployment.
That is a valid reason to use youtube shorts for once.
You are a legend Jeff. I had been hearing about this in the news. Thanks for explaining it in a simple way that's easy to understand.
Also made a Playbook and ran updates overnight when updates became available.
Hadn’t read up on the vulnerability yet, looks much more serious than I had initially assumed. Very cool to see that Ansible makes the processes less painful.
Yep, someone on Reddit was able to use ansible to patch 40 vmware vCenter servers. Very cool to have that kind of automation but like with anything new it needs to be tested before applying it to production. Linux are very unforgiving machines and will do what it's told with few safeguards in place.
I've been working on that same issue at my work -- it was a Happy Firetruck Friday
Always on a Friday!
This vulnerability caused nightmares for lot of developers for months. And thanks to minecraft using java to identify this bug, else its their for years, until it's discovered.
good luck getting updates for any " smart thermostat " or older router..
The modded Minecraft server space was on fire for a while (and still is). There have already been documented uses of this exploit. (I don't have a source for that, lost it)
Ldaps into your server,
obtains root shell,
updates your java,
refuses to elaborate.
Disconects.
Working as a security analyst, it was a tough weekend! 😅
We thank you for your commitment :)
I am so glad I retired from that.... and only have to worry about a handful of things in my DMZ. Which I just turned off and unplugged until I feel like dealing with. I feel for you.
Why one earth would anyone design a logging package that allow remote code execution.
It's basically feature creep-some company at some point probably needed to be able to pull in templated bits from some other server/service, this feature was added with what they thought was adequate protections... but it was obviously not enough :(
This is one reason many libraries/projects are very trigger-shy when merging any new code that interfaces with remote protocols/servers. Even really precise, well-written code can have obvious-after-the-fact bugs that are hard to spot until exploited.
@@JeffGeerling This is why I never give OpenBSD any crap for refusing anything that doesn't have sane default values and verifying that they themselves can inspect the code. If most people were even aware of that setting, they would prefer to turn it off than have it run by default.
@@johnnystorm4139 Exactly. Secure by default. If you want a potentially dangerous feature you must explicitly opt in.
@@johnnystorm4139 oh I'm sure the next release of log4j will have formatMsgNoLookups true by default lol
@@andyhall7032 I can't tell if you're joking or not, but that's literally what the fix does.
Affected is only v2 log4j library. v1 is safe and older devices does not need the patch 😇
It pays to stay behind 🤪
Log4j v1 has some staggeringly bad security bugs and worse isn’t supported anymore.
I'm using a private thermostat, how i can know if is thermostat uses java 🤷😱😢
Pour some coffee on top, see what happens...
(Don't actually do that 😅)
@@JeffGeerling 🤣 (only tech people will understand)
@@JeffGeerling 🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣🤣
I work at one of europes biggest B2B IT Providers and you just gave me flashbacks...
I remember the evening this happened and I had a guy looking into it in the Minecraft community. I saw server after server go down.
This is a big deal for sure. I have already seen comments from other IT professionals on reddit and the like that are not taking this seriously (good luck gents/ladies on finding another job if you get hit).
Do you happen to have automation side handy for this? or Put up a tutorial video on how you did it? It would be a good exercise for us Ansible-ites
In my case, for Hosted Apache Solr, since I was already running all the affected apps (Solr instances) inside Docker containers, the easy fix was to apply a solr option flag in the container image itself. I just updated the image (using my existing GitHub Action that builds and pushes the image to Docker Hub), then had an Ansible playbook pull the image to each server, and restart all affected containers.
Otherwise, if Solr was running directly on the servers, I would've had Ansible replace the /etc/default/solr.in.sh file with the right flag.
This issue is by far not as severe as it sounds. The message would have to be logged and whatever system that evaluates the expression, would have to be available and actually be able to resolve and execute the remote resource.
@@eslofftschubar206 a 10/10 CVE and able to remote execute payloads. Surely you jest?
@@ReonBalisty Yes. what that 10/10 doesn't take into account is the probability of all preconditions to be available. Of course an update is required and it's also quite a stupid issue, but nothing that has an immediate impact, like heartbleed had.
In a production environment, you usually have to few logs and due to data protection you wouldn't be logging client data anyways.
In my 13 of java, I had to smell the issue more often than i could figure it out from logs.
A minecraft server is in danger, a banking application isn't.
@@eslofftschubar206 a lot of servers can resolve JNDI queries and route outbound just sayin'
Would have been good to mention that you should block outbound LDAP or mitigate some other way first. Then you can figure out what to patch.
Heard about the vulnerability a few hours after it was released as a CVE, which made for a fun weekend. Remember Java loves talking ab its 3 billion+ devices that employs it service wonder how many will never be updated :)
The vulnerability highlights the danger of using "eval". The feature might have been safe in a vacuum for its original intended use case (although my gut says it is doubtful), but such things are often not safe under maintenance or when looking at the behaviour of the system as a whole. Even with sanitization, eval involving potentially user provided data is risky. I think whoever implemented this misfeature probably never intended it to be used on user input, but forgot or didn't know that the format mechanism they used was applied globally.
Confounding the whole thing, when I first read about the cause of the vulnerability it wasn't immediately clear to me that it was an RCE because the eval is slightly obfuscated. The documentation made it seem like it was some sort of obscure LDAP thing, perfect to make anyone's eyes glaze over. So I can understand why it was overlooked for as long as it was.
Yeah honestly the first reaction was "oh so if you don't have a local LDAP server, this won't cause any problems." But it wasn't obvious that the LDAP server could be anywhere on the Internet-even one under a malicious actor's control...
@@JeffGeerling It's actually worse than that, Jeff. Apparently, the 'feature' was enabled for LDAP, so the LDAP call is the trigger, but no LDAP server anywhere is needed for this exploit to work, which is why it's so dangerous.
@@johnnystorm4139 no LDAP server is required for data exfiltration, but an LDAP server (not necessarily a real one, just something that will return the serialized object) is required to get RCE
When I first read about the vulnerability, I couldn't believe it. When I finally believed it, I came to the conclusion what a piece of shit this library is.
@@johnnystorm4139 Yeah, I think he was responding more to my comment that the documentation made it seem like it was the type of thing you could ignore if you weren't using LDAP. I think a lot of people did
So you are telling me the monero hash rate is on its way up
Heh, probably!
just in case anyone needed another reason to hate on java
lol, delusional
Ty for the tldr I probably would have ignored otherwise
It gave me such a headache when this came out. Worked entire weekends to scan and mitigate thousands of servers. Horrible. Don't want to remember!
The sky is blue and a JAVA implementation has a security bug.
But Log4J is not even a bug, the code did exact what is was written for. It worked as intended, specified and documented. If you had tried to file an Issue with that some weeks ago you would got a "NOTABUG, WONTFIX" ... and then crap hit the fan.
Minecraft is literally the root of finding bugs with it gigantic community now.
Log4j is a Java library that is used for logging errors and other software activities. ... The exploit lets an attacker load arbitrary Java code on a server, allowing them to take control.
Squire by Fender means I can’t play guitar but I still hang it on the wall anyway
Good thing I don't use the Internet
Wait a second... how did you post this comment!?
@@JeffGeerling Psychic (psycho?) verson of Pre-beta Win-12.
he's safe, he used an app, not the internet.
@@JeffGeerling he wrote a letter
Any organisation using the Allied Telesis enterprise range of X Series switches and UTM Router Firewalls will be unaffected. The Allied Telesis Vista Manager centralised management software latest few versions are also safe, but older versions are vulnerable.
We were lucky there was only one third-party software that was vulnerable, but still, that kept us on our toes throughout the weekend. 😁
I remember when the news broke and thousands of minecraft servers were destroyed in days, with people gaining full access to other people's computers
Regarding automation: Can you make a tutorial/ video how to automate software updates with several Raspberry Pi’s and a caching service (apt-cacher-ng)?
That would be great!
I have got a Pi hole with apt-cacher-ng installed, but I am struggling to automate update so the other Pi's can grab their cached updates from this machine.
As an IT Pro I.appreciate this video.
Feel sorry for anybody running JAVA. No JAVA for me.
This has nothing to do with Java.
@@31redorange08 well u dont have the vulnerability if you don’t use java. 😅
@@ratulsaha9487 While true, your statement is useless.
Lesson learnt: using print is better than using log, I knew I was right
How can one audit their devices for this issue?
More important, how can we check whether the ISP-provided modem-router is vulnerable?
Same here. Burned my weekend doing the same, then all week with long days and nights, though working as part of the incident response rather than the IT side of life.
The biggest issue... the amount of BS and FUD spread by twitter re: 2.15, 2.16 and 2.17. Trying to keep on top of the higher priority concerns and work out suitably simple enough mitigation actions for our IT staff to follow.
LOG4j May Be Massive In The Web And Server Space, But Pales In Comparison To Java Minecraft. All Hell Broke Loose Last Week On Minecraft Java.
Oh man what a day I had..
I do not get it why companies still have no automation in place like Ansible. I use it for everything. Nothing gets ad-hoc done anymore.
Weird... Seems to be exactly as intended
Ok I think I’m covered. I just ran Sudo apt-get update.
Only the oldest anarchy server in Minecraft could find a way to grief real life companies
I you used the good fix that doesn't allow any JNDI access.
For what’s is worth: no, not all Java is under attack, only when it’s used log4j.
I simply can't comprehend the fact that this was discovered with Minecraft.
I'm glad my server is not vulnerable to log4j
Over my head. But sounds serious! I’m concerned for my storage access over internet!
Thanks 🙂
Hmmm...
👍👍👍Ansible automated update - that's would be an interesting video tutorial 😉👌👀👀
Keep your Minecraft version at 1.8.9 or higher when using online
thanks i am now very stressed
Never Forget
Did you really have to put an ad in the short?
Jeff, how does compromising a process running log4j allow the attacker to gain full control of a server? Surely the process wouldn't be running as root? A more detailed follow-up covering this step would be interesting.
If your Java processes are isolated and not able to access other things on the server, then it would narrow the scope of what could be done on the server, but basically, anything the running application can do (most Java apps are deployed with superuser-level access), the exploit could control.
Since Java isn't the most popular thing to containerize or jail off, though, and many many legacy applications use Java, it's probably more rare to have that true process isolation.
And this is why I don't think a toaster needs a computer, how many people do you really think are going to update it or even know to update it? I doubt many people even a year later have even heard of log4shell.
We are slowly migrating from manual management to ansible. And guess what wasn't there yet, exactly, the system to update this. So I spend 6 hours friday night patching/mitigating ;) Automation helps, a lot.
I bet the priority on automating that system just got bumped :)
I wonder if Geerlinguy had an ansible playbook he could share that identifies hosts that run log4j in the first place?
Rather than patching affected hosts identifying vulnerable ones might be a bigger issue than you think.
I am not talking about those which run java code from say some jboss app server that was installed from e.g. rpms of an official repo, but what about those that were surreptitiously infiltrated by a log4j lib from some java app that you haven't even been aware of that it existed on the host, let alone that it was using this kind of logging mechanism?
P.S. ok one could always run a find over every mounted local file system but maybe there are more efficient or clever ways?
It's tough because you can't just identify a java app and mark the server as vulnerable. There are a _lot_ of conditions that must be met to positively identify an application as being vulnerable, but the easiest thing would be to see if any log4j libraries are in the app, and if so, what version the jarfiles are.
@@JeffGeerling
Thank you for your reply, Jeff.
The last couple of days since the log4j CVE popped up my colleagues have been scouring through our many hosts for occurences of log4j usage with various scripts they had tinkered up or were given by colleagues from other departments, some of which initially only looked for file names which I think are inadequate.
That was why I asked how you devised a more reliable method.
Do you think it is necessary to unjar the archives and check their contents for suspicious libraries?
Thanks for the tip sir.
Who wants to take down a java empire 😊
Java vulnerabilities are actually very old...VBE and JAVA libraries on windows are notorious for being full of bugs which give space to backdoors.
They're not even useful and the system is bloated with them without any reason whatsoever.
I shred them all.
Thanks.
Smart word is not so smart.
I love you Jeff
I think this vulnerability highlighted something that the IT world has been trying to do for over a decade at this point... get away from Java!
60% still use jave 8 lol got luck.
Hold the phone horizontally, it’s 2021
The world needs to be patched
Java needs to be patched out of existence.
How did you fix it using Ansible PlayBooks?
In my case it was simple enough, since my Solr applications were all running in Docker containers. I updated the Docker image with the mitigation, then the Ansible playbook pulled the latest image, then restarted all affected containers using Docker Compose. I already have an inventory set up for all the servers so the playbook was pretty easy to write up.
it would be very nice if you could setup a small demo...i have seen so many sugar coated tablet for babies to feed but not some intermediate (not advanced) level of explanation
If you're running Java then you had it coming..
Slf4j is pretty great.
Jeff - you point at asustor divice! Is this an issue for these devices? I have one - should I panic?
Not as far as I can tell. They just happened to be behind me :)
Only Java apps are affected, and I don't have anything running on my NAS that uses Java.
@@JeffGeerling Oh yes - it can have dockers or other apps running using java, but I don't think I have. Hopefully then mine is also safe then 🤞 Thanks! Panik avoided then 😁
Another reason to never use a crappy language like java.
I tried updating my cat but he won't let me
JavaCat
Have some pity for old infra admins like me who have to maintain several OLD versions of java. Java versions 6 - 8. Nothing new from this century at least.
Despite my hatred for all-things java - WHAT even needs to be updated? I doubt we're affected -- we run everything as the "oracle" user, which is non-privileged -- but we do run a TON or Oracle apps, all of which include log4j.
It's a bit of a tangled mess, but at least if you're running older 1.x versions of Log4J, some of them are probably unaffected.
You been vulnerable for over 10 months….
What's log4shell? It's one of worst exploits in a while. The internet is on fire.
Can we please stop posting shorts in this vertical orientation? How did this become a thing anyway? Everyone is doing it....I can't stand those bars.
Watch on mobile like it was meant for..?
*laughs in Nginx*
I'm sure that nginx will have its turn in the 0-day-vulnerability spotlight soon enough...
@@Steamrick when that happens I'll be laughing in Caddy lol I'll be migrating soon :)
Minecraft servers patched most of it
I was hoping for an AWS outage video. I think you can sell a lot of “It was DNS” shirts!!!
Heh, I don't have enough extra information to make an intelligent video on it.
@@JeffGeerling I just wanted to say “It was DNS” 😂
You sounds interesting 😀
Or don't use java
quick, update to windows 11
Thanks.