VFX Artists DEBUNK CSi "ENHANCE" Effects
Vložit
- čas přidán 14. 05. 2022
- Brought to you by Raycon. Go to buyraycon.com/corridorcrew for 15% off your order!
Join OUR Exclusive Streaming Service ► www.corridordigital.com/
THIS EPISODE ► Wren and Niko debunk some classic photo enhancement scenes from Hollywood shows and movies. Can modern technology even compare?
SUPPORT ►
Join Our Website: bit.ly/Crew_Membership
Instagram: bit.ly/_Corridor_Instagram
Twitter: bit.ly/_Corridor_Twitter
Buy Merch: bit.ly/Corridor_Store
OUR GEAR, SOFTWARE & PARTNERS ►
Our Go-To Gear: bhpho.to/3r0wEnt
Puget Systems Computers: bit.ly/PC_Puget_Workstations
ActionVFX: bit.ly/TheBest_ActionVFX
Lighting by Aputure: bit.ly/CORRIDOR_LIGHTS
Cinema4D: bit.ly/Try_Cinema4D
Insydium: bit.ly/Insydium_Plugins
Octane Render by OTOY: bit.ly/Octane_Wrender
Boris FX - Mocha, Sapphire & Continuum: bit.ly/2Y0XLUX
Motion Captured with Xsens Suit: bit.ly/Xsens_MoCap_Suit
Reallusion: corridor.video/Reallusion_3Ds...
Unreal MegaGrant: bit.ly/Unreal_MegaGrant
MUSIC ►
Epidemic: bit.ly/Corridor_Music click this link for a free month! - Zábava
The trick is that the cameras just take 8K photos and they only have 1 team member that knows where the Zoom button is.
JUST 8k? XD
@Steven Shorts 🅥 no
8K is only 33 megapixels lol
@@Velgeiss bsd
800k maybe.
I love the simpons joke where Barts on a computer, says 'zoom in and enhance' and Lisa just shrugs and pushes barts head closer to the screen.
Which episode was that from?
@@lightcycle87 I think it's when Bart was trying to find out what made skinner such a boring square, so it's the episode where Bart finds a prankster who put worms into a swimming pool.
I could Google the episode name, but I'd guess its called 'pranks and all' or something like that.
Y'all should do "Sound Designers React!"
Like: How they made the T-Rex or Godzilla roar; How they make body sounds for creatures that don't even exist; Even the subtle ways they have to add basic clothing, movement, and walking sounds into animated films.
You know those punching sounds in old action movies? WHAT IS THAT?
great! as a recording engineer i would love to see this!
Foley artists. There are a few CZcams vids about what they do.
We need that series considering the fact that I've just came to know that the dolphin sounds use to censor slurs was actually the sound of a kookaburra in faster speed.
Like the Squidward Walking sound
Oh I would love if they did that. I remember there was a tutorial they did once on punching sound effects, but I'd love to see a bunch of audio stuff broken down.
I have had better results with Topaz Video Enhance AI, because that does not only use the one pixel, but takes into account how it changes over time. It's a bit like image stacking.
That's what I was thinking. If you have one-second 24 fps video, 1 frame is only 1/24th of your available data. Use it all!
It doesn't help that they were also doing this with a 2D 'mask' instead of an actual 3D face.
I was like screaming use the plate number enhancement tool, it tracks the plane and stacks with opacity bringing the details up
This would be a very cool followup!
yes please that sounds like it has a lot of potential!
There was a French canadian show where they parody this. From security footage, they zoomed in a client's sweat drop, and now had a 360 view of the store, and zoomed toward the door where the suspect was exiting, then zoomed in his hand, he was holding a credit card, then zoomed in to read the guy's name. But that CSI eye reflection tops it all because it was intended to be serious.
Are you sure you're not thinking of Red Dwarf? (Which was British)
@@jsl151850b No, the Red Dwarf scene was "uncrop, forehead, through a window at other end of street, a doorknob, a faucet, out the window again, a car door, a building's glass door.
Video ID 2aINa6tg3fo
what's the name? I'd love to see that lol
I’m dead bro lolololol
What's the name of the show?
You guys should make a CSI video where the characters try to enhance images with the most ridiculous results
I second this!
i would love that!
also you have a lovely hat.
This is a totally awesome idea. I want to see it happen now!
Love this idea!
yess
A tv show that really took the whole enhance trope to the extreme was one episode of Red Dwarf, where they were zooming and enhaceing through reflections in reflections several times over and each time was just as clear and crisp as the first.
Un-crop!
😂😂😂🚑
Red Dwarf was a comedy, so that just shows how long that's been a joke...
Unless that joke was in the revival series?
I love how they always have the tech furiously click-clacking away at the keyboard for these enhance sequences, as if every calculation by the computer required a key press
Because simply clicking one '+' to zoom in would make a boring show :P
@@Shade01982 I would argue that using this tired old cliche scene (wait what’s that? Zoom and enhance!) to advance the plot already makes for a boring show
@@matthewwright2524 True...
That's how brilliant the technician is. He's programming ai in real time
The technology in CSI is simply advanced alien technology that humanity hasn’t even come close to replicating yet.
Maybe we already have and they're just holding back on the public?
@@ROBOHOLIC1 if so, we wouldn't have so much unsolved murderers and criminals.
Fourth
.
So you’re just ignoring when they actually enhance?
When they were looking at license plates, that’s the technology needed. It is just based on numbers because we don’t have the processing power to be able to enhance a hat.
I'd like to imagine the CSI crew just has access to an advanced neural net with centuries of training, just vibing underneath their building. Catching criminals is the only way to satisfy its bloodlust
Maybe CSI is in the same universe as Person of Interest.
A little bit like SIBYL in Psycho-Pass? xD
@@taylankammer
Yeah...
Honestly, that's a Netflix anime adaptation I'd actually watch. CSI but it's in the same universe as Psycho Pass
Even such net wouldn't be able to recreate whole picture from one pixel.
We saw this unfold with the Kenosha self defense case. Prosecution was trying to submit a video as evidence that had been enhanced by software to make an argument. Luckily that argument wasn’t even clear with the enhancement, but it could be potentially dangerous using enhanced video as admissible evidence in court.
Enhance! Great video. I’m actually blown away by how much you’re all able to do. I remember using genuine fractals ten years ago, it’s come a long ways with AI.
It really has! the licence plate demo straight up blew my mind with how the AI took basically nothing, just incomprehensible smudges, and figured out what it said.
Out of darts comments : 7 likes
How
@@justleo5997 The license plate demo didn't use AI (5:18 - 5:35), Niko explicitly points out how that was just mathematical algorithms. If the application did use AI there would be no way to guarantee what was real information and what was generated information. Given a solid enough model and the correct type of AI algo it could be sort of possible to do this with a high enough confidence to have this be admissible evidence, but I wouldn't trust generative AI to be able to do that, maybe a neural net that is well tuned but in general nothing is better than just solid math.
Not everything is AI, software and math have been doing these things incredibly well for a long time....
I guess the Mark Ruffalo thing was impressive as long as you discount that the team was primed with information about the image, and then refined the generation with the additional information. This would mean that if used in a real court, not only would a eye-witness testimony (which at this point DNA evidence and project innocence have proven eye witness testimony is nearly never reliable) be a part of this, but the results would be dependent on the type of model, the input parameters, and the model training to perform the enhancement of the "Evidence". AI is a great tool for some use cases, but its not a panacea, and its certainly not something that should be trusted with matters of fact.
There was a case I remember reading about where the investigator had a blurry image of a license plate but he also had precise locations of the lights and the camera (it was at night) and geometry for the car and street. So he recreated the scene on a computer and rendered it many, many times. Each time he used a different license plate number trying to match the pixels in the blurry original with the pixels in the blurry render. He ended up with a few matching license plate numbers (fewer than 10 I think), which was enough to find the car and solve the case. IIRC this was the first time this technique was used.
Seems like you guys are well qualified to do something similar.
this is the same reason why blurring or mosaic-ing text is not enough to obfuscate it since you can just iterate through blurred text and see what matches
Mans was on the grindset
Man the police really go to the nines to catch those speeders.
That sounds like some reverse logic and should not be allowed in court
@@rp9674 I think the principle is that it was able to narrow down a list of potential suspects who could then be investigated and one of the suspects panned out. So the license plate enhancement couldn't be used to convict the person as it was inconclusive, but the evidence from the investigate that followed could.
Now it would be in error to assume that one of that group of suspects must be guilty and that the most plausible suspect was in fact guilty because the pool of created suspects might not include the guilty party.
But it would not be in error to investigate and see if you could find the guilty person among them with more conclusive evidence. Like maybe you eliminate the vast majority who had great alibis, then focus on the remaining few.
The Klingon “Enhance” blew up the Enterprise D in “Star Trek: Generations” and I was like “If Klingons built computers; they’d be like Ataris”
And the pieces of the enterprise fell back down to earth in the form of “Chocolate rain”
You may have stayed dry but the others felt the pain
They literally build interstellar spaceships. I think they can go a bit better than Atari level computers.
RLM recently brought this up if I recall correctly. It is fun to imagine how Klingon culture deals with scientists and nerds.
They've got starships and all, but...how?
@@FangsFirst they probably have a planet in the empire that is tech support with a conquered people on it. "Welcome to Zainar Technologies how can I help", "Tell me how to make the view screen work WORM", "So I understand you have a problem with the view screen, is this a Brell or a K'tinga?", "THIS IS A VORCHA P'TAK", "Oooh sorry, the Vorcha uses an M117 Digiplex Viewscreen, you want a different support line. Please demonstrate your honor by holding." *music starts playing*. Klingon Captain: "I'm not dishonorable, I will wait through your music. *taps foot*"
To be fair, it wasn't the Klingons that pulled it off. Soran was the one smart enough to bug Geordi's visor. The Klingons just had to watch the footage.
One could argue that the CSi "Enhance" feature exists because the team has huge arrays of sensors lodged in every crevice of everywhere in the world, and the Enhance feature simply loads in data from those sensors and displays it in a way that is consistent with the viewing angle of the footage, or in some cases, completely inconsistent with the viewing angle for a better view.
so, google street view
Exactly it feels plausible in a "big brother" surveillance state
@@SOLIDSNAKE. No, it really doesn't. It kinda would make sense in futuristic scenes like Star Trek or something to that effect, but a scene that takes place in current day, no. Even assuming all those devices are even aimed at what you want to see, the majority of those are not going to be of any better quality than what you already have. The resolution is not going to get any better. You know just have two blurry images instead of one.
not sure if that'd explain why the image of "Nuevitas" was flipped vertically
clearly, they thought - well, words appear backwards in a mirror, so we'll flip the image horizontally,
then rotate it to where we need. That'll be fine, right?
Wren: *does a single thing in after effects*
Wren: "I think I've pushed after effects to it's limits"
that's actullay very accurate
The most impressive thing isn't the enhancement, but the fact the investigator needs to tell the technician to enhance. I mean, if the algorithm always work why even present the blurred image? The image/video app should just enhance by default when zooming in
What if the reason they don't enhance immediately is because the computing power is so much, every time you press enhance it costs thousand of dollars lol
I can imagine if such a thing is even possible it would require using some super computers to completely recreate things
The enhance feature only works in the presence of a trained technician.
A goat must be blood sacrificed every-time they enhance, so they try not to do it too much (blood sacrifice is the only logical answer for there enhance "algorithm")
@@durza7173 a trained technician guided by an authority figure, of course
They are actors, they love to have lines to speak however silly they might be.
Several years ago in the UK a driver was charged for speeding caught by a camera. They used computer algorithm to read the plate and that was submitted as evidence. The defence asked for a copy of the original picture where the plate was completely unreadable. The defence argued the evidence was generated by a computer algorithm and who can say how accurate it was.
The defence won.
The only way you could use an enhanced image like that would be to find a vehicle (or person) and then find additional REAL evidence to use in court.
@@angelbear_og indeed and with speeding, usually the photo is the only evidence.
was it an AI algorithm adding detail, or simple math translations?
"But . . . they did it on CSI!"
I think prosecutors are hoping for jurors who don't know a thing about image enhancement. I'm glad the defense saw through the state's BS.
Ah, the idiocy of archaic courts.
Appreciate the emphasis that AI upscaling invents details from nothing, it doesn't reconstruct real ones. Too often people have the misconception that the upscaled image is supposed to be accurate in any way, shape or form. Imagine the damage if a tech-illiterate judge accepted something like this as evidence
It apparently happened in the Kyle Ryttenhouse trial
I mean, your brain does the same thing for what you're actually looking at in real life, so this is really just semantics. The AI definitely does *not* invent details from nothing. It reconstructs (yes, reconstructs) detail from context. The real question is how accurate the AI is at reconstructing this detail. That VFX artists don't fully understand upscaling AI is not surprising.
@@skipfred the difference between inventing and reconstructing (the way you're using the word) in this context is just semantics. You can't use AI to prove or even indicate what was there irl
@@soundninja99 The way I'm using them is exactly how OP used them, so yes it is semantics, that was my point, but it's not a semantic argument because we're agreeing on OP's original usage. Saying that you can't use AI to prove or indicate what was there is like saying that you can't use blood splatters to prove or indicate that a stabbing occurred. Yes, you very obviously can because of *context*. The question is whether or not the investigator (or the AI) is capable and reliable, not whether it's technically feasible.
@@skipfred fine, it can be used to indicate what was there to the same degree as some investiigator staring at it and guessing what was there
You did a good job demonstrating how predictive AI should not be court admissible. It is the equivalent to an artist's rendition from a blurry image.
A suggestion for another vfx relate movie trope:
How satellites are able to see someone suspiciously drinking coffee from space.
Well, google maps and other civilian services provide satellite photos that are clear enough you can sometimes spot individuals on them. Stuff not for public access like government and military sats should be of even higher fidelity, so it's not particularly impossible.
while not satellites..... there are actual military drones these days that can do that (UAV miltary drones that fly around cloud height, some use up to 500 separate sensors similar to a smartphone and than record and send that feed to a technician in real time). the key difference with high tech miltray drones is they are equivalent to 2500 combined megapixles.......for perspective 4k video is 8 megapixels
@@scottmantooth8785 maybe because it doesn't exist? Even it existed it should've died from old age
@@DarthBiomech Google map doesn't use satellite data for the most detailed zoom levels. They use photos from planes overflying areas, which is also why the maximum detail level can vary from area to area, since it depends on whether they have airplane photos from each area.
@@Carewolf Commercial satellite resolution used to be limited to 50cm/pixel by U.S. law until 2014 - then they allowed up to 30cm/px native resolution, however some companies started offering even better 'synthetic' resolution - basically A.I. upscaled. Best I've seen is 15cm/px by Maxar.
Spy satellites are said to have resolution up to 5cm/px which is still not enough to read license plates or recognize faces.
It's much easier, cheaper and far less regulated to achieve higher resolutions using planes or drones.
I used to work in this area, so here's some insight: The rule of thumb is that any image clarification needs to be performed in a way that can be reproduced by a third party using the same tools. and by "reproduced" I mean each step in the process needs to produce identical md5 or sha hash values. Some people may be using AI, but I wouldn't touch that stuff.
Is this so that it is verifiable in court? Really brings up an interesting topic in the near future as more and more black box algorithms are used to run things.
@@philipfahy9658 Yeah, basically. Black boxes can be a nightmare. As a technician, I documented my process as best I could. If opposing council has an issue with my results from a black box, they may just have to bring it up with whoever created it.
@@peterehowell Yeah and it gets worrying when you realize how poorly most of our politicians understand the internet. I can only imagine what level of familiarity or lack thereof different public institutions have with this technology.
If Kyle Rittenhouse trial is representative of your average trial then the CSI one is closer to reality where the prosecution just phtoshops shit in to fake evidence.
@@XAJUSS
And yet it was still allowed in.
Later we had the HD recording leaked that showed that the "reconstruction" is bullshit, they cropped, reduced the resolution and sharpened it up to make sure you cant tell whats going on.
But yeah, the prosecution was awful, and showed how corrupt the legal system is. My favorite part was when the prosecutor pointed a gun at the jury with finger on the trigger while talking about gun safety, peak comedy ...
The even more ridiculous tech CSI had was when they would magically have recreations of the crime scenes in 3D with animation to go with it. I think with Lidar scans you could get close to mapping the room but all the animation stuff that happened seemingly automatically was ridiculous. As an animator myself I always laughed at that stuff.
Would you be able to do photogrammetry with multiple security cameras? I know that there's some pretty good software for it nowadays
@@xWood4000 Yes, in fact you can. You get a blurry mess as an animation but you can. Look for radiohead-house of cards for an example.
Closest tech to this now is NeRF i guess. They generate 3d model and textures from one photo
We laugh at it, until we make it real :)
@@xWood4000 actually in batman arkham origins they do make a holoprojeciton of the crime scene and you have to connect the evidence to fill in missing data, is pretty cool
Here's an idea: recreate "zooming into someone's eyeball to reveal the suspect in the reflection", but using 3d software and 3d scans of your heads. Position the head models so that when the 3d camera uses some ridiculously high focal length, you can actually see the reflection off the eyeball.
You can do that with good DSLR, no need for 3d modeling.
As a graphics artist at a TV station, I want to say THANK YOU for doing this video! Cops bring video to us all the time, and my producers are always saying, "just zoom in and enhance this"
The only "Zoom and Enhance" that I ever let slide was in the Loki show. The pocket computers of the TVA had a pretty low resolution display, but had very high resolution surveillance footage. In that instance it made sense that you could scale a part of the image up without losing detail.
I was willing to allow _Blade Runner_ as well (which appears in here).
Fancy super future technology where replicants exist? Sure, why not.
@@FangsFirst That’s a very fair point!
There was a TV show called I think Continuum, where the lady was from 200 years in the future and her nanosuit thing accessed the security cameras of every business downtown and created a 3d reconstruction of a vehicle moving from their footage and in cases like that, yeah just go with it.
For the TVA, they can do whatever they want, they have Time Magic
some pornbot stole your comment, so i subscribed to you
The original comment
I watched Enemy of the State a couple weeks ago and that scene caught my eye, something they did actually added to the legitimacy, which was they called out that they weren't able to look "inside" the bag, and what they found "could just be a shadow" or "the bag shifted". I liked that they tried to ground it in reality somewhat by giving it limitations.
You guys should've looked at an early classic "Enhance" scene from the original Blade Runner (1982). As I recall, Decker scans a polaroid, zooms in on a tiny, out-of-focus bathroom mirror in the background, fixes the focus, adds detail, and then rotates to see a label on a medicine bottle that WOULD be reflected in the mirror if the photo were taken from a different angle.
I would argue Blade Runner gets a pass because it's not a Polaroid, it's supposed to be a science-fiction-y holographic photograph with unspecified 3D-type features built into it, which the machine he inserts it into can manipulate. In the movie, he adjusts the perspective of the camera such that he sees around the frame of a doorway to reveal what's behind it, and I suppose in any case that's pretty implausible. But at least they make it clear this is all some technology that's foreign to us, something in the future that doesn't currently exist. CSI doesn't have this excuse.
@@mikekohary1075 That's actually pretty good.
The mark ruffalo thing could benefit from some temporal solver. You know, combining data from 30 frames instead of one. Is there any AI-solvers that could handle that?
I was thinking the same thing. That would provide a lot more "detail" (or at least "data") to work with.
@@justaviking yeah! Astronomy programs can stack frames like that, so that’s an obvious way to be more ”state of art” if implemented well
Wouldn't that just reinforce the blocky, pixelated features? Give the AI 30 frames of black holes for eyes and over exposed face and it will just assume that's what it's supposed to look like and not correct it.
@@radioethiopiate9327 It's how jpg handles brightness, combining some greyscale preset images (quantization table, technically) to generate a more detailed composition, the only issue for this usage case is nailing the alignment, but I've manually done this in the past and athought still very soft, figures can become way more recognizable.
I saw some tiktok about something like this where a guy took a video of a spinning record and then took the average of all the frames in photoshop or smth and it was suddenly really readable
The "enhance" effect is one of my BIGGEST pet peeves in movies/TV shows and it makes me so happy you guys finally debunked it.
In order to zoom in on tiny objects on security footage like they show in the movies, there camera resolution would have to be extremely high to pick out those details.
Mine is the lack of marksmanship by law enforcement officers that have supposedly spent years training with firearms missing shots that I know I could make cold even with my limited ammo supply and range time.
As a programmer, the 1000 word-per-minute typing in a lime green command console to hack into the FBI in 10 seconds tips it for me.
@@TheMightyHams sounds like you're just not good enough at your job
@@phosphatepod Very true, I'll download some more RAM, switch my editor to green highlights, and deploy some swanky UI loading bars with flashing red warning labels. FBI will stand no chance.
There was an episode of CSI: Cyber where they tracked a guy down by finding his phone. Apparently when he touched the screen, it left a "digital fingerprint" made out of code that they could find, reconstruct, and identify the person by. Like the phone screen can randomly copy fingerprints onto the memory by any touch. Then the technique was never used again in the show. Couldn't keep watching after that.
It sounds like a rather extreme extrapolation of keystroke recognition, the existing software that can capture and individualise people's typing patterns?? 😆
My phone screen does have a fingerprint reader built into it, so this is not entirely ridiculous. A lot of phones do.
i bet this would be a great tool for enhancing older aerial photography. I work in archaeology and really want to try this for locating old buildings
The car numberplate goes way beyond sharpening, it's done with deconvolution. It's been in the Wolfram Language (Mathematica) for a long time. It's not as easy as you'll see it described as because slight errors in the lens shape, or rounding in the calculation etc… blow up into a huge mess really easily.
They didn't call it sharpening? They called it deblurring. And they were referring to motion blur.
@@plzletmebefrankthey called it both sharpening and motion blur, you can use deconvolution to remove both motion and disc blur. The ripples around the edge are an indicator.
@@alan2here Fair. They are probably just using terms people like me can understand though.
It's definitely more complicated than they make it out to be, but the software hides a lot of that complexity within itself.
That's something I hadn't thought about! If you're talking about trying to flatten an angled image (as per the second license plate example in this vid), does the type of lens used in the original camera make a difference? Like, do you need to know what kind it is so you can calculate how much it may be distorting the image? Or can the software just work out all of that?
One of my favorite "zoom and enhance" scenes was from Gotham. Gordon was looking at a tiny TV wired to a crappy security camera. The suspect was wearing his work shirt, but the logo was blurry. So Gordon grabs his partner's glasses, and holds them up like a magnifying glass. Bloom! Crystal clear!
You serious?
lmfao
@@kappadarwin9476 yeah, it was great! That's why Gordon is the best! He's a magician
this was an awesome video that was actually really interesting. it also felt like you guys said, 'hey we need to do a video. lets do that simple idea we had' and sat in some chairs and killed it.
16:19 I legit just did the voice right before that Team America clip came up. Good job editor
I know this isn’t a “react” video, but I do like how Wren is genuinely reacting to CGI stuff. That being said, what doesn’t Wren react to? 😂 love you guys
@Don't read profile photo I read your mind, you sick puppy!!!!
Unfortunately the OneWheel scandal...
Everybody needs a wren in their life. He is always so interested and energetic about everything. I love it.
you can be your own Wren
@@holliswilliams8426 not like wren. Wren is a special breed 😇
For ten minutes, then you just want to lock him in the fridge for some peace and quiet.
?acting
The human Labrador Retriever
The part in every csi/fbi/ncis/etc show when they use nonexistent tech to "enhance" or run facial rec on hopelessly blurred photos always makes me laugh!! And THE most implausible bit is where do these agencies get the budget for this stuff, the size of the monitors alone would bankrupt most offices!! The show "Bones" was the most unbelievable in it's over the top tech 🤣🤣🤣
9:59 ... I do gig work for a photography studio that prepares memorial portraits for funeral homes. We get sent some of the worst post card size (sometimes just a drivers license) family snapshots and are tasked with turning them into poster sized portraits worthy of a memorial service (pouty mouthed selfies are the worst imo).. Topaz puts out some great tools to help, the studio makes much use of them especially Gigapixel!
Please Do Muppets From Space! Would love to see you break down those effects. Especially the Noah scene, the Gonzo getting a message from Space scene, the invisibility spray scene, and the spaceship scene at the end! Thanks!
Thoughts while watching the video: The potion seller inverse distortion reminded me of a case where someone had censored their face using the "swirl" filter in some software and it was easily reversed by applying the swirl in the opposite direction.
Second: You can do some limited superresolution if you consider the movement of the camera or the subject in front of the camera because if the subject moves by a sub-pixel amount, it can theoretically give you more information than a perfectly static image. I remember using filters like that like over a decade ago. The results aren't mindblowing, but they're pretty good. I think the software was Anti Lamenessing Engine, after a short websearch.
Third: I remember seeing people dunking on some magic upscaling algorithm after using a low resolution photo of Barack Obama as the input and the algorithm outputted some white guy.
Yeah, I'm pretty sure you could get a much better image of Mark Ruffalo if you took the raw footage, then motion tracked the face and stabilised it accounting for distortion, then superimposed all of the frames and averaged them out. If you think about it it's the same idea as the motion blur number plate example Niko was showing, except spread out frame by frame in the video. If you took all the frames and composed them together you can get more data out of it.
I was thinking exactly the same thing. I was half expecting the swirl face case to come up in the video. It was actually a nice case of karma. The guy in question was a prolific producer or trader of child pornography and his online avatar image was his face made unrecognisable - he assumed - by that swirl algorithm. The police 'unswirled' the image and then found and arrested him. His name is Christopher Paul Neil and he even has a wikipedia page if you're interested in the story.
@@bucklogos That's the type of thing intelligence agencies were doing years ago with specialized software that could provide greater resolution than what the camera alone could provide. One guy who had done work on satellites, though limited in what he could say, pointed out that while a camera in orbit taking a picture had a certain minimal resolution, taking _two_ images in rapid succession could, hypothetically and with the right processing, be functionally the equivalent of taking a single image with the optics of the camera being the size of how far the satellite had traveled between images, thus vastly greater resolution.
@@bucklogos I agree, stacking multiple frames from video should give additional information (after correction for changes in angle). Astronomy software such as Registax does something similar
The
Would love for you to break down scenes where characters 'hack into the mainframe', or take over systems. Usually its verbal garbage with visual garbage so would be awesome to see them break it down!
A gigabite of RAM should do the trick
Matrix Reloaded is one of the few movies to depict hacking accurately.
NCIS has a pretty good one where 2 characters to counter-hack the mainframe by pressing random keys on the same keyboard. Like they each take one side of the keyboard, and then another character walks in and unplugs the computer. It goes from complete fantasy to the easiest and most realistic solution in a single scene and its fucking hilarious.
Possibly my favorite video you guys have done. This has long been my biggest pet peeve with CSI in particular, but I remember shouting angrily when they pulled it in X-Files.
During one Imaging Science workshop while I was getting my first forensics degree we were tasked with enhancing a photo to get a licence plate readable. It was a great deal of messing with sliders and asking my friend who was next to me: "Hey, does it look better like this, or" [moves slider slightly] "like this?"
"Uhh... first one maybe?"
"Thought so too. But they're both awful."
"Yup."
Sounds like those differential lens tests the optometrist always does when one is getting fitted for a new pair of glasses! 😂 "Is it better like this....? Or like this?"
You guys should make the most ridiculous unrealistic "enhance" scene you can as a sketch
The victim was standing in front of the mirror. We can't see the perp, but we can partially see the rest of the room in the image. What's that? Enhance the victim's eye in the mirror. It's a fishbowl. Apply inverse focal length and water light refraction. Zoom in on the fish's eye. It's our guy! It was the butler all along!
@@mac1991seth lol, exactly 😂
There was a scene like that in an episode of Red Dwarf.
@@mrcydonia "Uncrop!"
Well they kind of already did it was in blade runner the original one
"You see that reflection?"
"Yeah, sure do."
"Zoom and enhance."
*Swivels chair*
"Sir, i can do exactly one of those things and it's not the useful one."
This was really interesting and it would be fun to see the team tackle other similar tropes that has infested a bunch of shows.
"Is there police departments using this kind of tech" yes, a fair amount I reckon and fairly advanced too. Some students from the computer graphics major at my uni worked on a project requested by the police to generate 3D heads from mugshots to facilitate identification on videosurveillance footage. Basically they would position the 3D head at the same angle that the camera is watching the real person for comparison and that would help them decide wether or not it is the person in the video they are looking for.
17:37 the way Wren said that is go great😂
I remember seeing papers or technical demos on "enhancing" / de-blurring video frames by looking at the frames as a set, rather than individually. That can produce even more detail. Essentially, the lens and camera is constant so small movements result in features being pixelated in a variety of ways, which in turn provides sub-pixel level information.
More detail could definitely be drawn from the footage this way if we had the ideal algorithm.
The Google Pixel’s camera does this to enhance its clarity. It will take a quick set of photos and combines the info captured in them to allow an incredibly impressive improvement to its digital zoom. Our eyes actually do the same thing. If look closely a focused eye doesn’t stay still, it has these microjitters essentially using a “set of frames” from slightly different offsets to get as much information as possible from it sensor cells.
I believe this is how someone was doing some...DS9 remasters, I think it was? They looked pretty dang good.
Amateur Astronomic Photographers use this method to get ultra sharp photos of the Moon and planets.
Dozens of photos superimposed.
This was fantastic. You should look into stuff like the mission impossible scene where they use a screen and a camera in a hallway to fool the guard.
I really enjoyed this one, I always thought the enhance thing was too good to be true, but you guys almost did it! 🤯
"Enhance" and "Compensate", two of the most magical words in fiction.
Coincidentally, also the most common words on gas station boner supplement packaging
Anybody that used VR goggles to try to read small text might understand this: If you look at the text with no head movement it's a blurry, unreadable mess. If you move your head in a circle, the letters become visible BECAUSE you're actually moving the physical pixels in the headset around "fixed" digital image where what they're averaging is changing letting your brain have more info to interpret.
That's also why a good trick when watching videos in VR headsets is to set the video to somewhat larger size than your field of view (FOV), such as have the video's vertical size match your vertical FOV. This makes you want to look left and right more to see the video giving more virtual information than the raw pixel count would suggest.
That first paragraph described what temporal upscaling is essentially. Just done within your brain. It's the most high level straight upscaling method we have, reconstruction is the hard part to tune for a clear image though.
A real world analog might be riding in a car past a slat fence and being able to see what's going on on the other side almost as I'd there was nothing in the way. But if you stop suddenly you can only see through the slits. The human brain is great at averaging information over time I to a synthetic image.
9:30 Lmao he roasted AE so subtly
I too use After Effects as my go-to photo editing program lol
I love the way that Red Dwarf did it: scan a photograph, uncrop the image, zoom in on the detail that was cropped out of the original image, zoom in on a piece of jewelry, enhance the reflection to get an image of the street, zoom in on a lamp post, look for piss droplets from a dog, zoom in, enhance the reflection off the piss, zoom in on a window, enhance reflection again, flip image horizontally, zoom and rotate, boom: the suspect's business address, which was in the phone book the whole time.
I have watched RD many times, but don't remember this. Which episode was it in?
@@lotuselansteve Red Dwarf: Back to Earth Episode 2
I saw a really interesting technique for enhancing a while ago - the issue, as they say, is that you can't enhance something where the data just doesn't exist, where it's just a couple of pixels.
But the technique I saw used video and processed multiple frames at the same time. The area of interest may only be a couple of pixels wide, but each frame captures those pixels slightly differently. By combining the data from many frames, it's possible to find artefacts far smaller than the static pixels that contain them.
Yup, the whole time I was thinking of this. I don't remember where I saw it but it is definitely possible to accurately calculate sub-pixel details by using the data from multiple frames, if the subject shifts even a fraction of a pixel all the pixels will shift slightly in brightness so the more frames the more detail can be extracted
@@LennertK This is how temporal anti aliasing also works (TAA).
I actually tried this on the footage in the video this afternoon. Unfortunately (not helped by youtube compression) the results after stabilizing and stacking weren't very good. But perhaps with access to the original raw footage it would be better.
I was thinking this same thing the whole time. If you have, say, 3 seconds where a face is visible (and moving) then you have potentially 60-180 different versions of that object. Then instead of training the AI with 'real world' data, you select the clearest frame and train the AI with the other 179 frames. I'm sure it's much more complicated than that but you're no longer using invented data, so more admissable perhaps.
Such a fun video. I'd love to see more like this.
One technique they didn't talk about was image stacking, where you take multiple pictures and average the pixels values; this dramatically lowers grain, and when paired with upscaling this can get some pretty good results. You can't do this with just 1 picture but you can with video because each frame is like a single image which can go into collecting the data to create a more detailed final image.
Another thing you could do with a video is use a super resolution algorithm, that utilizes more than one frame. That way really get an image with higher resolution than the input video.
That's exactly what they did for the license plate enhancement shown in this video.
@@blindleader42 As I understood it they just watched an example video of the software, but didn't use it themselves. Using multi frame super resolution on the CCTV video should yield much clearer results, if you get enough good frames.
1:51 "It's informed by knowledge"
Hmm yes, the floor is made of floor
The Red Dwarf take on the old "enhance" was great. it's called "Red Dwarf Blade Runner Esper Sequence" on youtube.
I am SO glad you guys made a video about this, it was one of my pet peeves in spy/police shows
The "enhance" camera effect in CSi has better quality than the security cameras at the bank.
It's worth mentioning that one of the first instances of "zoom and enhance" was from Blade Runner (1982), where it made sense as far-off futuristic sci-fi technology (even though Blade Runner was set in the then far future of 2019).
Thank you. Now l know l am not the only one.
they are just extrapolating existing film photography, like for example they are already familiar with micro-film where they could zoom in and see a document otherwise unintelligible to an unaided eye.
I’d have loved to see multi-shot techniques applied to the problem. Use a couple of consecutive frames where the subject is still and light-path jitter will allow you to interpolate more pixels.
Am I the only one who noticed that weird AF gavel strike that the judge used at 3:10?
I'm a graphic artist and I can't begin to explain how frustrating it is to have the sales personnel at my job hand me a thumbnail and tell me to "just enhance the image" so it looks good on a 15 foot banner
That happened to me. They asked me to enhance a client’s low res logo. I eventually got it done and they were happy. They asked me how I enhanced it at the end, I told them I just redrew it in illustrator
Been there too. 🤓
as someone who has been asked that before (not an illustrator but, for other things IT related) my response was it'll only work if you can give me the project file. you're out a lot of time or are boned otherwise.
As working in the same kind of profession, I feel your pain... A LOT!
3:35 Look Wren, Black Magic is a great camera but I don't think even it could capture that detail.
the "enhance" meme began with Bladerunner... and that was on a frickin' POLAROID PHOTO!!! Cheers!
For astro photography I"ve used an program which can compose multiple lowres images to a higher resolution one and made more detial visible . Maybe something like that can be also used.
I always heard about the "Enhance!" bit being ridiculous, but I never realized just HOW ridiculous it actually was til you showed these examples. It's worse than the parodies of it!
Y'all should do a review of Robin Williams in Law and Order SVU. He brings up how the AI is guessing details in an enhanced image and gets it removed from court. I think it's in season 9.
I saw a very powerful camera one time that you can take photos from at a great distance and when you zoom into one, itll show every tiny detail on a thing like it were taken from right up close
Potion Seller is an all time classic... glad to see it get some love.
The "Enhance" from Blade Runner maybe COULD work. The reason is that its future-past cyberpunky aesthetic seems to still use some analogue technology, which could include film cameras. Because digital cameras use a limited pixel resolution, "enhancing" requires a lot of extrapolating gaps in information. However, when a picture is captured on film, it is captured with a chemical process which goes down to a molecular level. So as long as the lighting is sufficient, etc, it is actually possible to see extremely tiny details through close analysis.
doesn't he get the image of the woman from extrapolating a mirror I.age of parts if the room you cannot see? :)
Analog photography is limited by film grain
Film grain is small but never on molecular lvl, hence all the ISO standards. Same problem, we cannot control particle size to that degree be it pnp-based sensors or silver precipitate particles. I think record with (gold) particles atm is nano scale or something
Nerd
@@noahduncan5524 good thing nerd is a compliment now
That was the case for me too actually. We have decided to digitize and enhance old family photos, and when scrolling through and zooming in, I have discovered a bearded person face in the air between my family members. Originally it was blurred (out of focus) background made of shadows of a tree leaves. I was scared to shit, lol.
John Candy was the best ... I love how you guys have fun whilst you are enhancing these photos and when you debunk so called paranormal footage from CCTV .. you guys are the best. I love to watch and learn all about this stuff. thanks guy ..
The end conversation reminded me of a tv show. There was some old CSI SVU episode staring Robin Williams where Williams argued that the enhanced image guesses what the zoomed image looked like and therefore can’t be trusted to be accurate.
My favourite enhance scene is in season 1 of Barry. The detective asks the tech guys to enhance footage taken by a lipstick camera, they give each a look, and in the next scene as she's questioning the acting students she holds up the blurriest picture, everyone squints and start debating what it's supposed to look like.
Remember when Binger tried to pass a fake image off in court. We remember. The worst part was that he had the close up image and hid it from trial.
Good to see some content from that case here. Truly one of the trials of the century.
Poor little Binger.
I'd love to see the effects in the Mentalist critiqued!
I was waiting for this!
Actual safety cam footage looks more like if I had a dollar for every pixel here I'd have 3 dollars. 🤣
LOL such a goated video idea, this was so entertaining but that potion seller clip was not expected ahahaha
.
you!
welp, time to watch it again, I guess!
"There's high resolution hidden in that pixely blur!"
10:33 thats a pretty sus door
This was such a cool cool video man
Edit: like a PERFECT video
0:41 To be fair, the Enterprise probably has sensors that register at the Planck scale.
Real talk, the first time I watched Enemy of the State, I actually assumed it was a "mistaken identity" movie because the agency had _framed_ Smith with that obviously fabricated footage. Even tween me was like, "Wait you can't see the reverse of this, they're obviously making it up," so I was waiting the whole runtime for the reveal of why they _really_ wanted to catch him. It was very confusing.
That’s hilarious.
How many movies were ruined by one too-illogical VFX shot? We’ll never know…
when you introduced potion seller, i dropped my food smiled and immediately liked and commented. as always great content.
Yes MORE debunking videos, thanks!!!
Unless I missed it in the video, I only heard one type of upscaling mentioned. There is upscaling that looks at only a single frame, but there is another type of upscaling that looks at the frame(s) before and after. By looking at additional frames, this provides significantly more information for upscaling.
I mean, the zoom and enhance is such a movie trope that even movies like "Blade Runner" way back in the 80's used it. In a very cool way btw.
you should have used a temporal solution. Two minute papers has a few videos about it. Some algorithms that can use the data from several frames can get crazy detail back, it's amazing
Hai guys, excellent topic , next time include digital forensics also like , recovering damaged data , creating whole image from pieces of data , location etc... Thanks 👍
Wren telling Niko he solved the case and Niko staying perfectly calm and going "you did? Wow." Was so damn funny.
One thing they could have done with the security camera footage, which would be more complicated but seems like just the thing the Clonix software might also be capable of, is combine several frames to create a slightly more detailed version of the face.
Essentially, this is comparable to what they did to get those images from the black holes with the Event Horizon project.
i thought the same
The problem with mathematically adding resolution is you could still get an image that is not representative of reality in the sense that you would come to the same conclusions if the image was simply taken at the higher resolution.
Trying to figure out who the celebrity was was so fun to watch you should do it again