NVIDIA’s AI: Virtual Worlds, Now 10,000x Faster!
Vložit
- čas přidán 21. 06. 2024
- ❤️ Check out Microsoft Azure AI and try it out for free:
azure.microsoft.com/en-us/sol...
📝 The paper "Factory: Fast Contact for Robotic Assembly" is available here:
sites.google.com/nvidia.com/f...
📝 My paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.at/~zsolnai/
Twitter: / twominutepapers - Věda a technologie
I work in factory automation and this is absolutely a hard problem. I look forward to this solution making it into standard toolsets.
You'er not gonna be working at all my man onces this get here.
But then I can't spend 8 hours a day tightening bolts 😞
@@SIRICKO I'm pretty sure that op doesn't just tighten bolts all day...
I'm a miniature war game player. I hate painting.
I can follow instructions on how to make a robot arm.
What a time to be alive!
I want something more realistic than Factorio or Satisfactory, this seems like a cool game to make.
Those H100 are going to be very cheap after the second AI winter starts very soon when the LLM bubble pops.
Time for cool and useful things to be done with them.
What a time to be alive.
1994: "Playstation released a week ago it is so dope! Can you imagine what we'll have in 2024?"
2024: *SIMULATING* *A* *GAZILLION* *NUTS* *IN* *PARALLEL*
We really went and grew up
we managed to surpass their expectation
so many NUTS
@@FusionC6 Thts what she said
@@FusionC6 deez nuts
You're telling me this thing can plug in a USB correctly on the first try? Uh huh ok pal, lets easy down with all of the sci-fi
this is evidence based. stop your ignorant dreams
Yes, actually someone taught me to look for the EDIT: "seam" on the connector and face that down. It usually means the logo faces up. Regardless once i figured that out, I got them right more often than not
I can do that too...
with usb-c
i heard usb-a are 4 dimensional which is why you gotta flip em over twice
@@Ou8y2k2 edit so it makes more sense. Sorry.
Finally getting down to the real nuts and bolts of AI research!
You didn't screw this analogy, you nailed it!!
What a tool.
@@peanutnutter1This is but a bolt in building autonomous AI agents. The future is gonna be nuts!
This is nuts!
You think you're funny, huh?
Well, guess what!
You are, I laughed.
and bolts!
It is fairly screwy.
They don't screw up a lot. Mostly down.
Dr Zsolnai-Fahér, I don't know whether I've mentioned this before or if it's relevant to your interests in anyway but I am Malawian (southern Africa). I hope you know the impact of your work. These quick updates with clear explanations of topics offer first timers a chance to learn something new without being overwhelmed. thank you for creating this channel and uploading so frequently.
Personally, when I started watching your content I never thought I'd be watching entire Nvidia tech exhibitions and product launches but now I find my self sitting through hours of the launches and enjoy it thoroughly. Thank you and "What a time to be alive!"
What a Time to be alive!
That is absolutely amazing and really made my day. Thank you so much!
@@TwoMinutePapers You're welcome sir! 😄
THIS is the level of physics I want to use in VR games and apps! 🤩
...Why in those? Like, I understand AI training for robots in the real world or for simulating designs for factories, but under what circumstances would you want to hyperaccurately screw on a bolt in a video game?
@@michaelleue7594 Literally any game that requires physics simulations, special in VR.
Of course you have to calculate the trade off between realism, performance and fun, but there is always some niche communities that want to just maximize realism at all costs.
At the moment games would typically solve this situation with a "that's good enough"-style trigger box, and scrubbing through a pre-baked animation.
The excitement here is we could skip this jarring and unimmersive experience, with a much more uniform and intuitive solution.
Also it's getting closer to simulating a more granular and 'realistic' world, which is inherently more immersive.
@@michaelleue7594
@@michaelleue7594 massive melee combat sims!
@@marcoshalberstadt7646 So basically it all boils to down to porn, as always
This will become a metric unit someday. Nuts and bolts per second.
Don't think so, easy to cheat on this particular case
Americans will literaly use anything but the metric system smh
You should see his video where they had a cats per second measure.
And rural areas will still be getting 10 Nbps while rich urban centers will be seeing 1000 Nbps. "I just crash my car a block from home every time, because I can assemble a new one faster than I can find a parking spot." What a time to be alive!
We already have a metric unit for such things: Hertz.
Game engine developers: 👁️👄👁️
Game Engine developers make use of SDF based physics all the time, it's just this level of fidelity is rarely required in games so those clock cycles are used for other things.
@@locinolacolino1302 "is rarely required".
As someone who plays Space Engineers I disagree, clang is my enemy, I wish I could stack 8 GPUs in parallel and increase the simulation to 800hz.
I always thought it was janky because the physics runs at 60fps, but that's too slow.
Increasing the mesh detail decrease the performance exponentially, not linearly, that's why SDF based physics don't do that.
But this paper proposes a solution that doesn't need more detail, so you can actually increase the simulation to 800hz and thus decrease janky.
I think I'm going to make a factory game that's actually physics based instead of just being a logistic simulator. Imagine real items in the conveyor belts, that's going to be fun.
@@monad_tcp in Godot, a simple chair falling lags the whole game, though maybe I'm doing something wrong idk
Do game engines use SDF for collision? I have seen things like Niagara in UE5 use it, but generally collision is done with a collision mesh, or using original geometry (very expensive)
@@voinea12 you can get the godot jolt physics addon which should work much better for most things
No WD-40 simulation?
What does Water Displacement recipe 40 has to do with bolts?
Nah, I get, don't worry.
Now that would actually be a massive paper. Love it!
I tell ya hwat
What a time to be alive.
Factorio 2 is going to be AWESOME
This right here is a prime indicator of what I think is going to be humanity’s biggest innovation over the next decade: an increase in our ability to simulate complex things like this in a super efficient way. Can’t wait to see how this tech scales in the next few years
The best simulation has to be the simulation of football simulants at 0:47 - so accurate!
Accurate if someone dropped some crystal m-e-t-h in their gatorade, maybe...
So the breakthrough was using sdf collision detection?
i think that houdini has some options to use sdf for collisions, but I don't remember it being particularly fast
I had to simulate bolt interactions for VR. It makes much more sense to approximate the bolt. But if you use this example as a way to describe other complex collisions, then sure, valuable.
as a guy who works on this stuff, i can say that complex geometry simulation of the bolt threads is not necessary. as soon as the bolt enters the nut, it is enough to turn them into a single physical object, where there are only 2 interconnected degrees of freedom between the nut and the bolt (rotation with movement along the axis of the bolt).
physical stimulation is needed only at the very edge of the bolt and nut to determine whether the bolt has entered in the right way
And how i am supposed to study the series of events that causing a jamming ?
It's impressively embarrassing that you don't understand why the physical sim is an absolute necessity in this context. Especially seeing as you work in the field, that fact makes it significantly more embarrassing.
What a time to be alive!
Thanks for the content!
Thank You for your video's. They really help me understand Light transport and Physics. I cannot imagine how un-educated on this subject i would be without your efforts. This video made me think of a zillion possibilities. :) Be nice to see that technique for Collison detection in Unreal Engine.
4:19 - Dumbledore plugged in USB cable calmly
So, soooo nice to see new research around SDFs!!! It almost makes me believe that my recently finished thesis involving SDFs wasn't for naught xd
Wow! Extremely cool paper. I was also stoked when you showed The Witness footage, one of my favorite games of all time... do you enjoy it much as well? Seems kinda out of place here, but I'm not complaining.
This is really cool! Is it just kinematically consistent or are they simulating contact forces with friction accurately or are they fudged…
a mass collision system, that doesnt explodes, shakes & runs in real time?! thats just cool, another piece needed for a Holodeck!
HOOOOOOOOOLY, this will be insane...looking forward to implementing this on my own
Could this be used in juntion with physics simulation? IE having a model create an organic structural model, then testing its validity via simulating its ease of assembly? Could be used to make things for machines to put together, rather than human hands.
can be the potential game physics engine revolution as we've been stuck at the lack of a effective collision detection method for games for a while now, these days real time path tracing is pretty much viable, we are only waiting for the same revolution happen to the physics aspect of the game.
Finally useful things that are not related to LLM !
I want to make a factory game that's super realistic in the way things are built, now I can.
What a time to be alive
Once they programmed into the robotic arm "Lefty loosey, righty tighty", they were halfway there really.
"You can even plug in that USB-A adapter in a virtual world accurately; a feat almost impossible to mere mortals." 💀I laughed so hard
The source in the bottom left says 2022, so is this very new or 2 years old? I am confused
Accurate and fast collision detection is especially important in VR Games, as you get really close to the object you are holding in VR. So I really do hope to see an implementation of this in a game engine like Godot
I would use this for simulating agriculture equipment.
I want to make a platform where we can train over robotic configurations in order to develop efficient robots for harvesting berries in fields.
This needs fine detail for the plants but it also has to be a simulation to support weather events and robot damage over accidents, excessive use, normal wear and tear, etc.
The USB joke was gold
Humans learn to correct the alignment of the nut with the bolt by feel, with trial and error on every attempt. A robot could very easily learn to derive the proper thread alignment from a visual analysis of the bolt threads and the nut threads, to start the nut from the ideal position every time.
thank you Ren for letting me know about whatever this is
This is going to be huge for optimising manufacturing processes.
Wow I can't wait for your next video called "NVIDIA's new AI: 50,000x faster Virtual Worlds with Supercharged Ray Tracing!"
The actual optimisations are what we want to know, I'll be pouring over this paper.
Oh wow, factories with machines that produce our stuff for us?
What a time to be alive ! ❤❤
I never thought that a realistic simulation for robot training was particularly hard. If techniques like this go forward robots may actually become useful in day to day tasks. Kinda neat/terryfying.
Whoa! That's nuts! ...and bolts.
wow, I immediately thought of the Bobiverse series of books and being able to design everything in virtual. Until now this was still a huge roadblock.
In a couple papers we could get Open Source factory solution competitions.. build widget x with fewest steps.. or the most :D
and not to far along from deploying the original machine to make all the rest ;)
Accurate factory simulations to cost-effectively optimize factory layout and performance before ordering all the factory equipment and parts, leading to more productive factories and increased production across the economy. The second industrial revolution, cool.
Can this technique translate to video game collision/physics simulations? Is it general enough?
Those gains, if combined with automated enough approach to setup the properties for in-game objects, could result in actually interactive sandboxes and immersive sim games, at last.
craziness - I liked the bowls!
That's just nuts!
Favorite new CZcams channel
This will soon be implemented in videogames, and it will make simulating mechanical things, as well as destruction physics a bajillian times more realistic
I was somewhat expecting that USB joke
Favorite new CZcams Channel
i‘d use it for building houses. i honestly think this is going to be a huge application in the future.
this is potentially game-changing for robotics
i have recetnly thought about SDFs being used for complex geometry! to now see it in action is really cool. there must be some sort of algorithm in place to create SDF functions from any given geometry. cool! SDFs can do much more than what triangles can offer. the geometry is mathematical, it can be infinitesimaly detailed!
Oh man, this would be great to have in Unreal Engine.
This is crazy amazing. At the start of the video I was like this is surely impossible.
maybe an even more elegant solution to problems like this is a combination of the visual rendering of the mechanics for the optical sensors of the machine learning algorithm together with pressure sensors for the algorithm to learn in combination - when i plug in a USB port or tighten a bolt my eyes are rarely what i rely on to know it's on tight, and i'm not blind.
for the calculations of the nut and bolt - why calculate the whole bolt and not the main line of the thread and from there calculate the surface area that will come in friction?
asking wanting to genuinely know
2:21 - Bolt SDF (SDF stands for SIgned Distance Field, btw) visualized as point cloud, Bolt SDF visualized as mesh. Both of these are a mistake for a mechanical part being made to a mesh approximation of a solid, both the source and the cause of all of the collision detection problems that follow. The moment you triangulate and _then_ do collision detection on the triangulated meshes, you are effectively doing collision detection wrong. It doesn't matter whether the resultant mesh is made of quads or nGons or how "perfect" it is topologically speaking, no matter the mesh, it's all triangles in the end and this plethora of triangles is what will fed into the collision detection algorithm(s). Compare and contrast this method with how any CAD application or mechanical application worth its salt, represents mechanical solids like literally nuts and bolts. They are represented as non-destructive solids and surfaces backed by exact mathematical formulas that describe them with a good heap tablespoon of Constructive Solid Geometry (i.e., bool operations like union, intersection, and difference, all performed mathematically and perfectly - try that with a mesh). The moment you map the solid to a triangle based mesh, even for something relatively simple in the grand scheme of things like a bolt, you get a proverbial crap-ton of polygons to consider for collision detection or you cheat and come up with a bolt that looks like it came out of some sort of FPS shooter with hexagons for curves. This is the case, at least, if the object of interest has curves and then requires a considerable amount of mesh geometry to approximate it. It can never be exactly mapped to a mesh, only approximated with greater and greater accuracy at the cost of an explosion in polygons (read, triangles) that are used to form those mesh based approximations. Then you have to deal with that vast sea of triangles when you do your collision detection, making it a very expensive process, even if you take shortcuts and break up triangle groups into volumes, along with other similar techniques used to cut down on the test of "which triangles are colliding, or worse yet, which surfaces are intersecting that cannot intersect, because they are rigid bodies.
it's always interesting to see the kind of things AI struggles with that you wouldn't even think of, and how well AI can do it once you've figured it out.
is this method (physic engine) is something general enough? is it something that could be also used by game industry ? (as you know in video games, physics is very approximative - in general complex visible shapes are approximate with simple convex shapes.
In this video 1:58, I see significant artifacts with convex decomposition. Are there really no good non-approximate convex decomposition algorithms, and is the convex decomposition shown in the video the best researchers currently have? I recently created a convex decomposition algorithm for colliders for a video game destruction system that quickly generates a small number of convex pieces, does not create new triangles and produces no artifacts - the convex pieces perfectly represent the initial mesh's outer shell. Could it be that my algorithm could be useful also in this kind of simulations?
Well, publish it
Is it posible to use spline lines following the peaks and valleys of the geometry to simulate physics?
That's nuts
SIGNED DISTANCE FIELDS?
Valve actually changed the world
I work in aircraft maintenance and since 2015 I knew my job was among the last to lose to automation. While I still think that is true I no longer feel so confident about the time frame.
Wait, I don't understand. Are they still using triangles with the bolt or they are using other stuff?
Instead of triangles they are using SDF (signed distance fields) a form of mathematical calculation of the volume of the object. Its what Unreal Engine 5 uses for good and fast collition detection and other effects
Great video!
Incredible!
"What a time to be alive!" ❤
Esto es el final!
This is nuts! Pun aside 2 more papers down the line and we are face to face with a revolution in our way of living?
That seems really silly. The obvious way to handle nut/bolt interaction is to take over with custom code so that the nut and bolt effectively become one object that is managed algorithmically, eliminating the need to even have a geometry interface between them.
I want a new Garry's Mod kind of game with this level of physics simulation.
Signed Distance Fields are either turning up in more applications lately, or I'm just now noticing them. We've come a ways from metaballs.
This can be done even easier with an algorithm that teaches borders and padding
To create the best simple robot that can talk and perform tasks, focus on enhancing a good small multimodal models and training them in 3D virtual worlds. This approach improves task efficiency and reduces training risks. Combining these models with verbal interaction capabilities will make robots highly practical and accessible for various settings. Let’s revolutionize human-robot interaction!
him: what would you fellow scholars use this for
my dirty mind ;)
What's the success rate of a custom-made machine? Is this a close enough match for general purpose robot arms to be implemented in a real-world setting? Or is an 85% success rate for the initial attempt still leagues away from being acceptable?
A practical machine for mass production purposes would need 99.99% accuracy and a reliable way to report the the 0.01%
30,000 b.c.e: me think. You think. What if rock think?
Today: rocks thinking
You missed the chance to say... "thats Nuts!"
I was hoping The Witness would get a mention beyond simply being an example of a video game!
When robots are able to knit a sock, manually - then we are screwed!
At last, a solution to my flat pack post traumatic stress.
the virtual robotics revolution, the digital answer to the industrial revolution, is here!
This is two minute papers with Dr. Károly Zsolnai-Fehér. But the troubles don't stop there :D
I think tactile feedback data is still crucial. Without tactile feedback, how does the AI knows when the screws don't match or there are defects.
You really went into the nuts and bolts of it lol
I think " robot tailored to a specific" task is as oxymoronic as "general purpose robot" is tautological.
So, when will this hit game engines like Unreal, and how close are we to Matrix-like virtual realities? I would think, if we can have precision simulations of physics at scales that small, this should make things much more realistic in a virtual setting, and maybe we can use those precise 3D models as a basis for accurate tactile information so touching things would feel real, though we would need a proper user interface that would allow us to feel it, or a brain interface that would send the right signals. I just really want the matrix.
Does that mean we're finally getting real-time liquid simulation for video games?
2022?.. Did I miss something or is this a history lesson..?
What a nut time to be hard alive! 🎉
Can't wait for AI to do at least part of the hard optimizing proccess
this is nuts
Let’s wait until this is implemented in real life
Awesome!!!
So we use this in conjunction with omniverse. Or we could use it with cortical labs research.
Why simulate nuts and bolts at the highest level whilst not being assembled? Once a bolt is in place it can return to being a static part. And when referring to "simulated" here, we obviously leave out actual engineering simulations using FEM, correct material, stick-slip, tribology etc etc. Granted, this will be done in the engineering stage, not the assembly stage. Anyway, I'm sure we'll reach the point where we can do both in the same environment and have AI assist us with everything from ideation, specification, design, validation and production planning/factory organization. It's just a crunching game 😁👍
does anyone else quickly gather their papers when he says to? its very exciting
3:40-3:46 I’m dismayed that the bowls show a distinctive polygon shape at the rim. I wound have thought that by now graphics boards’ algorithms would do circles as primitives. Fundamental building blocks.