- 19
- 30 287
Robotics & Embodied Artificial Intelligence Lab
United States
Registrace 10. 10. 2020
We are interested in developing algorithms that enable intelligent systems to learn from their interactions with the physical world to execute complex tasks and assist people.
Robotics & Embodied Artificial Intelligence Lab Tour
To learn more about the Robotics & Embodied Artificial Intelligence Lab (REAL) at Stanford, please visit real.stanford.edu/
zhlédnutí: 11 492
Video
Dynamics-Guided Diffusion Modelfor Robot Manipulator Design
zhlédnutí 366Před 3 měsíci
Dynamics-Guided Diffusion Model for Robot Manipulator Design Website: dgdm-robot.github.io/ Abstract: We present Dynamics-Guided Diffusion Model, a data-driven framework for generating manipulator geometry designs for a given manipulation task. Instead of training different design models for each task, our approach employs a learned dynamics network shared across tasks. For a new manipulation t...
UMI - 3D Printing Tutorial
zhlédnutí 2,8KPřed 3 měsíci
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots Project Webpage: umi-gripper.github.io Hardware Guide: docs.google.com/document/d/1TPYwV9sNVPAi0ZlAupDMkXZ4CA1hsZx7YDMSmcEy6EU/edit?usp=sharing Software Links: Ultimaker Cura: ultimaker.com/software/ultimaker-cura/ OnShape CAD: www.onshape.com/ Amazon Links: Creality Ender3 S1 Printer: www.amazon.com/Crealit...
UMI - Assembly Tutorial
zhlédnutí 3,4KPřed 3 měsíci
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots Project Webpage: umi-gripper.github.io Hardware Guide: docs.google.com/document/d/1TPYwV9sNVPAi0ZlAupDMkXZ4CA1hsZx7YDMSmcEy6EU/edit?usp=sharing UMI compatible GoPros: GoPro Hero9 (recommended for lower cost and energy efficiency): www.amazon.com/GoPro-HERO9-Black-Commerce-Stabilization/dp/B09J713ZS7 GoPro He...
RoboNinja: Learning an Adaptive Cutting Policy for Multi-Material Objects
zhlédnutí 809Před rokem
Website: roboninja.cs.columbia.edu/ We introduce RoboNinja, a learning-based cutting system for multi-material objects (i.e., soft objects with rigid cores such as avocados or mangos). In contrast to prior works using open-loop cutting actions to cut through single-material objects (e.g., slicing a cucumber), RoboNinja aims to remove the soft part of an object while preserving the rigid core, t...
Bag All You Need: Learning a Generalizable Bagging Strategy for Heterogeneous Objects
zhlédnutí 437Před rokem
Website: bag-all-you-need.cs.columbia.edu Abstract: We introduce a practical robotics solution for the task of heterogeneous bagging, requiring the placement of multiple rigid and deformable objects into a deformable bag. This is a difficult task as it features complex interactions between multiple highly deformable objects under limited observability. To tackle these challenges, we propose a r...
Cloth Funnels: Canonicalized-Alignment for Multi-Purpose Garment Manipulation
zhlédnutí 685Před rokem
Website: clothfunnels.cs.columbia.edu/ Abstract: Automating garment manipulation is challenging due to extremely high variability in object configurations. To reduce this intrinsic variation, we introduce the task of "canonicalized-alignment" that simplifies downstream applications by reducing the possible garment configurations. This task can be considered as "cloth state funnel" that manipula...
BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment
zhlédnutí 410Před rokem
We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual feedback for robot interactions. We also introduce a learning framework, BusyBot, which allows an agent to jointly acquire three fundamental capabilities (interaction, reasoning, and planning) in an integrated and self...
Iterative Residual Policy for Goal-Conditioned Dynamic Manipulation of Deformable Objects
zhlédnutí 1,3KPřed rokem
This paper tackles the task of goal-conditioned dynamic manipulation of deformable objects. This task is highly challenging due to its complex dynamics (introduced by object deformation and high-speed action) and strict task requirements (defined by a precise goal specification). To address these challenges, we present Iterative Residual Policy (IRP), a general learning framework applicable to ...
Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly
zhlédnutí 375Před 2 lety
website: seat.cs.columbia.edu Studies in robot teleoperation have been centered around action specifications-from continuous joint control to discrete end-effector pose control. However, these “robot-centric” interfaces often require skilled operators with extensive robotics expertise. To make teleoperation accessible to non-expert users, we propose the framework “Scene Editing as Teleoperation...
DextAIRity: Deformable Manipulation Can be a Breeze
zhlédnutí 748Před 2 lety
This paper introduces DextAIRity, an approach to manipulate deformable objects using active airflow. In contrast to conventional contact-based quasi-static manipulations, DextAIRity allows the system to apply dense forces on out-of-contact surfaces, expands the system's reach range, and provides safe high-speed interactions. These properties are particularly advantageous when manipulating under...
UMPNet: Universal Manipulation Policy Network for Articulated Objects
zhlédnutí 677Před 2 lety
Website: ump-net.cs.columbia.edu/ We introduce the Universal Manipulation Policy Network (UMPNet) a single image-based policy network that infers closed-loop action sequences for manipulating arbitrary articulated objects. To infer a wide range of action trajectories, the policy supports 6DoF action representation and varying trajectory length. To handle a diverse set of objects, the policy lea...
FlingBot: The Unreasonable Effectiveness of Dynamic Manipulations for Cloth Unfolding
zhlédnutí 1,4KPřed 3 lety
FlingBot: The Unreasonable Effectiveness of Dynamic Manipulations for Cloth Unfolding Website: flingbot.cs.columbia.edu Abstract: High-velocity dynamic actions (e.g., fling or throw) play a crucial role in our everyday interaction with deformable objects by improving our efficiency and effectively expanding our physical reach range. Yet, most prior works have tackled cloth manipulation using ex...
Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery
zhlédnutí 389Před 3 lety
People often use physical intuition when manipulating articulated objects, irrespective of object semantics. Motivated by this observation, we identify an important embodied task where an agent must play with objects to recover their parts. To this end, we introduce Act the Part (AtP) to learn how to interact with articulated objects to discover and segment their pieces. By coupling action sele...
GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion
zhlédnutí 532Před 3 lety
This paper tackles the task of category-level pose estimation for garments. With a near infinite degree of freedom, a garment's full configuration (i.e., poses) is often described by the per-vertex 3D locations of its entire 3D surface. However, garments are also commonly subject to extreme cases of self-occlusion, especially when folded or crumpled, making it challenging to perceive their full...
AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policy
zhlédnutí 997Před 3 lety
AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policy
Learning a Decentralized Multi-arm Motion Planner
zhlédnutí 1KPřed 3 lety
Learning a Decentralized Multi-arm Motion Planner
Fit2Form: 3D Generative Model for Robot Gripper Form Design
zhlédnutí 1,2KPřed 3 lety
Fit2Form: 3D Generative Model for Robot Gripper Form Design
Learning 3D Dynamic Scene Representations for Robot Manipulation
zhlédnutí 1,4KPřed 3 lety
Learning 3D Dynamic Scene Representations for Robot Manipulation
Can you support the bill of materials of UR5 robot gripper?
@real-stanford the paper mentions this project is robot arm agnostic - do you have a list of minimum spec requirements for a viable arm?
Excellent teaching tutorial for device assembly. Suggestion: include the full name of the device in the title of the video.
Good lessons, you cover everything!
This project is super exciting. Are there any plans to release more explainers on deployment, training the collected data, or data collection for multiple grippers? I see there is a notion page explaining data collection for one gripper, but I am wondering what process changes there might be for multi grippers.
I'd like to connect UMI to a much cheaper, ~$1000 robotic arm. I can't figure out how can I connect UMI with the arm. I guess some mount is needed, but how can I "translate" the arm movements to the camera desired state?
You should look into inverse kinematics en.wikipedia.org/wiki/Inverse_kinematics. May robot arms come with such functionality in their SDK
@@real-stanford thank you very much, that's exactly what I missed!
cool, question: it seems to me that such input device has a lot to do with the output part, say tomorrow there comes another cool robot with three fingers, is this two-finger umi device still compatible with it?
Using our learning method as-is probably won't work for three finger grippers. However, you can probably still reuse the data for cross-embodiment pretraining similar to RT-X robotics-transformer-x.github.io/
the reliabot linear rails (mgn9c @150mm) have been delisted, fyi. We ordered the 300mm rails with the mgn9c carriage blocks instead, thinking that we might be able to cut down the rail and still have the rail holes align with the 3d printed bottom plate. We also ordered rails/carriages off aliexpress, but that will take time to get here. Thanks for this project, its super interesting. Stanford really is the place to be right now. We've seen robotics solutions go from $300k to $30k with the same functionality in the last year. incredible. Can't wait for that next leap to $3k! :D Random thought, some of us have a multi material bambu labs printers. Would you all be interested in a design that can 3d print some if not all of the qr codes? it might save some time, align more consistently, be more durable, be simpler... for those who have the equipment. at .4mm nozzle width it might not be precise enough for the smaller QR codes, but at least for the large codes it could work. Not asking you all to do more work, just wondering if it might be useful. If so we might be able to find a volunteer or two to make a bambu specific 3d print. Anyway, keep up the good work! We're all big fans. -- PDX Hackerspace in Portland, Oregon
Why am i here?!? Where am I? MOM, HALP!
Thanks for making this video. Any plans for making videos for data collection with UMI as well?
Thanks for sharing
Hello mister, this shows how to get the "hands of the robot" but how to get the rest of the robot? the arms?
This video makes me feel optimistic about the bright side of human collaboration, thanks mister, to you and your collaborators, amazing job
awesome thanks for sharing!
Nice work!🎉
Goood job:)!!! Easy video ranking = P r o m o S M !
arXiv: arxiv.org/abs/2207.08192 Website: busybot.cs.columbia.edu Github: github.com/columbia-ai-robotics/busybot For questions, please contact Zeyi Liu lzylucy.github.io
😏 ᎮᏒᎧᎷᎧᏕᎷ
What are the range values of robustness
Wonderful work!