Blade Insertion Affordance Prediction
The Blade Insertion Affordance Prediction task focuses on predicting where and how to insert a blade into a bin given visual observations. In Amazon Vulcan Stow workcells, a blade is deployed into fabric-covered pods to create space for item placement. This dataset provides RGBD images, camera calibration, and full 3D motion trajectories for three distinct insertion strategies.
See How Amazon's Vulcan robots use touch to plan and execute motions for an overview of blade insertion behaviors.
The Blade Insertion Affordance dataset contains
~29,000 blade insertion cycles with data collected from Amazon Vulcan Stow workcells. The dataset supports predicting 3D blade insertion affordances: sparse key SE(3) poses and blade extension lengths, given bin observations.
Task: Given a bin observation O ∈ {RGB image, depth image}, predict the 3D blade insertion affordance, a set of K sparse key SE(3) poses P
k and blade extension lengths b
k that define where and how to insert the blade.
Insertion Strategies:
- Direct Insert (14,948 cycles): APPROACH → EXTEND_BLADE
- Corner Wall Insert (8,671 cycles): APPROACH → EXTEND_BLADE → WALL_SEEK → LIP_SEEK
- Engage Wall Insert (5,887 cycles): APPROACH → ITEM_PUSH → PREPARE_TO_SEEK_WALL → WALL_SEEK → FOLLOW_WALL → EXTEND_BLADE
Data per cycle:
- Bin Images (RGB): Cropped bin images at pre-stow, mid-stow, and post-stow stages (JPEG)
- Bin Depth Images: Corresponding 16-bit PNG depth images (values in mm)
- Full-resolution Stereo: Primary and secondary camera images (2448×2048)
- 3D Motion Trajectories: Blade tip SE(3) poses and arm trajectories per action primitive
- Camera Calibration: Intrinsic matrix, distortion coefficients, and camera-to-bin SE(3) transform
| Category |
Feature |
Description |
Format |
| Images |
bin_images/*.jpg |
Cropped bin RGB at pre/mid/post stow stages |
JPEG |
| bin_depth_images/*.png |
Depth images (divide by 1000 for meters) |
16-bit PNG |
| images_raw/*/primary/*.jpg |
Full-resolution stereo camera images |
JPEG (2448×2048) |
| Camera |
cam_H_bin |
SE(3) transform from camera to bin frame |
4×4 matrix |
| camera_matrix |
Intrinsic matrix [fx, 0, cx; 0, fy, cy; 0, 0, 1] |
3×3 matrix |
| distortion |
Lens distortion coefficients |
float array |
| Trajectories |
blade_translation |
Blade tip xyz position over time |
(N, 3) array |
| blade_rot |
Blade tip quaternion [x, y, z, w] over time |
(N, 4) array |
| arm_translation / arm_rot |
Arm end-effector pose over time |
list of xyz + quat |
| Labels |
insert_strategy |
DIRECT_INSERT / CORNER_WALL_INSERT / ENGAGE_WALL_INSERT |
string |
| action_sequence |
Segmented action primitives with per-action trajectories |
list of dicts |
Available on HuggingFace: amazon/armbench-blade-insertion-affordance. Because the archive exceeds HuggingFace's 500 GB per-file limit it is stored there as six parts,
blade_insertion.zip.part000 through
blade_insertion.zip.part005. Download all six parts, then rejoin them from the directory that holds them with
cat blade_insertion.zip.part* > blade_insertion.zip before extracting.
