Pillar 1 · Policy verification
ShippedDoor stress
Success-primary Door — grasp N/A, open/fail is the metric.
Three severity tiers of panel-scaled mass/friction stress on robosuite Door, paired seeds, Wilson confidence intervals. Geom-based grasp is undefined for the rotating handle, so the gate is success-primary (success ratio only). The curve is shallower than grasp tasks — expected physics, not a softer claim — and severe failures are logged as door did not open.
What we tested
Where does a Door policy break as panel mass and friction move further from nominal?
- success rate, nominal → severe
0.60 → 0.52
success rate, nominal → severe
- severe episodes: door did not open
48%
severe episodes: door did not open
- gate tier (mild, success-primary)
PASS
gate tier (mild, success-primary)
Charts
Success rate across severity
rate
Mean peak end-effector force
N (contact proxy)
Metrics
- gate: stress success ≥ 80% of nominal (success-primary)
- grasp-fail N/A (rotating handle)
- Wilson 95% CIs on all rates
| Metric | nominal | mild | moderate | severe |
|---|---|---|---|---|
| Success rate | 0.60 | 0.64 | 0.58 | 0.52 |
| Success 95% CI | [0.46, 0.72] | [0.50, 0.76] | [0.44, 0.71] | [0.39, 0.65] |
| Grasp-failure rate (N/A) | 0.00 | 0.00 | 0.00 | 0.00 |
| Mean peak EE force (contact proxy) | 209.02 | 185.57 | 199.09 | 191.91 |
| Score (tier / nominal) | — | 1.07 | 0.97 | 0.87 |
| Verdict at gate thresholds | — | PASS | PASS | PASS |
Methodology
- Nominal baseline. Run Door with default panel mass/friction; record success and peak EE force over paired seeds (grasp N/A).
- Tiered stress. Resample door-panel mass and sliding friction into panel-scaled mild → moderate → severe ranges while holding seeds paired with the baseline.
- Success-primary gate. Score gate tier on success-ratio only; grasp-fail is reported as N/A and is not part of the gate.
- Failure logging. Log each failed episode with sampled panel physics; classify as door did not open when hinge angle never credits success.
Seeds: 0..49 paired across conditions (50 episodes × 4 conditions)
- gate: stress success ≥ 80% of nominal (success-primary)
- grasp-fail N/A (rotating handle)
- Wilson 95% CIs on all rates
Failure case · Door did not open
Handle contact is not the score — the door never opens.
Door's rotating handle makes geom-based grasp always false, so a grasp-fail gate would be meaningless. Success-primary scoring puts the hinge open rate on the degradation curve instead.
How: Paired seeds across panel-scaled mild → moderate → severe mass/friction ranges. Failures logged when the hinge never credits an open; grasp rates stay N/A.
48%
of severe episodes: door did not open
| Seed | Mass (kg) | Sliding μ | Mode |
|---|---|---|---|
| 0 | 21.22 | 0.199 | door did not open |
| 1 | 13.77 | 0.267 | door did not open |
| 3 | 21.79 | 0.357 | door did not open |
| 9 | 12.87 | 0.237 | door did not open |
Door panel nominal mass is ~2.43 kg — packaged tiers use panel-scaled stress so mild/moderate/severe increase difficulty rather than lightening the panel.
Notes
Sim-only Door stress with success-primary gate. Shallower curve than grasp tasks is expected for rotating-handle physics — not a substitute for real-hardware validation.
Sim-only — validated concept evidence, not a substitute for real-hardware validation or the full platform vision. Grasp semantics differ from Lift/Stack/PickPlaceCan.