2023/09/19 by Tuba Girgin, Girgin, Tuba, Emre Uğur +2 · 1 voice · 1 citation
Computer Science · Engineering · Mathematics · #Affordance #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer science #FOS: Computer and information sciences #Field (mathematics) #Graph #Human–computer interaction #Mathematics #Object (grammar) #Robot #Robot Manipulation and Learning #Robotics (cs.RO) #Theoretical computer science #cs.AI #cs.RO
paper · pdf · doi:10.48550/arxiv.2309.10426
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/09/19 · arxiv published 2023/09/19 · openalex created_date 2023/09/21 · arxiv updated 2024/12/17 · openalex updated_date 2026/07/28
Learning object affordances is an effective tool in the field of robot learning. While the data-driven models investigate affordances of single or paired objects, there is a gap in the exploration of affordances of compound objects composed of an arbitrary number of objects. We propose the Multi-Object Graph Affordance Network which models complex compound object affordances by learning the outcomes of robot actions that facilitate interactions between an object and a compound. Given the depth images of the objects, the object features are extracted via convolution operations and encoded in the nodes of graph neural networks. Graph convolution operations are used to encode the state of the compounds, which are used as input to decoders to predict the outcome of the object-compound interactions. After learning the compound object affordances, given different tasks, the learned outcome predictors are used to plan sequences of stack actions that involve stacking objects on top of each other, inserting smaller objects into larger containers and passing through ring-like objects through poles. We showed that our system successfully modeled the affordances of compound objects that include concave and convex objects, in both simulated and real-world environments. We benchmarked our system with a baseline model to highlight its advantages.