TY - JOUR
T1 - Supervised Representation Learning Towards Generalizable Assembly State Recognition
AU - Schoonbeek, Tim J.
AU - Balachandran, Goutham
AU - Onvlee, Hans
AU - Houben, Tim
AU - Hung, Shao-Hsuan
AU - Kustra, Jacek
AU - de With, Peter H.N.
AU - van der Sommen, Fons
PY - 2024/11
Y1 - 2024/11
N2 - Assembly state recognition facilitates the execution of assembly procedures, offering feedback to enhance efficiency and minimize errors. However, recognizing assembly states poses challenges in scalability, since parts are frequently updated, and the robustness to execution errors remains underexplored. To address these challenges, this letter proposes an approach based on representation learning and the novel intermediate-state informed loss function modification (ISIL). ISIL leverages unlabeled transitions between states and demonstrates significant improvements in clustering and classification performance for all tested architectures and losses. Despite being trained exclusively on images without execution errors, thorough analysis on error states demonstrates that our approach accurately distinguishes between correct states and states with various types of execution errors. The integration of the proposed algorithm can offer meaningful assistance to workers and mitigate unexpected losses due to procedural mishaps in industrial settings.
AB - Assembly state recognition facilitates the execution of assembly procedures, offering feedback to enhance efficiency and minimize errors. However, recognizing assembly states poses challenges in scalability, since parts are frequently updated, and the robustness to execution errors remains underexplored. To address these challenges, this letter proposes an approach based on representation learning and the novel intermediate-state informed loss function modification (ISIL). ISIL leverages unlabeled transitions between states and demonstrates significant improvements in clustering and classification performance for all tested architectures and losses. Despite being trained exclusively on images without execution errors, thorough analysis on error states demonstrates that our approach accurately distinguishes between correct states and states with various types of execution errors. The integration of the proposed algorithm can offer meaningful assistance to workers and mitigate unexpected losses due to procedural mishaps in industrial settings.
KW - Representation learning
KW - assembly state detection
KW - assembly state recognition
KW - computer vision for manufacturing
KW - deep learning methods
UR - https://timschoonbeek.github.io/state_rec.html
UR - http://www.scopus.com/inward/record.url?scp=85205421859&partnerID=8YFLogxK
U2 - 10.1109/LRA.2024.3468157
DO - 10.1109/LRA.2024.3468157
M3 - Article
SN - 2377-3766
VL - 9
SP - 9915
EP - 9922
JO - IEEE Robotics and Automation Letters
JF - IEEE Robotics and Automation Letters
IS - 11
M1 - 10694722
ER -