Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings

Tim J. Schoonbeek; Hans  Onvlee; Pierluigi Frisco; Peter H.N. de With; Fons van der Sommen

doi:10.1109/VRW58643.2023.00283

Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings

Tim J. Schoonbeek
, Hans Onvlee
, Pierluigi Frisco
, Peter H.N. de With
, Fons van der Sommen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

2 Citations (Scopus)

2 Downloads (Pure)

Abstract

Understanding procedural actions is important, as it can be used to automatically analyze the execution of a procedure and provide assistance to users by warning for potential mistakes or forgotten steps. However, current approaches require a rigid, step-by-step execution order, laborious and impractical datasets. Furthermore, they are unreliable to variations in viewpoint, or measure the performance of actions rather than the actual completion of actions. To address these limitations and stimulate research in this field, this work proposes the novel task of procedure state recognition (PSR) together with a set of evaluation metrics.

Original language	English
Title of host publication	2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
Publisher	Institute of Electrical and Electronics Engineers
Pages	881-882
Number of pages	2
ISBN (Electronic)	979-8-3503-4839-2
DOIs	https://doi.org/10.1109/VRW58643.2023.00283
Publication status	Published - 1 May 2023
Event	2023 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2023 - Sangri-La Pudong Hotel, Shanghai, China Duration: 25 Mar 2023 → 29 Mar 2023 Conference number: 30

Conference

Conference	2023 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2023
Abbreviated title	VR 2023
Country/Territory	China
City	Shanghai
Period	25/03/23 → 29/03/23

Keywords

Computer vision
Action recognition
Procedural action understanding
Procedure state recognition
action recognition
augmented reality
procedure understanding
object state detection

Access to Document

10.1109/VRW58643.2023.00283

Cite this

Schoonbeek, T. J., Onvlee, H., Frisco, P., de With, P. H. N., & van der Sommen, F. (2023). Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 881-882). Article 10108640 Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/VRW58643.2023.00283

@inproceedings{842c8753825540ff91ed43d17b3301b9,

title = "Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings",

abstract = "Understanding procedural actions is important, as it can be used to automatically analyze the execution of a procedure and provide assistance to users by warning for potential mistakes or forgotten steps. However, current approaches require a rigid, step-by-step execution order, laborious and impractical datasets. Furthermore, they are unreliable to variations in viewpoint, or measure the performance of actions rather than the actual completion of actions. To address these limitations and stimulate research in this field, this work proposes the novel task of procedure state recognition (PSR) together with a set of evaluation metrics.",

keywords = "Computer vision, Action recognition, Procedural action understanding, Procedure state recognition, action recognition, augmented reality, procedure understanding, object state detection",

author = "Schoonbeek, \{Tim J.\} and Hans Onvlee and Pierluigi Frisco and \{de With\}, \{Peter H.N.\} and \{van der Sommen\}, Fons",

year = "2023",

month = may,

day = "1",

doi = "10.1109/VRW58643.2023.00283",

language = "English",

pages = "881--882",

booktitle = "2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "2023 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2023, VR 2023 ; Conference date: 25-03-2023 Through 29-03-2023",

}

Schoonbeek, TJ, Onvlee, H, Frisco, P, de With, PHN & van der Sommen, F 2023, Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings. in 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)., 10108640, Institute of Electrical and Electronics Engineers, pp. 881-882, 2023 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2023, Shanghai, China, 25/03/23. https://doi.org/10.1109/VRW58643.2023.00283

Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings. / Schoonbeek, Tim J.; Onvlee, Hans ; Frisco, Pierluigi et al.
2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Institute of Electrical and Electronics Engineers, 2023. p. 881-882 10108640.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Beyond Action Recognition

T2 - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2023

AU - Schoonbeek, Tim J.

AU - Onvlee, Hans

AU - Frisco, Pierluigi

AU - de With, Peter H.N.

AU - van der Sommen, Fons

N1 - Conference code: 30

PY - 2023/5/1

Y1 - 2023/5/1

N2 - Understanding procedural actions is important, as it can be used to automatically analyze the execution of a procedure and provide assistance to users by warning for potential mistakes or forgotten steps. However, current approaches require a rigid, step-by-step execution order, laborious and impractical datasets. Furthermore, they are unreliable to variations in viewpoint, or measure the performance of actions rather than the actual completion of actions. To address these limitations and stimulate research in this field, this work proposes the novel task of procedure state recognition (PSR) together with a set of evaluation metrics.

AB - Understanding procedural actions is important, as it can be used to automatically analyze the execution of a procedure and provide assistance to users by warning for potential mistakes or forgotten steps. However, current approaches require a rigid, step-by-step execution order, laborious and impractical datasets. Furthermore, they are unreliable to variations in viewpoint, or measure the performance of actions rather than the actual completion of actions. To address these limitations and stimulate research in this field, this work proposes the novel task of procedure state recognition (PSR) together with a set of evaluation metrics.

KW - Computer vision

KW - Action recognition

KW - Procedural action understanding

KW - Procedure state recognition

KW - action recognition

KW - augmented reality

KW - procedure understanding

KW - object state detection

UR - https://www.scopus.com/pages/publications/85159657718

U2 - 10.1109/VRW58643.2023.00283

DO - 10.1109/VRW58643.2023.00283

M3 - Conference contribution

SP - 881

EP - 882

BT - 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

PB - Institute of Electrical and Electronics Engineers

Y2 - 25 March 2023 through 29 March 2023

ER -

Beyond Action Recognition: Extracting Meaningful Information from Procedure Recordings

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this