TY - GEN
T1 - Unobtrusive measurement of subtle nonverbal behaviors with the Microsoft Kinect
AU - Burba, Nathan
AU - Bolas, Mark
AU - Krum, David M.
AU - Suma, Evan A.
PY - 2012
Y1 - 2012
N2 - We describe two approaches for unobtrusively sensing subtle nonverbal behaviors using a consumer-level depth sensing camera. The first signal, respiratory rate, is estimated by measuring the visual expansion and contraction of the user's chest cavity during inhalation and exhalation. Additionally, we detect a specific type of fidgeting behavior, known as "leg jiggling," by measuring high-frequency vertical oscillations of the user's knees. Both of these techniques rely on the combination of skeletal tracking information with raw depth readings from the sensor to identify the cyclical patterns in jittery, low-resolution data. Such subtle nonverbal signals may be useful for informing models of users' psychological states during communication with virtual human agents, thereby improving interactions that address important societal challenges in domains including education, training, and medicine.
AB - We describe two approaches for unobtrusively sensing subtle nonverbal behaviors using a consumer-level depth sensing camera. The first signal, respiratory rate, is estimated by measuring the visual expansion and contraction of the user's chest cavity during inhalation and exhalation. Additionally, we detect a specific type of fidgeting behavior, known as "leg jiggling," by measuring high-frequency vertical oscillations of the user's knees. Both of these techniques rely on the combination of skeletal tracking information with raw depth readings from the sensor to identify the cyclical patterns in jittery, low-resolution data. Such subtle nonverbal signals may be useful for informing models of users' psychological states during communication with virtual human agents, thereby improving interactions that address important societal challenges in domains including education, training, and medicine.
KW - breathing
KW - depth sensors
KW - fidgeting
KW - nonverbal behavior
UR - http://www.scopus.com/inward/record.url?scp=84860733827&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84860733827&partnerID=8YFLogxK
U2 - 10.1109/VR.2012.6180952
DO - 10.1109/VR.2012.6180952
M3 - Conference contribution
AN - SCOPUS:84860733827
SN - 9781467312462
T3 - Proceedings - IEEE Virtual Reality
BT - IEEE Virtual Reality Conference 2012, VR 2012 - Proceedings
T2 - 19th IEEE Virtual Reality Conference, VR 2012
Y2 - 4 March 2012 through 8 March 2012
ER -