Improving family health through acoustic sensing and conversational agents

In this project, we design, develop, and evaluate an innovative family routine sensing and feedback system. This system empowers families to actively engage in preventing child obesity and improving family wellness.

We develop unobtrusive sensing system using built-in audio, motion, and light sensors from off-the-shelf mobile devices (e.g., Tablet personal computer (PC), smartphone) and conversational agents. Tablet PCs or conversational agents will give feedback to the family users on collected sensor data that reflect the family’s progress toward improved routine. Family members will also see how other families are doing compared to them for encouragement and insights.

We plan to engage K-12 students during summer to play with the demonstration of our project, increase motivation for scientific research, and increase awareness of family wellness to the surrounding communities. Our technology will be shared as an open-source after the completion of the project for broader dissemination and development of the science we have developed from our project.

Our project develops new methods and paradigms of clinical support that engage all family members and a community of families in improving child and family health.

Example prototype: The virtual pet will become more active as the child and the family improves routines at home. The families can visually check other families’ statuses.

state1 state2 state3

NSF IIS #1622626

NSF IIS #1622659

Lead PI: Jina Huh-Yoo (HCI, Design)
MSU Site PI: Guoliang Xing (Mobile sensing), Wei Peng (health communication, behavior theory), Barbara Smith (Nursing, Physical activity in youth)