One-Shot Interactions with Intelligent Assistants in Unfamiliar Smart Spaces

Abstract

Smart space technologies have entered the mainstream home market. Most users currently interact with smart homes that they (or an acquaintance) have set up and know well. However, as these technologies spread to commercial or public environments, users will need to frequently interact with unfamiliar smart spaces. In such settings, users will be unaware of the available capabilities and the system maintainer will not be present to help. Users will need to quickly and independently 1) discover what is and is not possible, and 2) make use of available functionality. However, current smart spaces use a ``proper device name’’ paradigm that requires users to know which individual devices are smart and what their proper names are in order to interact with the space. This approach makes it nearly impossible for users to operate unfamiliar smart spaces, especially when using natural language intelligent assistants. Widespread adoption of smart space systems, and all the benefits they may confer, will not be possible until this discoverability issue is solved. In this work, we characterize the problems associated with the proper device name paradigm and evaluate potential solutions. First, we look at the limiting effect that device-centrism has on a user’s conception of system capabilities, and find that framing interactions around intelligent assistants can mediate these negative effects. Then, we examine how the use of proper names to refer to system resources poses particular challenges in smart spaces. To address these issues while preserving an assistant framing, we borrow patterns from human-to-human messaging and apply them to human-to-assistant communication. Our method of using contextual photo messages enhanced by two technologies – augmented reality and autocomplete – allows users to determine available functionality and achieve their goals in one attempt with a smart space they have never seen before, something no existing interface supports. The ability to easily operate unfamiliar smart spaces improves the usability of existing systems and removes a significant obstacle to the vision of ubiquitous computing.