Bootstrapping Conversational Agents with Weak Supervision

  • Neil Mallinar IBM
  • Abhishek Shah IBM
  • Rajendra Ugrani IBM
  • Ayush Gupta IBM
  • Manikandan Gurusankar IBM
  • Tin Kam Ho IBM
  • Q. Vera Liao IBM
  • Yunfeng Zhang IBM
  • Rachel K.E. Bellamy IBM
  • Robert Yates IBM
  • Chris Desmarais IBM
  • Blake McGregor IBM

Abstract

Many conversational agents in the market today follow a standard bot development framework which requires training intent classifiers to recognize user input. The need to create a proper set of training examples is often the bottleneck in the development process. In many occasions agent developers have access to historical chat logs that can provide a good quantity as well as coverage of training examples. However, the cost of labeling them with tens to hundreds of intents often prohibits taking full advantage of these chat logs. In this paper, we present a framework called search, label, and propagate (SLP) for bootstrapping intents from existing chat logs using weak supervision. The framework reduces hours to days of labeling effort down to minutes of work by using a search engine to find examples, then relies on a data programming approach to automatically expand the labels. We report on a user study that shows positive user feedback for this new approach to build conversational agents, and demonstrates the effectiveness of using data programming for autolabeling. While the system is developed for training conversational agents, the framework has broader application in significantly reducing labeling effort for training text classifiers.

Published
2019-07-17
Section
IAAI Technical Track: Emerging Papers