RSTOD: Novel Auxiliary Learning Techniques for Efficient and Controllable Task-Oriented Conversational Agents
ISEF · 2023 Robotics and Intelligent Machines
Overview
Conversational agents are increasingly used in a wide range of fields, from entertainment or manufacturing to finance or healthcare, since they facilitate human-machine interactions and can automate repetitive tasks. Recently, their text-generation abilities have been significantly enhanced by the adoption of pre-trained language models. However, these models are computationally expensive, slow to use because of the large number of trainable parameters, and sometimes fail to generate diverse responses. To address these limitations, we propose extending the training process with auxiliary learning - a method to improve the abilities of a primary deep learning architecture to generalize to unseen data by training it simultaneously with additional tasks. After experimentation with various autoregressive and diffusion-based language models, two techniques were found to be successful at producing high-quality conversations - using an auxiliary classifier to (1) distinguish distractors from ground truth responses or (2) distinguish synthetic responses from ground truth labels. Both response selection task-oriented dialogue (RSTOD) systems achieved state-of-the-art results on the MultiWOZ benchmark - a dataset with dialogues spanning over seven domains - and outperformed previous models having three times more parameters. Furthermore, we open-sourced the implementations of our techniques and integrated them into multiple software packages so researchers and developers can use them to optimize their conversational agents to be accessible on lower-end hardware, consume less power, and produce controllable less-biased texts.
Awards (3)
- King Abdulaziz & his Companions Foundation for Giftedness and Creativity: Full Scholarship from King Fahd University of Petroleum and Minerals(KFUPM) (and a $400 cash prize) $400
- Association for the Advancement of Artificial Intelligence: Honorable Mention (do not read aloud). Winners receive a student level membership. Information is included separately in the SAO Portal.
- Association for the Advancement of Artificial Intelligence: AAAI Student Memberships for each finalist that is part of the 1st, 2nd, and 3rd Prize Winning projects and 5 Honorable Mention winning projects (up to 3 students per project) (in-kind award / part of the 1st-3rd prize)
Competition history
- ISEF 2023
Resources
Related projects
ISEF · 2024
A Lightweight Linguistic-Enhanced Word Embedding Framework for Medical Q&A Chatbot
ISEF · 2023
A Single Usage Is All You Need: Generating an Entire Text by Using a Causal Language Model Once
CWSF · 2026
From Single-Task Imitation to Real-World Generalization: A Hierarchical Reason-Act Robot System
ISEF · 2026
Keep Your Data Close, but Your Failures Closer: Failure-Driven Adversarial Self-Evolution of Language Models
Closest projects by meaning, across every fair and year in the corpus.
Source: Regeneron International Science and Engineering Fair