Language is a multifaceted AI challenge that requires people to simultaneously understand tone, context, sentiment, logic, world knowledge, cultural references and much more. Teaching a computer to understand language and its inherent nuances requires a solution that goes beyond leveraging algorithms to decipher just text alone. There must be an understanding of how speech relates to text; visual and linguistic concepts must intertwine and be grounded in order to provide proper contextualization; and computers have to learn entirely new languages such as SQL in order to deliver precision and reliability. In addition, architectures have to scale to support these different tasks and models have to learn how each task relates to one another. Mastering these concepts makes natural language understanding one of the most difficult challenges facing AI research because it requires a single solution that solves for each of these facets.

Salesforce Research has made significant progress in deep learning and natural language processing (NLP) over the course of the last year that address these challenges. We’ve built faster, more scalable model architectures, developed a reinforcement learning agent that programs new neural network architectures, and improved training methods in order to take full advantage of the wealth of data currently available and to increase model performance for each individual NLP task. Today, I’m excited to announce new breakthroughs that bring us closer to a unified approach to tackling the many facets of natural language, paving the way for humans and machines to communicate more effectively and work side-by-side.

The best machine learning model is frequently dictated by various requirements such as the size of the available dataset, how quickly a model must run and the type of data that it will process. This means the optimal machine learning model should be tailored to each dataset and each usage, but there are not enough machine learning engineers and researchers to cover every possibility. We’re introducing a novel approach for automating architecture search to produce recurrent neural networks (RNNs) that are more flexible and take into account non-standard components. By using a domain-specific language (DSL) in automating architecture search it is possible to produce novel RNNs of arbitrary depth and width. The DSL is also flexible enough to define standard architectures such as the Gated Recurrent Unit and Long Short Term Memory and allows the introduction of non-standard RNN components such as trigonometric curves and layer normalization. By improving the efficiency of these architecture generation methods, Salesforce Research is helping move towards a future where the grunt work of testing several different architectures is left to a machine, leaving humans to focus on more challenging aspects of NLP.

When humans learn a new task, we leverage our existing skill set whenever possible. For instance, learning long division requires us to first know how to do simple division. Similarly, enabling models to combine simple, previously learned tasks alongside and as subcomponents of new skills allows them to take on more complex tasks. To do this, we introduce a novel framework for efficient multi-task reinforcement learning (RL) that identifies underlying relationships between skills. As the RL agents executes a task it breaks it into smaller actions using a hierarchy of “policy networks”-- also known as neural networks-- that predict when to use a previously learned policy and when to learn a new skill. This enables programing agents to continually acquire new skills more efficiently during different stages of training. To represent the learned task in an interpretable way, we trained an RL agent to communicate its plans and decisions using human language instructions such as "put down." We validate our approach on Minecraft games, which are designed to explicitly test the ability to reuse previously learned skills while simultaneously learning new skills. In addition to automating chatbot conversations, these results could make it possible to automate bot’s actions, such as tracking down inventory and placing an order on behalf of a customer.

Questions that require counting a variety of objects within an image remain a major challenge in visual question answering (VQA). The most common approaches to VQA involve classifying answers based on fixed-length representations of both the image and question, and interpreting individual components from each section of the image. We see counting — an important yet challenging subtask of VQA — as a sequential decision process that relies heavily on interpretability of the discrete visual representations within an image. We introduce a new Reinforcement Learning Counter (IRLC) model, which learns to count by numbering relevant objects in the scene and sequentially deciding which objects to add to the count. Specifically, the model sequentially selects from detected objects and uses inferred relationships between objects to influence subsequent selections. This approach has the advantage of intuitive and interpretable output, as discrete counts are automatically grounded in the image. Our method outperforms the state of the art architecture for VQA on multiple metrics that evaluate counting.