The AI Economist is an AI framework from the Salesforce Research team, developed to study how to improve economic design using AI with the goal of optimizing productivity and social equality for everyone. It is designed to simulate millions of years of economies – in parallel – to help economists, governments and others design tax policies that optimize social outcomes in the real world.

Open Image Modal

Image Modal

You might ask, why is a CRM company applying artificial intelligence (AI) and reinforcement learning (RL) to economics? We sat down with Chief Scientist and Head of Salesforce Research, Richard Socher, and his team of researchers, including Senior Research Scientist, Stephan Zheng, Lead Research Scientist, Nikhil Naik, and Senior Research Scientist, Alex Trott to see what the AI Economist is, why the team pursued this research, and how the AI Economist created a model that is 16% more effective than a leading tax framework.

Q: What does the Salesforce Research team do? Why economics?

Richard: The Salesforce Research team tackles problems important to our customers, to the field of AI, and to all our stakeholders. For example, we’ve developed deep learning APIs that enable our customers to build AI-powered apps with Einstein Vision and Einstein Language, and speech recognition technology that powers Einstein Voice. In the field of AI, we do pure research and applied research to drive AI innovation and improve the state of the world. We’ve discovered breakthroughs in natural language generation, explainable AI and we’ve created AI models for protein generation.

At Salesforce Research, we like to tackle big problems. If you look around the world, one of the most pressing global issues of our day is economic inequality. Studies have shown that large income inequality gaps can negatively impact economic growth and economic opportunity. With the AI Economist, we’ve applied reinforcement learning algorithms to discover how novel tax frameworks can reduce inequality and improve economic productivity, and ultimately make the world a better place. We believe this is going to be a very impactful future direction for the AI research community and eventually all economic stakeholders.

Q: How does the AI Economist work?

Stephan: Taxes and subsidies are important tools governments use to reduce inequality and redistribute wealth. However, we still haven’t quite figured out how to implement optimal tax policies for a wide range of social objectives, such as the trade-off between equality and productivity. Economic theory cannot fully model the complexities of the real world, and careful real-world experimentation with taxes is almost impossible.

Through the AI Economist, we’re bringing reinforcement learning (RL) to tax research for the first time to provide a simulation and data-driven solution to defining optimal taxes for a given socio-economic objective.

The AI Economist uses a collection of AI agents designed to simulate how real people might react to different taxes. In the simulation, each AI agent earns money by collecting and trading resources and building houses. The agents learn to maximize their utility (or happiness) by adjusting their movement, trading, and building behavior. One way to do this is to maximize income while minimizing effort, for example, making as high of an hourly wage as possible.

Simultaneously, the AI Economist learns to optimize taxes and subsidies to promote global objectives.

Open Image Modal

Image Modal

Q: Why is the AI Economist exciting from an economics point of view?

Richard: Economics is highly complex and we fully acknowledge that we aren’t economists. We partnered with David Parkes, a CS professor running an Economics and CS research group at Harvard, for example, who showed us that some of our assumptions were too simplistic. Nikhil Naik on our team was also integral for his prior experience working with a Nobel Prize-winning economist.

Nikhil: Our model is incredibly powerful. Economists have previously relied on theorems, but theorems require simple math and are predicated on people behaving rationally. Our world today is getting more complex and economic theories of the future need to be able to seamlessly incorporate additional requirements such as environmental protection. In addition, economic agents often exhibit complex, irrational, competitive or collaborative behaviors. AI helps to model such complexity and a broad spectrum of behaviors.

Our model gives economists and policy-makers additional tools on which they can base their decisions. By simulating millions of years of economies and finding a variety of tax frameworks, the AI Economist can predict how people would actually respond to a tax, like whether it will incentivize them to work more or work less.

Q: How could this model be applied to help navigate us out of times of economic uncertainty, such as what we’re experiencing with COVID-19?

Nikhil: Currently, the AI Economist is solely focused on taxes. However, we think RL is promising for economics and could potentially be applied to navigating the economic aftermath of COVID-19. Economic simulations can factor in human behavior by using real-world, human data. Together with our RL algorithms, this could lead to AI-designed economic policies that could help accelerate real-world economic recovery. We are already thinking of ways to approach this and encourage researchers thinking about this to reach out to us.

Q: Why is the AI Economist an interesting research challenge?

Richard: Reinforcement learning has made a number of breakthroughs through game-playing, think Alpha Go for example. However, in the end, games are just games — when chess was ‘solved,’ the rest of the world didn’t really change that much afterwards. If instead of playing games, we as AI researchers focus on improving the realism and scale of these economic simulations and the abilities of the AI agents and the AI Economist to improve the overall outcomes we can have a lot of positive impact.

Alex: The AI Economist is a first step in broadening the application of RL to areas with the most potential for positive impact. We’ve found that a whole lot of economics can be usefully served through RL tools, and we’re excited to bring these fields together with the AI Economist. Our hope is that the AI Economist can empower economists to make informed policy decisions, and in the future, politicians can use the tool to optimize for a specific social objective, like helping the middle class.

Q: And what did you learn?

Stephan: In our experiments, the AI Economist improves the balance between equality and productivity by at least 16%, compared to an adaptation of the Saez tax formula, (a prominent tax framework), US Federal income rates, and the free market (no taxes).

The AI Economist learns to set different tax schedules: higher top tax rates and lower rates for middle incomes, which yields higher net subsidies for low incomes.

Alex: Economic AI agents learned to “game” taxes to lower their effective tax rate. Every time the AI government intervenes or changes its tax policy, these agents are smart enough to find loopholes or ways to game the system, like alternating between tax periods with high and low incomes. From a technical point of view, that means things can become very unstable. Equality can fluctuate. Because AI agents are trying to maximize profits in a fluctuating environment, it means the simulation is closer to what it’s like in real life.

Richard: It also shows how the AI Economist can learn to anticipate such behavior and try to find counter measures.

Stephan: And perhaps most importantly, the AI Economist also achieves improvements in equality and productivity with real people earning real money in our simulation. In particular, the AI Economist achieves significantly higher inverse-income weighted social welfare, an alternative, established social objective used by economists. This objective weights the utility of lower income workers more than higher income workers.

Q: So, what impact do you hope it has on economics and AI? What’s the next step?

Richard: Right now, governments are making concrete economic decisions based on models that are advanced, but aren’t able to replicate the complexity we get with the AI Economist. We hope that the AI Economist eventually gives countries the ability to predict and foresee the ramifications of their tax policies and then come up with better, transparent ones. In addition, these new simulation based models can optimize multiple new objectives that incorporate social outcomes, environmental outcomes while also keeping a focus on overall productivity.

Nikhil: We hope that economists see this for what it is — a tool that gives economists and governments unprecedented modeling capabilities to augment their research and hypotheses. We want to partner with more economists and governments to help them run simulations on the AI Economist and see how this applies to the real world.

Stephan: And finally, we’d like for this research to gain traction in the machine learning community and inspire other researchers to tackle some of the most intractable societal issues we face today. This is a great technical challenge, and we believe this kind of research has enormous potential to achieve lasting social and scientific improvements.

At Salesforce, we believe AI has the potential to make a huge impact. AI is only as good as the people and data behind it, so we believe making this research public is a first step in ensuring other researchers and the world benefit. We made a deliberate decision to not patent this research. We will open-source our simulations and algorithms. We’ll present and circulate this research at upcoming conferences. We want to be as transparent as possible.

Editor’s Note (8/6/2020): The Salesforce Research Team has launched open source project to build an AI Economist for the real world, and wants to work with you. To find out more, check out this blog post, website, and github.

Publisher’s Note: Updated research and details on a recent publication of the AI Economist work can be found in this story.