Every night, our internal forecasting platform generates nearly four million forecasts that power our global infrastructure. They decide how much cloud capacity we reserve, which workloads to migrate, and whether an infrastructure budget is going to run over. Each forecast instance is defined by just a few settings: what data to use, which model to run, and how accurate it needs to be.
That simplicity in our forecasting platform let us scale from five use cases to more than a hundred without a matching increase in data scientists. But launching a forecast is the easy part. Keeping it accurate means constantly tuning it. The metrics we forecast constantly change and models drift, so each forecast needs a data scientist to work out what changed and test a fix.
In order to scale our model tuning, we built an internal self-evolving data science agent that investigates forecast drift, tests candidate model fixes against recent history, and opens a pull request (proposed code change a data scientist must approve). “Evolving” here means the agent reuses lessons from past investigations and reviewer feedback. It doesn’t deploy models on its own.

To date, the agent has proposed more than 100 forecast model fixes like swapping models from Prophet to Moirai, tuning change point sensitivity, and adjusting the training window. Although results vary by forecast, we observed a 14% reduction in sMAPE error during backtests, with some forecasts showing significant improvements. More importantly, a typical agentic investigation takes 27 minutes instead of hours. This transition from reactive data scientist debugging to autonomous agentic tuning enables our platform to self-evolve with the ever-changing metrics at scale.
From a Forecast Miss to a Tested Change
Consider a storage forecast for a single AWS availability zone. Teams use it to decide how much capacity to reserve months ahead. Through the first half of 2026, usage grew at a steady pace. Then, in June, growth flattened as new workloads started landing in other regions. The forecast kept projecting the old trend. Left alone, it would have had us reserve, and pay for, about 100 TB of storage we’d never use. But the agent’s pull request swapped in Moirai, Salesforce’s a time-series forecasting model, which helped us avoid reserving unneeded capacity

Here are the six steps of how the agent caught and fixed the forecast model.

1. Localize. Find which forecasts are off track.
2. Diagnose. Examine changes in the data and search Slack for relevant context.
3. Remediate. Try new models and settings, including foundation models like Moirai 2.0.
4. Re-evaluate. Test the new forecasts against past data.
5. Decide. Open a pull request for a data scientist to review if a new forecast model is more accurate.
6. Evolve. Update the investigation playbook with the fix that worked.
The agent can’t change a production forecast. It can only open a pull request, and only when a fix beats the current model by at least 5% on data it hasn’t seen. Each investigation has a fixed testing budget, and when the evidence is unclear, it hands the case to a data scientist.
Architecture: One Agent, Many Workers

The agent doesn’t replace our ML stack. It sits on top of it. Experiments still run on the standard tools: Kubernetes and Spark do the computation, and MLflow tracks and ranks the results. What’s new is the agent between Slack and those engines, which gives data scientists one conversational interface to the whole system. A data scientist can ask a question in Slack. The agent works out why a forecast missed and which fixes to try. A single investigation can mean dozens of backtests, far too much compute for the agent to handle itself, so it sends each job to the right engine. Smaller jobs run on Kubernetes, one pod per variant, and larger ones run on Spark, where one configuration is tested across many series at once. MLflow scores every candidate and sends the top models back to the agent, which picks the winner and automatically opens a pull request for review.
What’s Next
Before our self-evolving agent, data scientists had to investigate each forecast miss, with AI playing only a supporting role. Now we’ve turned that around. The self-evolving agent does the data science work, and data scientists help by reviewing its fixes and giving feedback. That feedback carries into the next investigation, so the agent keeps improving. Our team now spends its time building new forecasts instead of maintaining old ones.
We’re extending the self-evolving agent beyond model tuning:
- Forecast pipeline health. An accurate model only helps if the job that runs it finishes. We’re extending the self-evolving agent to monitor the data pipelines behind each forecast. As with tuning, what the agent can do on its own depends on what a wrong action would cost.
- Automatic forecast onboarding. A business planner could ask for a weekly AI token forecast by model and region. The agent would identify the source data, check that it has enough usable history, and pick the best model before opening a pull request for production.
- Multimodal intelligence. Right now the agent reacts after a forecast misses. We want it to read context like launch plans and migration announcements alongside the numbers, so it can adjust a forecast before the change lands. Recent research on text-conditioned forecasting suggests models can do this by reading numbers and plain language together.
These steps move us toward forecasts that don’t just recover from misses but anticipate them, and an agent that keeps learning from every investigation.















