As you can imagine we are getting a lot of questions about forecasting at this time… “How can it be done?” and “How accurate can it be when things are uncertain?”
What we are doing at conDati, for the most part, remains the same as pre-COVID-19. In brief, we use an advanced statistical approach using machine learning that is highly accurate. The predictive models use historical data to understand the seasonality and trends of a company and takes regressors or potential reasons for variation into consideration. If you want more in depth information on this, I encourage you to watch our on-demand webinar “What it Takes to Accurately Forecast Revenue” or read this blog from earlier this year or reach out to us for a demo, we love to talk about this stuff!
How we are adjusting forecast processes
So, while the math and data science remain the same, how often we run models, the historical data we look at, and the horizon for the forecasted results has completely changed. This is to ensure there is a level of accuracy as we move into and through the “recovery phase” and on to the “new normal”. To do this we are rerunning models weekly for our customers to learn from the most recent performance data. For US customers, we are capturing the impact on revenue and conversions from the everchanging landscape of the pandemic by looking at performance data starting March 13, the date the national emergency was declared. Through many hours of experiments, what we have found is going back further only muddies the water. As for the horizon, that has been drastically reduced, at least for now, from a 12-month window down to predicting results two weeks out.
So why only two weeks? When there is uncertainty and potential for big changes like we are experiencing in the economy right now the farther you look into the future the more unsure predictions can be. To help illustrate this change in confidence over time let’s look at some of the numbers we are seeing.
In this digital marketing revenue example, the first 6 weeks of actuals (the solid green line) were used to create the forecast model and predict weeks 7 through 12 (darker dotted blue line). The confidence in the prediction is represented by the confidence band (lighter blue). The first predicted week (Wk7) shows confidence of ±15%. If we kept this model and didn’t rerun it weekly and learn from the most recent performance data, by the fourth week (Wk10) of predicted results the confidence jumps to ±40%. As you can see the shorter window is allowing our customers to see what is directly ahead of them so they can confidently make necessary spend and campaign decisions. And because we are updating the model weekly our customers continue to have the foresight to anticipate and be agile to positive shifts. Something to keep in mind as you look at this example is confidence levels are highly dependent on historical data and model parameter so not all forecasts will start with a confidence of ±15%. In general, if the model is able to find a structured pattern in the data the confidence band would narrow. But what you will see no matter where it starts is the widening of the confidence band over time through periods of uncertainty. Anyone using forecasts with predictions a quarter or more out that are not continually being updated should proceed with caution when drastically changing marketing strategies.
Looking toward a better tomorrow
As you navigate the current situation and look toward recovery, understanding ‘your today’ is vital in determining where you should be focusing efforts and potentially increasing spend to help shape a better tomorrow. But understanding ‘your tomorrow’ can be powerful and a great advantage for those that get it right. It provides the ability to respond rapidly to changing environments and will help predict the bending of your revenue curve… the positive signal many of us are hopeful won’t be faraway.
Stay safe and healthy.
Watch the ''What it Takes to Accurately Forecast Revenue ... conDati Shows You How' recorded webinar, to hear conDati VP of Revenue Marketing, Kelly McKeown and VP of Data Science, Iris Lieuw, discuss how to accurately forecast marketing revenue using machine learning.