Predict Consumer Demand in COVID 19 with a Short-Term Demand Forecasting Model Using ML

COVID-19 is lingering on and has become a “new normal.” More and more regions and countries begin to reopen their economy, but the consumer sentiments are mixed due to uncertain future as COVID-19 case continue to increase.  Business globally are seeing this new normal being an uncertain phase and are trying to adapt to the new reality. The COVID-19 pandemic has fundamentally changed the world as we know it. People are living differently, buying differently and in many ways, thinking differently. Supply chains have been tested. Retailers are closing doors. Consumers across the globe are looking at products and brands through a new lens. The virus is reshaping the industry in real time, rapidly accelerating long-term underlying trends in the space of mere weeks.

Consumer behavior is seeing a dramatic shift in uncertain times

According to recent consumer surveys conducted by Numerator in US, 90% of respondents said that they have experienced a change in their shopping behavior due to the pandemic and 32% of respondents continue to stockpile goods that they usually don’t. As the stockpiling continues, out-of-stock has emerged as one of the most serious issues, with 65% consumers experiencing a shortage of products. News reports and social media posts showing empty shelves have exacerbated the issue, and CPG manufacturers and retailers are challenged in keeping up with demand, especially as the effects of the coronavirus become more wide-reaching.

Shopper behavior has changed, and they have moved to online shopping to avoid contact.  There is a growth in online grocery shopping, with 11% of online shoppers making their first purchase in last 6 months and an additional 6% making their first purchase ever. However, the sentiments towards continuing online shopping post the pandemic remains mixed with many customers planning to return to stores for grocery shopping. Survey respondents expect a 3 to 8% increase in store visits post Covid-19 for grocery shopping, but a 7 to 9% decrease for non-grocery products.

McKinsey global consumer survey report an expected increase of 15 to 30% in expenditure on groceries and up to 15% on household products during the pandemic. While many welcome the increased demand, however, meeting the consumer demand has its own set of challenges. Manufacturers need to assess how to best mitigate the risks of supply chain disruptions and capacity limitations.

CPG supply chain is stretched thin

The sudden & dramatic shifts in consumer behavior and buying patterns has impacted the CPG supply chain and its stretched thin due to:

• Slowdown in production due to gathering restrictions and slower movement of goods
• Conventional just-in-time inventory & replenishment not able to handle the disruption.
• Shortage of raw material
• Unplanned shutdown of plants for sanitization following infection among workers and
• Increased worker absenteeism

Understand the dynamic consumption patterns using internal and external data

The rapidly changing market has increased difficulty for sales and operational planning. The changing product preferences, new consumption patterns and shift in demand channels, handicap the existing inventory planning and demand and sales forecasting models during the pandemic as well as during the post-pandemic recovery period. Hence, to capture the new trend in consumption during and post forced lockdown state, we need to look at potential sources of data for capturing the dynamic consumption pattern. Following are few potential sources of data which could provide information on dynamic consumption pattern during this situation:

• POS Data: Point of Sales data for a few of the Key Accounts/Online Vendors with respective market share will help in deducing the new behaviour of these accounts.
• Weather: Consumer-based businesses should be paying greater attention to the weather as about 5% of their total annual revenue directly impacted by the weather, with much higher sales variability in seasonal categories. Based on research by the American Meteorological Society, current U.S. economic output varies by up to $630 billion a year (about 3.4% of 2016 gross domestic product) due to weather variability.
• Consumer Sentiment: Information like Consumer Confidence, Consumer Credit, Consumer Spending, Disposable personal income, Households debt to GDP and Consumer Savings can contribute a lot to understanding the consumer situation.
• Macro-Economic Data: Information like Consumer Price Index (CPI), Producer Price Index (PPI),  Economic Activity Index, Unemployment Rate, GDP from various sectors, GDP Growth Rate, GNP, Labour data and Housing Index will provide information on the economic situation of the region.
• Disease spread / mortality rate: Information on the number of Coronavirus Cases, Deaths and Recovered will be a strong influencer on how the customers will react to this outbreak. For example, in case of a sudden large spike of the COVID-19, customers might be motivated to stockpile items.

Develop a short-term demand forecasting model using ML

Due to COVID-19 pandemic, the true consumption pattern might have shifted as compared to historical consumption pattern by Categories, Sub-Categories and Region. Even there could be lot of variations across different types of Stores as well – Large, Medium, Short Format stores. So, it makes little sense to observe past historical data for a long duration. Hence using the mentioned data sources, we would develop a short – term Demand Forecasting model to generate potential demands of different Category/Sub-Category/SKU level at daily level by Geo-Spatial dimensions and Store Type using sophisticated Statistical/Machine Learning algorithms that would provide significant confidence level of forecast accuracy. This would enable the Trade distribution, Supply Chain and Manufacturing functions to take right steps and measures to cater to those future demands with high degree of confidence by reacting to those demand signals.

The core component of the model might be a traditional forecasting models like;

1. Multiple Linear Regression model with ARMA errors,
2. Box-Jenkins ARIMAX (Auto-Regressive Integrated Moving Average with suitably estimated parameters from the data)
3. More sophisticated algorithm like Dynamic Linear/Non-Linear Models – Gaussian State Space Models with Kalman filters which essentially includes models like UCM – Unobserved component Model, Bayesian Structured Time Series Model, Prophet
4. Highly computationally intensive models from the class of Deep Learning Models like LSTM using daily historical POS data in the presence of other data sources as mentioned earlier along with the data representing consumer behavior.

All these models are capable enough in producing accurate forecasts with reliably high degree of confidence when applied to short term time series data, provided enough variation in trend and seasonality exits in the data. Final model for production usage would be decided based on the accuracy metrics and model robustness.

Consider a small historical window for forecasting models

The main difference between the traditional models used so far in the industry are quite prominent. While standard forecasting models consider a sufficient historical timeframe to capture the long-term trend, we will consider a small historical window, say last 3 months. This is because due to COVID-19, the trend and pattern of total sales would have shifted dramatically. Hence it makes little sense to observe past historical data for a long duration. Also, an in-depth region-specific analysis of the causal relationship between COVID-19 needs to be carried out. It is to be remembered that the overall COVID-19 trend of a country is not enough to capture the local trend. For places which have a high spread of COVID-19, the average value per transaction (ATV) will be high due to bulk shopping, but the visit count will be low, as customers will be reluctant to leave their homes. Whereas for areas where COVID-19 spread is low, the buying pattern might be more regular. So, the causal relationship analysis must be more granular. For each of these regions, a separate model needs to be created after accommodating the influence of COVID-19.

So, in conclusion the Short-Term Demand Forecasting model should primarily composed of 3 parts:

1. Downstream Data Integration: Analyzing point-of-sale data from different regions, markets, brands and distribution channels to better understand consumer behavior.
2. Measuring the Impact of Demand Shaping Actions (DSA): Recording and determining the impact of so-called demand shaping events like COVID19 in this case.
3. Latency Reduction (LR): Model demand more frequently — weekly, or even daily

Our short – term Demand Forecasting model is more practical in the sense that it is more attuned to the current scenario and changing dynamics incorporating rational downstream data integrations, capturing demand shaping events Through frequent model recalibration including newer data, the model would deliver superior performance in forecasting future demand.

Do get in touch with our Data Science team and CPG SMEs who has decades of experience in building CPG domain-specific analytical models. We have been helping world’s largest CPG companies with their customer insights, supply chain planning, sales/trade and distribution planning and operations. We can help you thrive and revive your business in this new normal.

Author:
Sumit Mukherjee
Sumit is a Principal Data Scientist

Anindya Neogi
Anindya is GM, Chief Data Scientist at ITC Infotech

Reference:

1. https://www.nielsen.com/us/en/insights/article/2009/h1n1-impact-implications/
2. https://www.numerator.com/resources/blog/impact-covid-19-consumer-behavior
3. https://www.mckinsey.com/business-functions/risk/our-insights/covid-19-implications-for-business