vermorel 2 weeks ago | flag | on: Architecture of Lokad

The predictive optimization of supply chain comes with unusual requirements. As a result, the usual software recipes for enterprise software aren't working too well. Thus, we had to diverge - quite substantially - from the mainstream path.

The 5 trends as listed by the author:

  • 88% of small businesses supply chains will use suppliers closer to home by next year.
  • Small business supply chains are moving most or all suppliers closer to the U.S. faster than predicted
  • The strained economy and low inventory are top stressors
  • Software-based emerging tech is on the rise while hardware-based ones lag behind
  • 67% of SMB supply chains say their forecasting techniques were helpful in preventing excess inventory

As data were a byproduct and at best second concern for the businesses since the first introduction of ERPs and other enterprise software, it should not come as a surprise that the vast majority of data analytics and digital transformations fail. GIGO hits you when you don't know what are you looking for. Decisions, not the data should be primary focus when you start. Then it helps to narrow down your focus on improving only the data that matter.

Very interesting article! I always like to recall that risk is defined by (probability of occurrence) x (impact on the business). And human tend to underestimate high impact-low probability events and overestimate high probability-low impact events.

Very informative video! The development of digital twins - - is undoubtfully important. with all the benefits they bring, I think soon it will be almost a must-have for some businesses.

This problem is referred to as censored demand. Indeed, this is not the sales but the demand that is of interest to be forecast. Unfortunately, there is no such thing as historical demand, only historical sales that represent a loose approximation of the demand. When a product goes out of the assortment, due to stockout or otherwise, sales drop to zero, but demand (most likely) does not.

The old school approach to address censored demand consists of iterating through the historical sales data, and replacing the zero segments with demand forecast. Unfortunately, this method is fraught with methodological issues, such as building a forecast on top of another forecast is friable. Furthermore, in the case of products that are not sold during for long periods (not just rare stockout events), say summer, forecasting a fictitious demand over those long periods is not entirely sensical.

The most commonly used technique at Lokad to deal with censored demand is loss masking, understood as from a differentiable programming perspective. This technique is detailed at:

Hope it helps, Joannes

There are several questions to unpack here about seasonality. (1) Is seasonality best approached as a multiplicative factor? (2) Is seasonality best approached through a fixed-size vector reflecting those factors? (hence the "profile") (3) How to compute the values of those vectors?

Concerning (1), the result that Lokad has obtained at the M5 competition is a strong case for seasonality as a multiplicative factor: The literature provides alternatives approaches (like additive factors); however, this don't seem to work nearly as well.

Concerning (2), the use of a fixed size vector to reflect the seasonality (like a 52-week vector) has some limitations. For example struggles to capture patterns like an early summer. More generally the vector approach does work too well when the seasonal pattern are shifting, not in amplitude, but in time. The literature provides more elaborate approaches like dynamic time warping (DTW). However, DTW is complicated to implement. Nowadays, most machine learning researcher have moved toward deep learning. However, I am on the fence on this. While DTW is complicated, it has the benefit of having a clear intent model-wise (important for whiteboxing).

Finally (3), the best approach that Lokad has found to compute those vector values is differentiable programming. It does achieve either state of the art results or very close to start of the art with a tiny fraction of the problems (compute performance, blackbox, debuggability, stability) associated with alternative methods such as deep learning and gradient boosted trees. The method is detailed at:

Hope it helps, Joannes

Patrice Fitzner, who contributed to the design of the Quai 30, a next-gen 21st century logistical platform explains the thinking that went into this 400m by 100m monster of automation.

Last mile delivery services where attaining incredible valuations only a year ago. It seems as the honeymoon phase is over and the reality of a challenging route to profitability of the overall business model is setting in.
The model in itself is not new at all and the service is incredibly desirable from a customer perspective, however, the question remains why the big supermarket giants have not implemented it themselves 10 years ago already - maybe the business model is not as amazing as believed?
Future will tell whether clients will be willing to pay the premiums that the remaining 'monopolists' will have to charge to become profitable.

Factorio is the grandfather of factory simulation games, and it's centered around production chains, but the default play style is rather light in terms of supply chain (just create a factory that, if fed with raw materials for long enough, produces everything you need eventually). I recommend going for a train-based play style (especially against hostile aliens), as it tends to center a lot more around what you need to deliver, where, when, and in what quantities. The space exploration mod is also great on those aspects.

Satisfactory is also interesting for those who like more of a 3D feel.

My favorite is Dyson Sphere Program as it forces you to set up interstellar supply chains: in Factorio and Satisfactory one tends to create a central base into which all raw inputs are fed, but in Dyson Sphere Program it's usually better to transport your intermediate goods to a planet with oceans of sulfuric acid that are needed for its processing, rather than shipping the sulfuric acid back to a central planet. Or, to move energy-hungry processing to a tidally-locked planet right next to the sun.

Transport Fever 2 has excellent gameplay, with half of it being centered around industrial supply chains. The Industry Expanded mod is very good in this aspect The Cities: Skylines game with the Industries expansion is also quite good in the same vein.

Banished is also interesting in that you are expected to set up an entire medieval supply chain from scratch. It includes dealing with overfilled warehouses, random demand (weather for firewood, illness for medicine), investment (planting an orchard takes years to yield fruit), out-of-stock penalties (starvation, running out of tools which cuts productivity across the board), and so on.

Very nerdy Factorio rocks

My personal favorite remains
The entire Anno series is outstanding to understand basic concepts of raw material - semi finished - finished product transition + lead times.
Would be keen to know whether there are other games that come to mind!

Just to clarify the terminology that I am using the following: the EOQ (economic order quantity) is a quantity decided by the client, the MOQ (minimal order quantity) is a quantity imposed by the supplier. Here, my understanding is that the question is oriented toward EOQs (my answer below); but I am wondering if it's not about picking the right MOQs to impose to clients (which is another problem entirely).

The "mainstream" methods approach EOQs, especially all of those that promise any kind of optimality suffer from a series of problems:

  • ignore variations of the demand, which is expected to be stationary (no seasonality for example)
  • ignore variations of the lead time, which expected to be constant
  • apply only to "simple EOQ" that apply to a single P/N at a time (but not to a EOQ for the whole shipment)
  • ignore macro-budgeting constraints, aka this PO competing against other POs (from other suppliers for example)
  • ignore the ramification of the EOQs across dependent BOMs (client don't care about anything but the finished products)

Do not expect a formula for EOQs. There isn't one. A satisfying answer requires a way "to factor in" all those elements. What we have found at Lokad for better EOQs in manufacturing (not "optimal" ones, I am not even sure we can reason about optimality), is that a certain list of ingredients are needed:

  • probabilistic forecasts that provide probability distributions at least for the future demand and the future lead times. Indeed, classic forecasts deal very poorly with irregular flows (both demand and supply), and MOQs, by design, magnifies the erraticity of the flow.
  • stochastic optimization, that is the capacity to optimize in presence of randomness. Indeed, the EOQ is a cost-minimization of some kind (hence an optimization problem), but this optimization happens under uncertain demand and uncertain lead time (hence the stochastic flavor).
  • financial perspective, aka we don't optimize percentages of errors, but dollars of error. Indeed, EOQs is typically a tradeoff between more stock and more overhead (shipment, paperwork, manhandling, etc)

In my series of supply chain lectures, I will be treating (probably somewhere next year) the fine print of MOQs and EOQs in my chapter 6. For now, the lecture 6.1 provides a first intro into the main ingredients needed for economic order optimization, but without delving (yet) into the non-linearities:

It will come. Stay tuned!

vermorel 3 months | flag | on: Goodbye, Data Science

An incredibly perceptive discussion that reflects my own experience with data science in general.

vermorel 3 months | flag | on: Cycle Count Manager

A small side software project dedicated to inventory counting.

The earliest occurrence I could find of the concept is 2016 with the presentation:

The 2018's book is available at:

vermorel 3 months | flag | on: The Saga of Supply Chain Innovation

ATP (used in the article) stands for Available-To-Promise.

I am very much in agreement concerning the list of stuff that didn't work: Consolidations Decimated Value, Consultants Failed to Deliver Value Through Software Models, Barney Partnerships Bled Purple, not Green, The Saga of Venture Capitalists and Private Equity Firms, New Forms of Software Marketing Creates Haze not Value

Concerning the value of cloud and NoSQL. Well, yes, but it's a bit of an old news. Lokad migrated toward cloud computing and NoSQL back in 2010. A lot did happen since then. For a discussion about what a modern cloud-based tech look like

Excel is ingrained in day-to-day work of current working generation. It will be difficult to replace (barring select-few companies).

I strongly feel that post-generation-Z (born post 2001) workforce will expect (and work towards developing) better decision engines - powered by better technologies. This generation is using some form of AI/ML in day-to-day life, in schools, colleges. They wouldn't expect anything less in work. This will get progressively better.

vermorel 3 months | flag | on: Prioritized Ordering [video]

A couple of relevant links:

Hoehner 4 months | flag | on: Slower, lower, weaker [pic]

Parkinson's law is the adage that "work expands so as to fill the time available for its completion."

In most of Western Europe, my (tough) take is that, career-wise, those certifications are worth the paper they are printed on. The vast majority of the supply chain executives that I know have no certification.

More specifically, the example exam questions are ludicrous, see

MCQ (Multiple Choice Questions) is the exact opposite of the sort of problems faced by supply chain practitioners. MCQs emphasize super-shallow understanding of vast array of the keywords. Worse, it treats those keywords (eg data mining, integer programming) as if they were encompassing any kind of cohesive body of work (or tech). This is wrong, plain wrong.

vermorel 4 months | flag | on: Supply Chains are Healing

Minor: Edited the title to avoid the question mark. I am trying to reserve the question titles to actual questions addressed to the community.

This is great! We need to hear more stories about “the day of small beginnings” as I like to call it. Too often we read marketing buzz and press releases on the glory of a great achievement. That’s great and all, but every human achievement starts with people and there are usually some interesting dynamics around how something goes from zero to something. We need more stories around the underlying climate preceding the best innovations. THAT will inspire and spark excitement to build. There are plenty of lessons learned that can be shared in that regard (Paul Graham and Y Combinator essays do this). Who else has an early day supply chain story to share?

“In 2011, Amazon’s total revenue reached nearly $48 billion, and it was already clear to the senior leadership that the company’s scale would require the automation of buying and the management of inventory; monitoring spreadsheets was not a long-term solution. Indeed, even then the sheer range of products offered by Amazon meant the “illusion of control” was already kicking in among the groups managing inventory, says Bhatia. In fact, Bhatia notes, the sheer complexity and scale meant the challenge was beyond the scope of any team, let alone an individual.”

Light retrospective on the evolution of Amazon automated decision-making tech for supply chain. Interesting nugget, Amazon appears to be still using their multi-horizon quantile recurrent forecaster (1) as it appears to have taken several years to cover the full scope (which is not unreasonable considering the scale of Amazon).

(1) A multi-horizon quantile recurrent forecaster
By Ruofeng Wen, Kari Torkkola, Balakrishnan (Murali) Narayanaswamy, Dhruv Madeka, 2017

The book can be purchased from

The main message by Schuh et al. is that a collaborative relationship with suppliers can be vastly more profitable that an oppressive one solely focused on lowering the supply prices. While the idea isn't novel, most companies still favor aggressive and antagonistic procurement strategies which leave no room for more profitable collaborations to emerge.

I recommend Jason Miller (MSU SCM head) and Warren Powell’s posts from LinkedIn. Both are widely recognized thought leaders in the quantitative SCM space.

10 years ago, Amazon was acquiring Kiva Systems for $775 million.

The quote is from The Testament of a Furniture Dealer by Ingvar Kamprad, IKEA founder. The original document can be found at:

Forecasting and S&OP initiatives almost invariably turn into bureaucratic monsters.

A team from Lokad took part in the M5 competition. The method, which landed No1 at the SKU level, has been presented at

Hoehner 5 months | flag | on: Flaw of Averages Trilogy

The average project completion time is greater than what is predicted by the average durations of the underlying tasks, because the project is not done until the last task is done.

The average profit is less than the profit of the average demand, because there is no upside if demand exceeds quantity ordered.

The average operating cost is greater than the operating cost of the average demand.

Yes, continuous improvement is the key to make a new technology become economically efficient, but also it may become enabler as you've mentioned. Though, by breakthrough we usually mean a change of principle how the same objective is achieved better/faster/cheaper etc. This is qualitative change in the first place. Then continuous improvement starts.

As written in the article, you have to choose between two models when developing a startup : either grow at all costs, and optimize later (like Uber), or keep the notion of cost and frugality as an essential part of every initiative. Amazon is doing that in its warehouses : Grow, but make sure to optimize for efficiency.

On the other hand, groundbreaking innovations are often made possible by incremental innovations in other areas (such as the improved precision of steel machining tools allowing the creation of cost-efficient gasoline engines).

Hoehner 5 months | flag | on: Supply Chain News on

A video explaining why this forum was launched.

While Reddit targets a young category of users and Wikipedia is bound to contain old and somewhat deprecated information, LinkedIn remains one of the only trustworthy source of information in the field, due to the experience of the professionals that are sharing their experiences. However, this information does not remain easily accessible after a while. Therefore, it would be interesting to have a place to discuss the specifics of the Supply Chain field.

For this purpose, Lokad has launched a new page called, an aggregator that consolidates interesting links to various Supply Chain topics and where the community can discuss and submit their own subjects. The final goal is to share links of interest for Supply Chain-minded people, giving them the chance to debate and contribute to further achievements in the field.

vermorel 5 months | flag | on: Software to simplify the supply chain

Interesting nuggets of this interview with Ryan Petersen, CEO of Flexport:

- 20% of the Flexport workforce is software engineering. The rest is sales and account management.
- The P95 transit time is a 95% quantile estimate of the transit time; part of the core Flexport promise.

Overall, a very interesting discussion, although the simplify part really refers to the Flexport product itself.

Most supply chain initiatives fail. Dead-ends are a given, although my understand differs a little bit concerning the root causes. Among the top offenders, the lack of decision-centric methodologies and technologies ranks very high. In the 'future' section proposed by the author, I see layers of processes to generate ever more numerical artifacts (eg: Market-driven Demand, Demand Visibility, Baseline Demand and Ongoing Analysis of Market Potential, Unified Data Model Tied to a Balanced Scorecard, Procurement/Buyers Workbench).

vermorel 5 months | flag | on: puts itself up for sale

A decade ago, - along with a couple of similar ecommerces - took extensive advantage of the payment terms of their oversea suppliers (mostly in Asia). Their supply chain execution allowed them to sell their goods while goods were still in-transit. This worked well for furniture as customers - at the time - were OK waiting a month or two to receive their order. I don't know where they stand now, but I suspect that the supply chain tensions (sourcing problems in Asia + surge of transport fees) do pose significant challenges to this business model.

If we look at other fields we can realize safety stock will never get buried completely.
We have modern medicine, but healers still do exist.
We have modern agriculture, but growing 'organic' movement (which is in fact less sustainable).
Etc, etc, etc.
Those advocating for "supply chain is not for tech, but for people" will always seek for dysfunctional but simplistic recipes.

My previous take on safety stocks:

In short, not only the normal distribution assumption is bad, but the whole approach is very naïve. It made (somewhat) sense before the advent of computers, but at present, safety stocks should be treated as a method of historical interest only.

vermorel 5 months | flag | on: Differentiating Relational Queries

This work is done by Paul Peseux who is currently doing a PhD at Lokad. In terms of research, it's a convergence between machine learning, mathematical optimization and compiler design; fields that are usually considered as fairly distinct - but that end up being glued together in the context of differentiable programming.

Inflation pressure in Europe may not only be fueled by increased energy prices but in addition by increased logistics costs.

tikhonov 5 months | flag | on: Demand sensing [pic]

Fully agree. The only case which might be relevant for "real time" data is dynamic repricing for SKUs pending stock out or promotion/discounting within the day for perishable goods. This might be relevant for e-groceries and other e-commerce businesses with tight competition where they closely tracking each other and have overlapping highly substitutable assortment.
For replenishment and purchasing "real time" data is overkill and waste of compute power. Daily batches are enough.

richlubi 5 months | flag | on: Demand sensing [pic]

Demand sensing = marketing tosh; nothing new or clever . . . I can't see evidence to the contrary.

Not to say, of course, that there is no benefit in getting an earlier feed of data on something such as demand, if - and only if - it can result in a better decision that is executed earlier than otherwise - giving an economic benefit.

The use cases do exist though. My (real) example is in the supermarkets world, specifically on the day that a new set of weekly promotions start. On this 'day one' of the promotional week regular and promotional pricing are reset, promotional 'aisle ends' are built as well as other changes in layout being implemented. For all the forecasting that is done beforehand, the actual sales by 11:00 on the first day provide a very accurate view of how all the changes - overlayed with each other - are being interpreted by the customer, resulting in sales. This early read on sales - which is net of all the inter-SKU relationships - 'could' result in a different POs being placed with suppliers a day earlier than if the daily sales were interpreted in an overnight batch process. This 'could' result in availability and sales better in line with demand, and fewer overestimates/overstocks in store which are also negative in terms store productivity. The challenge in this example is not so much 'the software' but rather the data flow and the timing of the execution of the calculation (e.g. purchase order calculation), in line with other time windows that need to be hit in order to get a PO out a day earlier. In this sense there is no difference between 'real time' and 'timely' - e.g. a batch process is perfectly OK as long as it can be executed quickly and when you need it.

tikhonov 6 months | flag | on: Demand sensing [pic]

Probabilistic forecasting should be considered as state of the art for supply chain planning.
Though, as we recently have shown it is doable to explain supply chain decisions optimization with use of probabilistic forecasts (we explained it using purchasing decisions) even in Excel:
If there is a substance behind Demand sensing - looking forward to see the Excel with simplified approach showing what value this approach adds.

vermorel 6 months | flag | on: Demand sensing [pic]
impossible to get a real sense of what's offered without some sort of PoC

Fully agree. This is why I abhor those made-up terminologies: it's pure vendor shenanigan's.

StefJensen 6 months | flag | on: Demand sensing [pic]

The 'real-time' part is an overstatement for sure. What they offer are batch job controlled short term (deterministic) forecasting.
All these buzzwords in planning/forecasting software is just annoying as a customer - its absolutely impossible to get a real sense of what's offered without some sort of PoC. Because only then you start to have the talks that shows, what's under the hood.

tikhonov 6 months | flag | on: Demand sensing [pic]

Demand sensing article on Wikipedia:

Compare it with probabilistic forecasting for instance:

vermorel 6 months | flag | on: Demand sensing [pic]

Q: Why a new buzzword if it's about repackaging techniques that already have proper names?
A: Occam Razor: to make the tech appear more attractive and valuable than it really is.

According to [1], SAP recognizes 'demand sensing' as 'a forecasting method that leverages [...] near real-time information to create an accurate forecast of demand'.

  • Why would a 'near real time' provides a forecast that is any less accurate than a batch forecast happening with a lag of, say, 10min?
  • Why should gradient boosting be even considered as a relevant technical solution for 'near real-time' tasks?

Remove demand sensing from the picture, and you still have the exact same tech with the exact same processes.


Hoehner 6 months | flag | on: The beer game

It would be cool to have a Lokad version of the beer game. I.e. focusing on the elements of the stock reward function

When I look at the market, I see major contributions of GroupThink:
- Failure of IT Standardization. SAP and IBM failed the market. The recent gains in market share of Kinaxis, o9, and OMP are largely due to the failure of SAP to drive thought leadership in planning.
- Private Equity M&A. Software mergers & acquisitions also slowed innovation. The technology roll-ups of INFOR, JDA (now BlueYonder), and E2open improved investors’ balance sheets, but did not drive value for their clients.
- Event Companies Are the Nemesis of the Industry. Event companies take large sums of money from technology companies and host events based on the Rolodex of a prior supply chain leader

A spot-on analysis. Low level IT standardization is moving forward nicely (think federated identity management), but it's not the case for high level IT (think workflows). The success of products like Tableau reflects that there is a major need to cope with the lack of standardization.

M&A in enterprise software is almost always resulting in large about of technological debt. It's very hard to get good software engineers motivated about clean-up millions of lines of code of haphazard codebases where stuff has just been "thrown together".

Event Companies are a severe form of epistemic corruption. I discussed the case in

As it was suggested in, there is also the visit of Amazon warehouses in different countries

I tried visiting their American facilities, it was great!

Interesting, would be cool to have a collection of warehouse visits to give new professionals a better idea of what an actual warehouse looks like. I came across this one from Maersk

The sheer scale of this piece of engineering is incredible. We are talking of a machine that is 400m x 100m x 12m big. The whole thing maximizes that can be done in terms of inventory storage and inventory throughput while taking advantage of every cubic meter that is available.

Digital talent remains the Achille heel of many (most?) supply chain initiatives. Nonsensical tech decisions keep being made due to a lack of understanding of what is at stake. The challenges pointed out in this post a few years back have only become more acute since then.

Q5: There is a comment in the file recommending using this approach for educational purposes only, because such a forecast will be simplistic and will not account for seasonality or trends etc. How to use this approach beyond educational purposes for real inventory optimization problems?

A5: First, it should be noted that probabilistic forecasts in the file are synthetic probability distributions that are built using Excel functions.
It is possible to build probabilistic forecast that would be based on historical sales data and would take into account all systematic causes of variation like seasonality, trend etc.
We intentionally left it behind the scenes for two reasons:
1) We would have to provide historical sales data that probabilistic forecast would be based on. Here we have a problem that all users are from different industries and got accustomed to see different sales patters. We wanted to avoid "this doesn't look like my sales data" kind of impression.
2) Building production grade probabilistic forecast in Excel is technically speaking a nightmare and an open problem since we've never seen anybody done it so far. Taking into account Excel capability limitations it should be also very slow to rebuild the forecast once you update the data.
At Lokad we use domain specific programming language called Envision where probabilistic forecast can literally be build with several lines of code. Those who are interested about the procedure can play the recording of our past supply chain lecture:

Q4: Can this approach of decisions prioritization with expected ROI be used for allocating scarce products from DC to stores?

A4: Absolutely.
This file was about purchasing optimization, but the same approach could be applied for DC to stores dispatch problem. ROI ranking perfectly works when there is limited stock at DC and inventory manager wants to make sure that dispatch plan is optimal.
The way it works is the first unit of the first SKU (can be arbitrarily chosen, but can also start from top sellers) and ask in which store (given current stock in each store and probabilistic demand forecast for this SKU for each store) it will have the highest ROI and/or fill rate gain? Then you rinse and repeat for all SKUs at a unit level for all the units (same as with purchasing). This enables to best allocate the existent stock on hand across all the stores in the network. Ranking procedure here is a bit different because at every step we make final decision to which store we send Nth unit of a given SKU. At the end of the process after considering all micro level decisions we chose termination criteria (no more stock at DC or target ROI or target fill rate etc.) and aggregate numbers per store per SKU to get dispatch lists.

Q3: What goes into engineering economic drivers? How critical it is to get them right from the very beginning or is it more like learning curve?

A3: Decision making using economic drivers in combination with probabilistic forecasts to some extent imitates human level decision making process. Costs and revenues are straightforward, but the stock out cover is not. It is completely context dependent and can be called heuristics. This is where human intelligence lies. Having them properly set enables to automate decision making so that there is no more need to say split 500k SKUs among 50 demand planners to manage 10k SKUs each. All decisions for all SKUs can be generated automatically by the single numerical recipe. Litmus test here is simple - if demand planner sees no insanity in decision recommendations, then it should be concluded that economic drivers are production grade, but for sure can be fine tuned over time.

Interestingly enough the EU exports approximately 7 million metric tons per year - which is about 4% of the production - thus the demand/supply gap in Europe shouldn't be too significant in the short term.

EU steel plants suffer from weak demand and high energy costs, decreasing competitiveness of EU steel producers. Shut downs are across all EU countries. However, in comparison to 2022 the steel production is only reduced by 6.7%

The stochastic gradient descent (SGD) is used for a whole variety of supply chain problems from demand forecasting to pricing optimization. From a software performance perspective, the crux of the SGD problem is to increase the wall-clock rate of descent while preserving the determinism of the execution. Indeed, as far as parallelization is concerned, indeterminism is the default; it takes effort to achieve a reproducible flavor of the algorithm. The report introduces a technique that delivers a 5x speed-up at a 6x increase of compute costs.

10 years is a good ballpark assessment to produce a good software product - assuming there are people who will stick around for a decade to see it through. See Written 20 years ago, but the points are still largely valid.

Acknowledgement from that the probabilistic approach is superior, or at least will offer more resilient decisions. How long to get to market in a core product that actually works? 10 years?

vermorel 6 months | flag | on: Conformal prediction materials

Conformal predictions are one of the flavor of probabilistic forecasting, leaning toward high dimensional situations. This repository is an extensive compilation of the papers, phds and open source toolkits that are available for conformal predictions.

The blog post talks about range forecasts and probabilistic forecasts. Also they talk about relatively small number of scenarios ("at least 5-10...") so it brings the question of accuracy with such small resolution.
Feasible decisions ranking and economic drivers are absent while they are essential components for choosing decisions with highest return on investment.
Level of details is underwhelming so far.
Though this is great news. They will definitely accelerate the adoption of the term.

We strongly believe that probabilistic methods play a key role in the future of supply chain planning. They represent the right way to go for areas that contain uncertainty, including for all types of forecasting and the subsequent planning situations.
Traditional deterministic planning methods base their decisions on the mistaken assumption that uncertain values can be approximated by a single average number. As a direct consequence of this assumption, plans are often infeasible at the time they are created and manual interventions are continuously needed.

Better late than never! However, let's immediately point out that the SAP IBP architecture is very much hostile to probabilistic modeling. More specifically, the high memory consumption of HANA is going to add some massive overhead on top of methods that not exactly lightweight in the first place.

tikhonov 6 months | flag | on: Proxy Variable [pic]

Another metric of this kind would be stock levels expressed in dollar value or in days of sales. Very easy to cut and show "benefits", especially if you are in long lead time business. Cut them, update your CV and make diagonal jump to another business. Profit :)

vermorel 6 months | flag | on: Proxy Variable [pic]

Service levels are probably the favorite proxy variable in supply chain. Supply chain textbooks and consultants assume that "finely tuned service levels" automatically translate into better outcomes for the company, while those service levels say very little of substance about the quality of service actually perceived by customers.

arkadir 6 months | flag | on: Proxy Variable [pic]

It's tempting to solve an easier problem only superficially related to the actual problem.

One of the ways bureaucracies are ignoring difficult problems is by division of labor. Once the scope of the problem is broken up across multiple roles it becomes nobody's business. Calling cross-functional teams to rescue the business is not a solution as usually those teams are composed of multiple within-silo thinkers who can't abstract from their role and look at the problem as a whole.

I had the chance to do a guided tour yesterday evening, for their American facilities. It was great! I was surprised to be (almost) the only one asking questions. They took the time to answer all of them, not very extensively I acknowledge but the effort was there. I would love to visit their French warehouses now.

Supply chains are complex systems. It's maddeningly difficult to solve problems rather than displacing them. When confronted with incredibly difficult problems, bureaucracies are also incredibly good at ignoring them altogether. In supply chain, big problems are usually big enough to take care of themselves.

How far should we go to say that we have reached a point of say realistic representation of an agent? Also when we say accuracy , what does it really mean?

Right now, as far as my understanding of the supply chain literature goes, there is just nothing yet published to tell you whether a simulation - in the general case - is accurate or not. The tool we have for time-series forecast don't generalize properly to higher dimensional settings.

For example, if a simulator of a multi-echelon supply chain of interest is implemented, and then someone decide to refine of the model of some inner agent within the simulator, there is no metric that are even known to be able to tell you if this refinement is making the simulator more accurate of not.

Stay tuned, I am planning a lecture on the subject in the future, it's a big tough question.

Thanks Joannes, would it make sense to have more intelligent agent behavior? How far should we go to say that we have reached a point of say realistic representation of an agent? Also when we say accuracy , what does it really mean? Is it like ,say , we predicted something up or down and we are correct 90 times out of 100 so our accuracy is 90%?

Holy, surprisingly its popular in south korea which was unexpected for me.

Hoehner 6 months | flag | on: Live reporting of Port congestions

Interesting to see that that the major congestions that were deemed to create backlog for all 2022 have mostly vanished

Lokad tries to push a lot of (hopefully) quality supply chain materials in the open. Unlike many vendors, we don't attempt to shroud our technology in a veil of mystery. However, we need backup. If you think that you help us produce videos, guides, articles ... then, drop your resume at

Q2: After we made and executed purchasing decision there is no more uncertainty, then how would we evaluate whether the predicted probability forecast is the correct one?

A2: Even after we made and executed the decision in reality there is still uncertainty. We don't know when our order will be received because lead time is also probabilistic. Quality of goods can be probabilistic as well (consider fresh food for instance). Also we will not be able to estimate accuracy of our probabilistic demand forecast until the responsibility window (over which we built probabilistic forecast) becomes the past.
With respect to the metrics that are used for probabilistic forecast quality assessment you can check this lecture:

Use time stamps to watch the lecture parts where the metrics were discussed.

Answering a question on YouTube:

As per my understanding the following are the core concerns -
1) Accuracy
2) Doesn't necessarily represent reality based on Agent behaviour
3) Gives me insights, ok now what should i do? Don't give me numbers, tell me what to do. If an employee sits down and tweaks parameters then how do i make sense if the decision is correct?

Yes, in short, the two big gotchas are (a) your digital twin may no reflect the reality (b) your digital twin may not be prescriptive.

Concerning (a), measuring accuracy when considering the modeling of a system turns out to be a difficult problem. I intend to revisit the case in my series of supply chain lectures, but it's nontrivial, and so far all the vendors seem to be sweeping the dust under the rug.

Concerning (b), if all the digital twin delivers are metrics, then, it's just an elaborate way to waste employee's time, and thus money. Merely presenting metrics to employees is suspicious if there is no immediate call-to-call. If there is a call-to-action, then let's take it further and automate the whole thing.

This is brief summary of my Q&A with one of the users of Excel file. I will provide each question and answer as separate comment.

Q1: How are we creating the probability distributions for the products in this file? If I have historical sales data for a particular SKU, how should I get the probability distribution for that?

A1: We haven't made capabilities to build probabilistic demand forecasts using historical data in this Excel file. This is very hard to implement in Excel due to its limitations. Programming language like Envision is more appropriate for that, but one could also use Python or any other language. How to build Probabilistic forecast with historical data was discussed in this lecture:

This file uses synthetic distributions via built-in Excel functions for normal and negative binomial distributions. Changing parameters you can change the distributions and the ranking of micro level decisions respectively. This is educational tool and the primary goal was to show how having probabilistic demand forecast and economic drivers demand planer can optimize purchasing decisions.
Though, the question of building probabilistic forecast based on historical data still remains valid. It is just not in the scope of this document because it is harder to understand, but also harder to explain and show through Excel.

Indeed. Although, as aircrafts get dismantled, it tends to introduce a lot of spare parts into the market. Thus, most of the time, the parts of weaning aircraft types become cheaper despite the lack of production of parts. However, as you correctly point out, there are parts that become rare and very expensive, making the aircraft type economically unviable.

Making noise and making money out of the noise, especially consistently, is not the same. I think that growing number of companies that adopt probabilistic perspective through appropriate software is the only adequate metric that tells anything about the adoption of the term.

That's is interesting. At some point there will be very few aircrafts of this model so servicing parts for them will become nightmarish as demand will be even more sparse than it is now meaning the parts rotation will slow down and carrying cost at some point will make it unprofitable for those remaining aircrafts to fly.

A nice illustration the sort of stuff that characterizes aviation supply chain: aircrafts are both expensive and modular. Thus, the option is always on the table to take a component from an aircraft and move it to another aircraft. Most of time, exercising this option is pointless, but sometimes, it's an economically viable move. Here, this is what Boeing is doing with aircraft engines. Aviation supply chains are not about picking safety stocks :-)

Working in the probabilistic space, it feels that the term is becoming more and more mainstream and anecdotal evidence confirms that. However, looking at google searches this is not at all confirmed. Maybe the topic isn't as mainstream as we feel - confirmation bias?

Funfact: Since 2012 the interest for the topic has x 100

Fun fact: Lokad started to implement digital twins of supply chains more than a decade ago; although I don't overly like this terminology. As a rule of thumb, I tend to dislike terminologies that try to make tech sounds cool, irrespectively of the merit of said technology. There are tons of challenges associated with large scale modeling of supply chain, the first one being: how accurate is my digital twin? Tech vendors are usually exceedingly quite about this essential question.

747 have been produced for 54 years. The one most notable evolution being the introduction of the fly-by-wire tech in the 1990s

This plane has massively contributed to the democratization of both air travel and air shipments. Considering that aircrafts are typically operated for decades, some 747 are likely to keep flying for the next 20/30 years.

Lion Hirth is Professor of Energy Policy at the Hertie School. His research interests lie in the economics of wind and solar power, energy policy instruments and electricity market design.

The document introduce marginal pricing - in the context of energy, and make three statements about it:

Marginal pricing is not unique to power markets.
Marginal pricing is not an artificial rule.
If you want to get rid of marginal pricing, you must force people to change their behavior

Three points are very much aligned with what is generally understood as mainstream economics. Those points are quite general and do apply to most supply chains as well.

I am not familiar with the specific Greek energy market.

However from a supply-and-demand perspective,

  • Intermittent energy sources do not meet the Quality of Service requirement which is essential for energy.
  • As long as the source can be turned into electricity, all energy sources are near perfect substitutes. Hence, one cannot isolate the price of a selection of sources vs the rest.
  • When demand is inelastic and grows, as it is the case for energy demand, it's the supply that has to grow.

Due to spiking electricity prices several stakeholders are arguing that the electricity market is malfunctioning and the pricing mechanism is flawed. The merit order model, that attributes the marginal (highest production price) to all producers is nothing else than the offer/demand model that we apply in all other markets as well.

vermorel 6 months | flag | on: Lokad is hiring a Supply Chain Scientist
The Supply Chain Scientist delivers human intelligence magnified through machine intelligence . The smart automation of the supply chain decisions is the end product of the work done by the Supply Chain Scientist.

Excerpt from 'The Supply Chain Scientist' at

Transit costs to low orbit are still beyond the realm of supply chain, however, it is notable the cost per kilogram has been going down by a factor 1000 over the course of 70 years. If progress keeps happening at the same pace, in a few decades, launches will become an option. The benefits of easier access to low orbit are somewhat unclear beyond telecommunications, but specialized micro-gravity factories has been explored many time in science fiction. At this point, orbit remains too expensive to even try to investigate newer / better industrial processes in orbit.

vermorel 6 months | flag | on: How to Measure Forecastability (2021)

The only way to assess "forecastability" of a time-series is to use a forecasting model as a baseline. This is exactly what is done in the article, but unfortunately, it means that if the baseline model is poor, the "forecastability" assessment is going to be poor as well. There is no work-around that.

Stepping back, one of the things that I have learned more than a decade ago at Lokad is that all the forecasting metrics are moot unless they are connected to euros or dollars attached to tangible supply chain decisions. This is true for deterministic and probabilistic forecasts alike, although, the problem becomes more apparent when probabilistic forecasts are used.

StefJensen 6 months | flag | on: How to Measure Forecastability (2021)

Many articles discuss how to measure forecastability for deterministic forecasting. However alot for in the trap of suggesting to simply use a coefficient of variation (CV) measure - even though it will count forecastable patterns as season and trend as variation and therefore mistakenly set them as hard to forecast.

The linked article here by Stefan de Kok does a good job of explaining the trap of pure CV and come with an alternative.

I'm a bit split though whether to use this type of measure or to compute the FAA of a simple benchmark (such as a moving average).
The FAA gives you a minimum acceptable accuracy level, but the proposed method here gives a measure which (typically) can be reported from 0 (unforecastable) to 1 (no noise).

Do any of you here have experience in implementing this and can share any experiences? Especially on the stakeholder/change management side.