Hello,

I have a demand forecast implementation with the following code to determine the dispersion and also to predict the future demand based on the probability distribution of the past demand.


///Dispersion calculation-------------------------------------------------------------------------

ItemsWeek.IsPast = ItemsWeek.Monday < monday(todayForecast)
ItemsWeek.FutureWeekRank = 0
where not ItemsWeek.IsPast
  ItemsWeek.FutureWeekRank = rank() by Items.Sku scan - ItemsWeek.Monday

ItemsWeek.One = dirac(1)
Items.Baseline = avg(ItemsWeek.Baseline) when (ItemsWeek.FutureWeekRank > 0)
Items.Alpha = if Items.Baseline < 10 then 0 else 0.05
ItemsWeek.RatioOfError = if ItemsWeek.Baseline != 0  then (ItemsWeek.Baseline - ItemsWeek.DemandQty) ^ 2 /. ItemsWeek.Baseline else 0
Items.AvgErrorRatio = avg(ItemsWeek.RatioOfError) when (ItemsWeek.IsPast)
Items.Dispersion = max(Items.AvgErrorRatio/2, 1)

And follow by the actionrwd.segment implementation for the future weeks prediction: 

where ItemsWeek.FutureWeekRank > 0
  Items.SLTDemand = actionrwd.segment(
                      TimeIndex: ItemsWeek.FutureWeekRank
                      BaseLine: ItemsWeek.Baseline
                      Dispersion: Items.Dispersion
                      Alpha: 0.05
                      Start: dirac(1)
                      Duration: Items.ActRwdSLT
                      Samples : 1500)

  Items.WOODemand = actionrwd.segment(
                      TimeIndex: ItemsWeek.FutureWeekRank
                      BaseLine: ItemsWeek.Baseline
                      Dispersion: Items.Dispersion
                      Alpha: 0.05
                      Start: dirac(1)
                      Duration: dirac(Items.ActRwdReorderLeadTime)
                      Samples : 1500)

  Items.TotalDemand = actionrwd.segment(
                      TimeIndex: ItemsWeek.FutureWeekRank
                      BaseLine: ItemsWeek.Baseline
                      Dispersion: Items.Dispersion
                      Alpha: 0.05
                      Start: dirac(1)
                      Duration: dirac(Items.ActRwdReorderLeadTime) + Items.ActRwdSLT
                      Samples : 1500)

What will cause the actionrwd.segment function to have dispersion value that is unreasonably high error? And what is the limit of the dispersion value that Envision allows?

Thank you and please let me know if you need more information to assist with this question.

s40racer Jul 25, 2023 | flag

Code before dispersion:


ItemsWeek.ItemLife =  1
ItemsWeek.CumSumMinusOneExt = 0
where ItemsWeek.Monday >= firstDate
  ItemsWeek.CumSumMinusOneExt = (sum(ItemsWeek.ItemLife) by ItemsWeek.Sku scan ItemsWeek.Week) - 1
ItemsWeek.CumSumMinusOneExtMonth = ceiling(ItemsWeek.CumSumMinusOneExt / 8)
ItemsWeek.WeekNum = rank() by Items.Sku scan -monday(Week)
nbWeeks = same(ItemsWeek.WeekNum) when(ItemsWeek.Week == week(today()))
ItemsWeek.ItemLifeWeight = 0.3 + 1.2*(ItemsWeek.WeekNum/nbWeeks)^(1/3) 
ItemsWeek.IsCache = ItemsWeek.Monday >= firstDate and ItemsWeek.Monday < today
ItemsWeek.Cache = if ItemsWeek.IsCache then 1 else 0
expect table Items max 30000
expect table ItemsWeek max 5m
table YearWeek[YearWeek] = by ((Week.Week - week(firstDate)) mod 52)
Items.SeasonalityGroup = Items.Category
table Groups[SeasonalityGroup] = by Items.SeasonalityGroup
table SeasonYW max 1m = cross(Groups, YearWeek)
Items.Level = avg(ItemsWeek.DemandQty) when(ItemsWeek.Monday >= today - 365 and ItemsWeek.Monday < today)
Items.Level = if Items.Level == 0 then -10 else
              if log(Items.Level) < -10 then - 10 else
              if log(Items.Level) > 10 then 10 else
              log(Items.Level)
maxEpochs = 1000
autodiff Items epochs:maxEpochs learningRate:0.01 with
  params Items.Affinity1 in [0..] auto(0.5, 0.166)
  params Items.Affinity2 in [0..] auto(0.5, 0.166)
  params Items.Level in [-10..10]
  params Items.LevelShift in [-0.5..0.5] auto(0, 0)
  params SeasonYW.Profile1  in [0..1] auto(0.5, 0.1)
  params SeasonYW.Profile2  in [0..1] auto(0.5, 0.1)
  SumAffinity =
    Items.Affinity1 +Items.Affinity2
  YearWeek.SeasonalityModel = SeasonYW.Profile1  * Items.Affinity1 +SeasonYW.Profile2  * Items.Affinity2
  Week.LinearTrend = ItemsWeek.Cache + (ItemsWeek.CumSumMinusOneExtMonth * ItemsWeek.Cache * Items.LevelShift / 10)
  Week.Baseline = exp(Items.Level) * YearWeek.SeasonalityModel * ItemsWeek.Cache *  Week.LinearTrend
  Week.Coeff =  ItemsWeek.ItemLifeWeight
  Week.DeltaSquare = (Week.Baseline - ItemsWeek.SmoothedDemandQty) ^ 2

  Sum = sum(Week.Coeff * Week.DeltaSquare) / 10000
  SumPowAffinity = (Items.Affinity1 ^2 +
                    Items.Affinity2 ^2 ) /\
                    (SumAffinity ^2)
  return ( \
          // Core Loss Function
          (1 + Sum) / (SumPowAffinity))
table ItemsYW = cross(Items, YearWeek)
ItemsYW.SeasonalityGroup = Items.SeasonalityGroup
ItemsWeek.YearWeek = Week.YearWeek
ItemsYW.Profile1 = SeasonYW.Profile1
ItemsYW.Profile2 = SeasonYW.Profile2
ItemsYW.SeasonalityModel = ItemsYW.Profile1  * Items.Affinity1 +
                            ItemsYW.Profile2  * Items.Affinity2
ItemsWeek.LinearTrend =  max(0, 1 + (ItemsWeek.CumSumMinusOneExtMonth * Items.LevelShift/ 10))
ItemsWeek.Baseline = exp(Items.Level) * ItemsYW.SeasonalityModel * ItemsWeek.LinearTrend

s40racer Jul 25, 2023 | flag

Continuing from the previous comment - For the same SKU, the values for SeasonalityModel, Profile1, level changed between two runs on different days. I am unsure what caused the change in these values - the input data remained the same.

vermorel Jul 25, 2023 | flag

Remark: I have edited your posts to add the Envision code formatting syntax, https://news.lokad.com/static/formatting

Envision is deterministic. You should not be able to re-run twice the same code over the same data and get different results.

Then, there is pseudo-randomness involved in functions like actionrwd. Thus, the seeding tend to be quite dependent on the exact fine-print of the code. If you change filters, for example, you are most likely going to end-up with different results.

Thus, even seemingly "minor" code change can lead to a re-seeding behavior.

As a rule of thumb, if the logic breaks due to re-seeding, then the logic is friable and must be adjusted so that its validity does not depend on being lucky during the seeding of the random generators.

vermorel Jul 26, 2023 | flag

The dispersion of actionrwd.foo is controlled by Dispersion:. At line 13, in your script I see:


Items.Dispersion = max(Items.AvgErrorRatio/2, 1)

This line implies that if there is 1 item (and only 1) that happens to have a super-large value, then, it will be applied for all items. This seems to be the root cause behind the high dispersion values that you are observing.

In particular,


ItemsWeek.RatioOfError = if ItemsWeek.Baseline != 0  then (ItemsWeek.Baseline - ItemsWeek.DemandQty) ^ 2 /. ItemsWeek.Baseline else 0

Above, ItemsWeek.RatioOfError can get very very large. If the baseline is small, like 0.01, and the demand qty is 1, then this value can be 100+.

Thus, my recommendations would be:

  • sanitize your ratio of error
  • don't use a max for the dispersion

Hope it helps.