Zero shadow prices for commodities not consumed/produced

[tsmbland]

Related to #992 and #1384

When commodities are not consumed/produced in a given year (but could be in the following year), the shadow prices often (always?) end up as zero in the timeslices where the commodity is not consumed/produced. This is even when we add candidate assets that could consume/produce the commodity.

I wonder if a solution here might be to add a small epsilon to the demand for these commodities, to force the candidate assets to activate. I've given this a very rough go and it does seem to fix these zeros, and give shadow prices in line with what I'd expect.

@ahawkes thoughts?

[ahawkes]

Good idea! Does this fix the dispatch issue in investment appraisal?

[tsmbland]

Good idea! Does this fix the dispatch issue in investment appraisal?

In some situations. We can still get zero shadow prices when commodities are over-supplied (#992). We can also still have situations where all existing assets are legitimately no longer feasible and choosing not to dispatch, e.g. if levies are introduced. That will require a different solution (#1347)

[tsmbland]

I've tried this (#1387), and it seems to work as intended, however I'm a little uneasy about it because it's extremely sensitive to the value of epsilon: too small and it has no impact (shadow prices still zero), too large and the problem becomes infeasible (candidates cannot meet demands). It's less than 2 orders of magnitude where it works as intended, and that's just on the relatively safe example models. (Small changes, e.g. 2x, have no impact, so this is a threshold effect)

Maybe it's fine, but something to bear in mind.

I'm wondering though: The point of this second dispatch run with candidates was so we can get shadow prices for commodities that aren't consumed/produced by actual assets in one year, but might be in the next year. Could a simpler approach just be to calculate the marginal cost of the cheapest candidate (which we have code to do anyway), and use that as a proxy for the shadow price?

[ahawkes]

Thanks @tsmbland - sounds useful, and noted on caveats (do we need to leave an open issue for that?)

Yes we need shadow prices for all commodities that could be produced in the next MSY for two reasons, (1) in case the user selects "shadow" or "scarcity" as pricing strategy for a commodity, and (2) the mini dispatch optimisation needs shadow prices to figure out dispatch - the idea being that we're assuming dispatch as per previous MSY.

The idea on using marginal cost instead is possibly a good idea. e.g. ditch shadow and scarcity as available price rules, and in the mini optimisation use marginal prices instead of shadow prices. The two (marginal and shadow) should be really similar (or rather, marginal and scarcity should be most similar), which suggests it's possibly a good approach. Shall we keep this possibility in back pocket for now, while we go through the whole investment umbrella?