Measuring performance in dynamic planning is much more complicated than in static planning. A method for static planning can be evaluated by comparing the solution obtained with solutions obtained by other methods. In dynamic planning, however, the decisions made at one point in time determine the options and alternatives at a later point in time. It is possible to compare alternative decisions at a point in time, but then carrying the effects of those decisions forward in time creates a problem.
In dynamic planning with rolling horizons any evaluation of operating efficiencies must address the issue of time period evaluation. The state of the system at the end of the evaluation period may have a dramatic impact on future performance. A dynamic planning method may perform quite well over a day or a week, but can have poor long-term performance, e.g. if all vehicles are sent to very distant regions where there is little hope of picking up a new load.
Dynamic planning methods must be interactive enabling dispatchers to verify model recommendations. In many cases dispatchers will agree with model recommendations. However, different problem knowledge and solution methods of computer and human dispatchers may result in contradicting decisions. Under the time constraints in dynamic planning it is often very difficult or even impossible to find out why the recommendation is being made. Is the discrepancy a result of “higher reasoning” or a simple data error? Typically, dispatchers will follow their own intuition and not the model recommendation. A dynamic planning system is often considered successful if dispatchers agree with model recommendations, but dispatchers often do not follow model recommendations. Obviously, the question arises that if dispatchers do not comply with model recommendations are the solutions produced good at all?