At the 4th IPC (ICAPS 2004, Whistler, Canada) not all planners used the same notion of plan quality. Some planners adopted the number of plan actions, while others used the plan metric specified in the PDDL2.2 problem description (e.g., the plan makespan for temporal planning problems). LPG-td used the number of plan actions, for simple STRIPS problems, and the specified plan metric in all the other variants of the test problems. We believe that, for temporal or numerical domains, the use of the specified plan metric is much more natural and practically useful than the use of the number of plan actions. However, overall, the solutions produced by LPG-td.quality and especially LPG-td.bestquality are better than the solutions produced by the other IPC4 planners  in terms of both the number of actions and the specified plan metric.

At IPC4 some domains had different formalizations (ADL or STRIPS), that the competitors were free to chose for their planner. The official results of IPC4, that are available from the IPC4 website, compared planners that addressed different formalizations of the same domain. Here we do the same, with the only exception of the temporal variant of the Airport domain, because the problems in this domain have optimal solutions with different quality in the STRIPS and ADL formalizations. We note, however, that the results of our analysis are similar when, for every domain, the planners are compared only if they use the same domain formalization.

• % of problems for which LPG-td found a solution with quality better than the quality of corresponding solution found by the other planner minus % of problems for which LPG-td found a solution with quality worse than the quality of the corresponding solution found by the other planner. The quality of two solutions are considered different, if their values differ by more than 1%;
  

• % of problems for which LPG-td found a solution with quality much better than the quality of corresponding solution found by the other planner minus % of problems for which LPG-td found a solution with quality much worse than the quality of the corresponding solution found by the other planner. A solution X produced by a planner is much better (much worse)  than the solution Y produced by another planner when the quality of X is at least 50% better (worse) than the quality of Y. E.g., If X has quality 180 and Y has quality 100, then X is much better than Y (Y is much worse than X).