Dear IPC-5 competitor, This mail intends to give a general picture of how (the deterministic part of) IPC-5 will be carried out and some general rules. Please take the time to read the mail carefully through to the end. You will install your planners on a machine located at the University of Brescia (olimpia.ing.unibs.it). We will incrementally make the IPC-5 domains and test problems available from this machine. You will run your planners yourselves and place the results into files of a given format. For every single problem instance, you can use at most 30 minutes CPU time and 1 GByte of memory. Every single process started from your accounts will be automatically aborted when it exceeds these bounds. If possible, please make your planner one single process. If your planner has to start more (sub)processes, when reporting the overall CPU time, you must sum the CPU time of every process (even if they are "parallel" processes). The total CPU time of your planner that will be reported in the output plan file must include the pre-processing and post-processing performed by the planner. There is no deadline when your results must be ready, except **APRIL 15**, when (a) the competition tests end and (b) the executables are delivered. During this period you can change the code of your planners as many times you like. But, of course, you are **not** allowed to do silly things such as hard-coding the names of the domains and implementing specialized routines/parameter values. Obviously, we won't look into your source code to check that, so we will have to trust you for it. The executable that is delivered should produce ALL results (across all domains) in your data files. After April 15 we will collect all results, evaluate them, and run some tests to verify that the executables produce the same results. The output plan files should be written in a standard format similar to the one used in the 2004 competition. Details about this will be sent in a separate message. As in the 2004 competition, every planner using some form of randomization should use the same random seed in every test. (This implies that a randomized planner is run at most once for every test problem -- there are many competing planners and test problems, and there are not enough computational resources to compute statistical data such as average or median CPU time.) The random seed to be used by these planners is: 6102006. This should also guarantee that every run of a randomized planner can possibly be repeated during the validation-evaluation period. If your planner is incremental, e.g., it is capable of producing multiple solutions with increasing quality, the CPU time reported in the output plan file generated by the planner should be the time when the process *ENDS* (a similar rule was used in the 2004 competition). So, for instance, if the planner generates a plan consuming X CPU seconds and then tries to generate a better quality plan using Y additional CPU seconds, but such a plan is not found, then the CPU reported in the file should be X+Y (and not X). Please take into account that each process will be automatically killed when the CPU usage exceeds 30 minutes, and so before being killed it should generate the output file (reporting 30min CPU-time). The competition machine runs Linux Suse Professional 10.0, it has 2 Intel(R) Xeon(TM) 3.00GHz processors and 6GByte main memory. If necessary, we will use an additional machine, which is exactly the same but with slightly less memory. The next planned deadlines -------------------------- (1) From Jan 26: creation of one account for each participating team on the IPC-5 machine; installation of planners on the IPC-5 machine. You will install your software, and get it running. We hope we will be able to provide the compilation tools needed by each team, otherwise you'll have to upload a statically compiled executable. If you have already sent us the your public key file(s), you should have already received the username of your account. For those who haven't sent the public key file(s) yet, please do it soon (if you have problems about the procedure to generate the key, please let us know). (2) Feb 3: we start making the IPC-5 domains/problems available. We will do so incrementally during the competition tests period. Each time a new domain (or set of problem) become available, we will inform the contact person of each team. Overall there will be 5-7 domains. You are required to run your planner, and collect the results (output plans with the required information in the specified standard format) into data files mirroring the structure of the directory where you will find the test domains/problems. Detailed instructions about this will follow when we make the 1st domain available. When running your planner, please consider the situation on the IPC-5 machine: if too many people run memory-consuming planners in parallel, then of course the machine will die. Moreover, please take into account that in the second part of the testing period, it is likely that many planners will be running, and so they will have to share the CPUs. This means you should not delay your tests too long, otherwise there is the risk you won't be able to complete them by the deadline (April 15). (3) March 1 [OPTIONAL]: deadline for submitting a paper to the ICAPS'06 workshop on "Preferences and Soft Constraints in Planning": http://www.cis.strath.ac.uk/%7Ederek/PSCinP.html (4) March 10: Deadline for submitting the abstracts about the competing planners. We will distribute these abstracts through the website of the competition, and hopefully we will make a booklet to be distributed at ICAPS'06. The abstracts should be 1-3 pages using the AAAI style. (5) April 15: deadline for delivering all data files and the planner executables producing these files. Moreover, this is the deadline for the final version of the abstracts. I think for the moment this is all. Cheers, Alfonso --------------------------------------------------------------- Prof. Alfonso Gerevini Dipartimento di Elettronica per l'Automazione Universita` di Brescia -- Via Branze 38, I-25123 Brescia, ITALY Phone: +39-030-3715-451(direct)/469 Fax: +39-030-380014 Email: gerevini@ing.unibs.it Http: //www.ing.unibs.it/~gerevini ---------------------------------------------------------------