Talk:Solver Statistics - XSokoban - Thinking Rabbit & Various Authors
From Sokoban Wiki
Hallo Brian, hallo Matthias, hi Minglw!
(sorry, only in german at time)
Ich würde mir wünschen, dass wir die Levels, die im Wiki unter dem Namen XSokoban laufen, auseinandernehmen. Dafür sollten wir Minglw bitten, entsprechende Listen anzufertigen, sofern entsprechende Levels im Test fehlen, was ja bei den echten Original der Fall sein wird, sollten wir Minglw diese Levels nachliefern. Ich möchte der Tendenz in der Szene entgegenwirken Originallevels und XSokoban in einem Atemzug zu benutzen (zu nennen).
Ich würde vorschlagen, dass die Levels aufgeteilt werden, in die 50 Originals von Thinking Rabbit (aus Spectrum Holobyte's -> Soko-Ban), nachträglich angefügt die 6 wirklichen Abweichungen (demolished levels by XSokoban, also 12, 20, 25, 37, 42, 50). Ich will betonen, dass die 6 Levels von XSokoban für mich keine Variationen im besten Wortsinne sind, sondern eigentlich nur verunstaltete...
Desweiteren würde ich gern die szeneintern "Extra levels" genannten XSokoban Levels, also Level 51 bis 90 gerne von den Originals separieren.
Somit würde auch an dieser Stelle klar hervortreten, und das sehe ich als Mission, dass XSokoban Levels KEINE Originals sind.
Beste Grüße
Sokobano
The XSokoban/90 level set is THE established test set for Sokoban solver benchmarking, and there is no way to change that. Many academic papers dealing with Sokoban solvers refer to these levels, so despite the awkward background (copyrighted levels, wrongly called "XSokoban levels" instead of giving proper credits to "Thinking Rabbit" for 50 of the levels, modified levels, and not to the better), this set is eminently qualified for a statistical survey like the one here on the Wiki.
-bd
It's maybe a good idea to have a second column on that page for the Original levels. This way everybody can see that there are some levels where the two collections differ. There the "40 extra" levels can also be separated from the 50 Original levels.
I also think that we shouldn't separate the 90 XSokoban levels because there are many comparisons using these levels (Rolling Stone, Takaken, ...) Hence, it's easier to compare the result on the Wiki with those pages when the "90 XSokoban" levels are listed.
Matthias
Well, the purpose of these statistics is to show what the Sokoban Solvers can solve. We are not here trying to infringe on copyrights or whether the set's content is proper or not. With that, I don't see there's further need to discuss about the content of the sets.
So far, I have purposely chosen level sets that are publicly available. The only exception is BoxWorld, but the program is freely available for everyone to download, and there is a program freely available out there for people to extract the levels from BoxWorld. Bottom line is, people have access to the levels freely. That's not to say that I won't release statistics on sets that are commercially available. I am not here to release the content/design of the level sets anyway, just the statistics on whether the solvers can solve them or not.
The main reason that I've chosen level sets that are freely available is so that people can verify the results if they want. Better yet, people with a faster computer and more RAM can help to test and update the results here. If people do that, I would appreciate it if they can send me the output file(s) from the solvers proving that the levels are indeed solved. (it's just a way to verify the stats are good). It's not good enough to send me just the solution for the level(s), that didn't prove anything other than saying that you have the solution to a given level.
Let's get back on topic with XSokoban statistics.... The current presentation of the XSokoban Statistics are nice. However, it's difficult to make any updates to them (unless a script/program is created to convert from the existing plain text format into some nice looking format.
Recently, I've some updated statistics for the YASS solver (using version 2.66). It would be a time consuming process to manually merge the new statistics into the existing format, and in the end, one would probably still wonder if the updates are indeed correct.
So, I am going to just post the plain text format. If you like to see the previous colorful table format showing which of the 90 levels were solved or failed, feel free to make the change.
-- Ming
