Solver Statistics - SokHard - Lee J Haywood
From Sokoban Wiki
This is a summary of running the SokHard level set (163 levels) using these solvers:
- BoxSearch v5.1 beta2
- Takaken v7.0
- YASS v2.63
- BoxSearch v5.1 beta2
For copyrights, credits and redistribution of these statistics, please read the disclaimer.
Though some people might be interested in how long it takes to solve a given level and/or how good the solution is (optimal or not), this summary disregards all timing associated with solving the levels and solution statistics.
These tests are mainly for gauging solvability, not timing nor how good the solution is.
- Not all Sokoban solvers are created equal
-
- BoxSearch v5.1 beta2 allows setting for max time limit and max memory.
- Takaken v7.0 allows setting for max time but lacks max memory option.
- YASS v2.63 allows setting for max time and max memory option.
- Test machine info
-
- Machine: ThinkPad T42P
- CPU: x86 Family 6 Model 13 Stepping 6 Genuine Intel
- Physical RAM: 2 GB
- OS Name & Version: Microsoft Windows XP Professional v5.12600 Service Pack 2 Build 2600.
- Initial tests
-
- BoxSearch is set with max memory limit of 1500 MB and 600 seconds. (using quick mode)
- Takaken is set with 600 seconds time limit
- YASS is set with 1500 MB memory and 600 second limit.
- Further tests
-
- For BoxSearch, 1 level failed with the initial tests settings but were solved with 9999 seconds limit.
- For Takaken, the levels were not failed because of time out. Hence, no further tests are needed with v7.0. (unless a new version is available)
- For YASS, 12 levels failed with the initial tests settings but were solved with 9999 seconds limit. As pointed out by Brian Damgaard, YASS's author, for small levels (like the levels in this SokHard level set), YASS performs much better with different settings, such as disabling the packingorder mechanism and/or running with less memory. A test was done with YASS using options "-maxtime 600 -memory 256 -packingorder 9999" and then it solves all levels with blazing speed. All of these solutions are also push-optimal.
Notes :
- where applicable, all solvers were given 1500 MB of RAM and 600 seconds to solve each level.
- each solver is tested separately to ensure that they don't compete for system resources.
The following table shows the levels where each of the solvers failed:
======================================================================
BoxSearch Takaken YASS
============== ============== ==============
2
3
6
8
9
11
12
13
14
16
17
18
19
20
21
23
24
25
26
27
29
30
32
33
34
36
39
40
41
42
43
44
45
46
48
49
52
56
60
61
62
63
64
66
67
69
70
71
72
74
76
77
78
79
81
82
83
84
85
87
89
90
91
92
93
94
96
97
98
99
100
101
103
104
105
106
108
109
111
112
113
114
116
117
118
119
120
121
122
123
124
125
126
127
128
129
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
============== ============== ==============
Total: 0 129 0
Total (unique level count): 129
All solvers failed on these levels:
===================================================================================
===================================================================================
Total count: 0
"BoxSearch" failed on these levels where at least one solver solved:
===================================================================================
===================================================================================
Total count: 0
"Takaken" failed on these levels where at least one solver solved:
===================================================================================
2 3 6 8 9 11 12 13 14 16
17 18 19 20 21 23 24 25 26 27
29 30 32 33 34 36 39 40 41 42
43 44 45 46 48 49 52 56 60 61
62 63 64 66 67 69 70 71 72 74
76 77 78 79 81 82 83 84 85 87
89 90 91 92 93 94 96 97 98 99
100 101 103 104 105 106 108 109 111 112
113 114 116 117 118 119 120 121 122 123
124 125 126 127 128 129 131 132 133 134
135 136 137 138 139 140 141 142 143 144
145 146 147 148 149 150 151 152 153 154
155 156 157 158 159 160 161 162 163
===================================================================================
Total count: 129
"YASS" failed on these levels where at least one solver solved:
===================================================================================
===================================================================================
Total count: 0
"BoxSearch" solved these levels where all other solvers failed:
===================================================================================
===================================================================================
Total count: 0
"Takaken" solved these levels where all other solvers failed:
===================================================================================
===================================================================================
Total count: 0
"YASS" solved these levels where all other solvers failed:
===================================================================================
===================================================================================
Total count: 0
"BoxSearch" failed on these levels where all other solvers solved:
===================================================================================
===================================================================================
Total count: 0
"Takaken" failed on these levels where all other solvers solved:
===================================================================================
2 3 6 8 9 11 12 13 14 16
17 18 19 20 21 23 24 25 26 27
29 30 32 33 34 36 39 40 41 42
43 44 45 46 48 49 52 56 60 61
62 63 64 66 67 69 70 71 72 74
76 77 78 79 81 82 83 84 85 87
89 90 91 92 93 94 96 97 98 99
100 101 103 104 105 106 108 109 111 112
113 114 116 117 118 119 120 121 122 123
124 125 126 127 128 129 131 132 133 134
135 136 137 138 139 140 141 142 143 144
145 146 147 148 149 150 151 152 153 154
155 156 157 158 159 160 161 162 163
===================================================================================
Total count: 129
"YASS" failed on these levels where all other solvers solved:
===================================================================================
===================================================================================
Total count: 0
BoxSearch solved 163 of 163 levels.
100%
Takaken solved 34 of 163 levels.
20%
YASS solved 163 of 163 levels.
100%
All solvers combined
solved 163 of 163 levels.
100%
