Matthias Meger: /* General benchmarking information */ new JSoko version

2026-03-11T20:48:12Z

General benchmarking information: new JSoko version

← Older revision		Revision as of 22:48, 11 March 2026
Line 44:		Line 44:

	\|-		\|-
	\|<span class="plainlinks">[https://jsokoapplet.sourceforge.io/ JSoko]</span> \|\| 2.26 \|\| Matthias Meger \|\| 64bit \|\| 1 thread \|\| <span class="plainlinks">[https://sourceforge.net/projects/jsokoapplet/files/JSoko/Version%202.24/ JSoko 2.25]</span> \|\| [[Solver \| Solver information]]		\|<span class="plainlinks">[https://jsokoapplet.sourceforge.io/ JSoko]</span> \|\| 2.26 \|\| Matthias Meger \|\| 64bit \|\| 1 thread \|\| <span class="plainlinks">[https://sourceforge.net/projects/jsokoapplet/files/JSoko/Version%202.24/ JSoko 2.26]</span> \|\| [[Solver \| Solver information]]
	\|-		\|-
	\|<span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html Rolling Stone]</span> \|\| 1.0 \|\| Andreas Junghanns \|\| 32bit \|\| 1 thread \|\| <span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/Src/ RS] \|\| <span class="plainlinks">[https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=rolling+stone+sokoban&btnG= Rolling Stone papers]</span>		\|<span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html Rolling Stone]</span> \|\| 1.0 \|\| Andreas Junghanns \|\| 32bit \|\| 1 thread \|\| <span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/Src/ RS] \|\| <span class="plainlinks">[https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=rolling+stone+sokoban&btnG= Rolling Stone papers]</span>

Matthias Meger: /* General benchmarking information */ New JSoko version

2026-03-11T20:47:44Z

General benchmarking information: New JSoko version

← Older revision		Revision as of 22:47, 11 March 2026
Line 44:		Line 44:

	\|-		\|-
	\|<span class="plainlinks">[https://jsokoapplet.sourceforge.io/ JSoko]</span> \|\| 2.25 \|\| Matthias Meger \|\| 64bit \|\| 1 thread \|\| <span class="plainlinks">[https://sourceforge.net/projects/jsokoapplet/files/JSoko/Version%202.24/ JSoko 2.25]</span> \|\| [[Solver \| Solver information]]		\|<span class="plainlinks">[https://jsokoapplet.sourceforge.io/ JSoko]</span> \|\| 2.26 \|\| Matthias Meger \|\| 64bit \|\| 1 thread \|\| <span class="plainlinks">[https://sourceforge.net/projects/jsokoapplet/files/JSoko/Version%202.24/ JSoko 2.25]</span> \|\| [[Solver \| Solver information]]
	\|-		\|-
	\|<span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html Rolling Stone]</span> \|\| 1.0 \|\| Andreas Junghanns \|\| 32bit \|\| 1 thread \|\| <span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/Src/ RS] \|\| <span class="plainlinks">[https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=rolling+stone+sokoban&btnG= Rolling Stone papers]</span>		\|<span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html Rolling Stone]</span> \|\| 1.0 \|\| Andreas Junghanns \|\| 32bit \|\| 1 thread \|\| <span class="plainlinks">[https://webdocs.cs.ualberta.ca/~games/Sokoban/Src/ RS] \|\| <span class="plainlinks">[https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=rolling+stone+sokoban&btnG= Rolling Stone papers]</span>

Matthias Meger: /* Disclaimer */ fixed URL

2026-02-08T22:06:22Z

Disclaimer: fixed URL

← Older revision		Revision as of 00:06, 9 February 2026
Line 328:		Line 328:
	Copyright (C) 2002-2006 by Lee J Haywood		Copyright (C) 2002-2006 by Lee J Haywood
	E-mail: fruise@gmail.com		E-mail: fruise@gmail.com
	Web Site: ~~http~~://~~leehaywood~~.~~org~~/games/sokoban/		Web Site: https://ljhaywood.uk/games/sokoban/
	<br>		<br>
	Sven level set		Sven level set

Matthias Meger: /* Open Test Suite */ added BoxSearch to Open Test Suite and removed a duplicate collection from all results

2025-08-14T15:18:29Z

Open Test Suite: added BoxSearch to Open Test Suite and removed a duplicate collection from all results

New page

A Sokoban solver program is a program that ''tries to'' solve Sokoban levels. There are several solver programs available. (See [[Links#Sokoban Solvers |Sokoban solvers]].)

Solver programs may have different objectives:
# Find any solution
# Find push optimal solutions
# Find push optimal solutions with best moves
# Find move optimal solutions
# Find move optimal solutions with best pushes

Given the computational complexity of these tasks, a solver program is usually designed for just finding any solution. If a solution is to be optimized, an optimizer program can be used. (See [[Links#Sokoban Solvers |Optimizers]].)

The statistics presented here provide a picture of what the best freely available Sokoban solvers can do. 
For copyrights, credits and redistribution of these statistics, please read the disclaimer.

== General benchmarking information ==

For practical reasons, almost all tests have been performed with a 10 minutes time limit. The tests are, however, not a 10 minutes contest but a showcase for what the solvers can do, so other time limits are entirely feasible.

The hardware used for the tests is a standard PC with a modern operating system (64bit).

Tests are performed with default solver settings.

{| class="soko-table feature-list"
!Solver || Version || Author || Architecture || CPU usage || Source code available || More info

|-
|[https://festival-solver.site/ Festival] || 3.1 || Yaron Shoham || 64bit || 8 threads || [https://festival-solver.site/ Festival 3.1] || [https://festival-solver.site/ Website with papers about the solver]

|-
|[http://sokolution.infinityfreeapp.com/ Sokolution] || 7.32 || Florent Diedler || 64bit || all threads || no || [[Sokoban solver "scribbles" by Florent Diedler about the Sokolution solver | Sokoban solver "scribbles" by Florent Diedler]]

|-
|[https://computerpuzzle.net/english/sokoban/ver740/index.html Takaken] || 7.4 || Ken'ichiro Takahashi (takaken) || 64bit || 2 threads || no || [https://computerpuzzle.net/puzzle/sokoban/ver731/index.html Description of the used algorithms]

|-
|[https://festival-solver.site/curry/ Curry] || 0.3 || Yaron Shoham || 64bit || 1 thread || [https://festival-solver.site/curry/ Curry 0.3] || [https://festival-solver.site/curry/ Website with paper about the solver]

|-
|[https://notabdc.github.io/puzzle007/EngPage/e_solver.htm#BoxSearch BoxSearch] || 7.0 beta 1 || 葛永 (Ge Yong) || 64bit || 1 thread || no || [https://notabdc.github.io/puzzle007/EngPage/e_solver.htm#BoxSearch Website of BoxSearch]

|-
|[https://sourceforge.net/projects/sokobanyasc/files/sokobanyasc/ YASS] || 2.153 || Brian Damgaard || 32bit + 64bit || 1 thread || [https://sourceforge.net/projects/sokobanyasc/files/sokobanyasc/ YASS 2.153] || [[Sokoban solver "scribbles" by Brian Damgaard about the YASS solver | Sokoban solver "scribbles" by Brian Damgaard]]

|-
|[https://jsokoapplet.sourceforge.io/ JSoko] || 2.25 || Matthias Meger || 64bit || 1 thread || [https://sourceforge.net/projects/jsokoapplet/files/JSoko/Version%202.24/ JSoko 2.25] || [[Solver | Solver information]]
|-
|[https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html Rolling Stone] || 1.0 || Andreas Junghanns || 32bit || 1 thread || [https://webdocs.cs.ualberta.ca/~games/Sokoban/Src/ RS] || [https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=rolling+stone+sokoban&btnG= Rolling Stone papers]

|}

Contact: SokobanoWiki@gmail.com

== XSokoban Test Suite ==

The XSokoban level set is the de facto standard test suite in the academic literature on Sokoban solver programming.

{| class="soko-table"
|-
! colspan="11" | Sokoban Solver Statistics - XSokoban Test Suite - Solved Levels


|- bgcolor="#f1efe5" align="center"
| width="115" align="left" | Collection
| width="160" | Author
| width=" 70" | Levels
| width= "90" | [https://www.researchgate.net/publication/2305703_Pushing_the_Limits_New_Developments_in_Single-Agent_Search Rolling Stone]
| width= "80" | [https://computerpuzzle.net/english/sokoban/ver740/index.html Takaken]
| width=" 80" | [https://sourceforge.net/projects/sokobanyasc/ YASS]
| width=" 80" | [[JSoko Solver | JSoko]]
| width=" 80" | [http://sokolution.infinityfreeapp.com/ Sokolution ]
| width=" 80" | [https://festival-solver.site/curry/ Curry ]
| width=" 80" | [https://festival-solver.site/ Festival ]
| width=" 80" | [https://notabdc.github.io/puzzle007/EngPage/e_solver.htm#BoxSearch BoxSearch]




|- align="right"
| align="left" bgcolor="#f1efe5" | [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/ XSokoban]
| align="left" | Thinking Rabbit, ...
| 90
| [https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html 59*]
| [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/Takaken%20-%20XSokoban.html '''90''']
| [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/YASS%20-%20XSokoban.html 89]
| [[SolverStatistics-jsoko-ThinkingRabbit-XSokoban | 67 ]]
| [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/Sokolution%20-%20XSokoban.html '''90''']
| [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/Curry%20-%20XSokoban.html '''90''']
| [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/Festival%20-%20XSokoban.html '''90''']
| [https://sokoban-solver-statistics.sourceforge.io/statistics/XSokoban/BoxSearch%20-%20XSokoban.html '''89''']

|}

*) 59 solved puzzles according to paper [https://sokoban.dk/wp-content/uploads/2016/02/Single-Agent.pdf Sokoban Enhancing general single-agent search methods using domain knowledge]

{| class="soko-table"
|-


! colspan="7" | Sokoban Solver Statistics - XSokoban Test Suite - Push-optimal Solutions
|- bgcolor="#f1efe5" align="center"
| width="115" align="left" | Collection
| width="160" | Author
| width=" 70" | Levels
| width=" 80" | [https://www.researchgate.net/publication/2305703_Pushing_the_Limits_New_Developments_in_Single-Agent_Search Rolling Stone]
| width=" 80" | [http://www.ijcai.org/Abstract/16/100 Pereira et al.]
| width=" 80" | [http://sourceforge.net/projects/jsokoapplet/ JSoko]
| width=" 80" | [http://sokolution.infinityfreeapp.com/ Sokolution ]


|- align="right"
| align="left" bgcolor="#f1efe5" | XSokoban
| align="left" | Thinking Rabbit, ...
| 90
| [https://webdocs.cs.ualberta.ca/~games/Sokoban/program.html 6 ]
| [[SolverStatistics-Pereira-ThinkingRabbit-XSokoban-Push-Optimal | 28 ]]
| [https://sokoban-solver-statistics.sourceforge.io/statistics/Push-OptimalTestSuite/JSoko%20-%20XSokoban.html 25]
| [https://sokoban-solver-statistics.sourceforge.io/statistics/Push-OptimalTestSuite/Sokolution%20-%20XSokoban.html '''73''']

|}

The statistic shows that Sokolution is by far the best solver when searching for push optimal solutions.

== Large Test Suite ==

The large test suite comprises level packs selected for being a challenge for the solvers, but not too difficult. The statistics shows the results for the leading Sokoban solver programs.

This test suite is not subject to change, as it is intended to be a "Large Standard Test Suite" for Sokoban solver programs.

{| class="soko-table"
|-
! colspan="7" | Sokoban Solver Statistics - Large Test Suite - Solved levels


|- bgcolor="#f1efe5" align="center"
| width="115" align="left" |
| width=" 70" | Levels
| width= "80" | [https://festival-solver.site/ Festival ]
| width=" 80" | [http://sokolution.infinityfreeapp.com/ Sokolution]
| width=" 80" | [https://computerpuzzle.net/english/sokoban/ver740/index.html Takaken]
| width=" 80" | [https://sourceforge.net/projects/sokobanyasc/ YASS]
| width=" 80" | [https://notabdc.github.io/puzzle007/EngPage/e_solver.htm#BoxSearch BoxSearch]

|- bgcolor="#f1efe5" align="right"
| align="left" | Total levels
| 3272 
| '''3208''' 
| 3207 
| 2953 
| 2899 
| 2749 

|}

'''[https://sokoban-solver-statistics.sourceforge.io/statistics/LargeTestSuite/ Click here to see the detailed results for the Large Test Suite].'''

Download the entire test suite [https://sourceforge.net/projects/sokoban-solver-statistics/files/Large%20Test%20Suite.zip/download here] or the individual level sets from [http://sokobano.de/de/levels.php www.sokobano.de].

== Open Test Suite ==

The statistics shows the results for the leading Sokoban solvers for various level collections. Please note that the grand totals are simple summations. The number of solved unique levels may differ slightly due to duplicate levels in the level packs.

This test suite may be changed by adding new level collections from time to time.

{| class="soko-table"
|-
! colspan="7" | Sokoban Solver Statistics - Open Test Suite - Solved levels


|- bgcolor="#f1efe5" align="center"
| width="115" align="left" |
| width=" 70" | Levels
| width= "80" | [https://festival-solver.site/ Festival]
| width=" 80" | [http://sokolution.infinityfreeapp.com/ Sokolution]
| width=" 80" | [https://computerpuzzle.net/english/sokoban/ver740/index.html Takaken]
| width=" 80" | [https://sourceforge.net/projects/sokobanyasc/ YASS]
| width=" 80" | [https://notabdc.github.io/puzzle007/EngPage/e_solver.htm#BoxSearch BoxSearch]

|- bgcolor="#f1efe5" align="right"
| align="left" | Total levels
| 10043 
| '''9596''' 
| 9516 
| 8721 
| 8339 
| 8111 

|}

'''[https://sokoban-solver-statistics.sourceforge.io/statistics/OpenTestSuite/ Click here to see the detailed results for the Open Test Suite].'''

[https://sokoban-solver-statistics.sourceforge.io/statistics/Sokolution/ Comparison for the Open Test Suite for all Sokolution versions].

== Push-optimal Test Suite ==

The "Push-optimal Test Suite" comprises level packs selected for being a challenge for the solvers, but not too difficult. The statistics shows the results for the leading Sokoban solver programs.

For this test suite the solvers must solve the puzzles push optimal.

{| class="soko-table"
|-
! colspan="6" | Sokoban Solver Statistics - Large Test Suite - Solved levels


|- bgcolor="#f1efe5" align="center"
| width="115" align="left" |
| width=" 70" | Levels
| width=" 80" | [http://sokolution.infinityfreeapp.com/ Sokolution]

|- bgcolor="#f1efe5" align="right"
| align="left" | Total levels
| 1588 
| [https://sokoban-solver-statistics.sourceforge.io/statistics/Push-OptimalTestSuite/ '''1529'''] 
|}

'''[https://sokoban-solver-statistics.sourceforge.io/statistics/Push-OptimalTestSuite/ Click here to see the detailed results for the Push-Optimal Test Suite].'''

== GroupEffort Test Suite ==

The only reason for benchmarking the GroupEffort test suite is to demonstrate the limited veracity of the statement made in section "6 Conclusion" of the thesis, where the author writes that the GroupEffort solver "outperforms existing solvers".

The benchmarks show that this was only true for the cherry-picked small test set at the time the thesis was published, never for the presumably more representative large test set.

{| class="soko-table"
|-
! colspan="8" | Sokoban Solver Statistics - GroupEffort Test Suite - Solved Levels




|- bgcolor="#f1efe5" align="center" 
| width = "115" align="left" | Collection
| width = "80" | Levels
| width = "80" | [https://baldur.iti.kit.edu/theses/SokobanPortfolio.pdf GroupEffort]
| width = "80" | [https://festival-solver.site/ Festival]
| width = "80" | [http://sokolution.infinityfreeapp.com/ Sokolution]
| width = "80" | [https://computerpuzzle.net/english/sokoban/ver740/index.html Takaken]
| width = "80" | [https://sourceforge.net/projects/jsokoapplet/ JSoko]
| width = "80" | [https://sourceforge.net/projects/sokobanyasc/ YASS]




|- align="right"
| align="left" bgcolor="#f1efe5" | [[SolverStatistics-GroupEffort-Small-Set | Small Set]]
| 200
| 196 (a)(c)
| [[SolverStatistics-Festival-GroupEffort-Small-Set | '''200''' (b) ]]
| [[SolverStatistics-Sokolution-GroupEffort-Small-Set | '''200''' (a) ]]
| [[SolverStatistics-Takaken-GroupEffort-Small-Set | 192 (b) ]]
| [[SolverStatistics-JSoko-GroupEffort-Small-Set | 180 (b) ]]
| [[SolverStatistics-YASS-GroupEffort-Small-Set | 191 (b) ]]




|- align="right"
| align="left" bgcolor="#f1efe5"| [[SolverStatistics-GroupEffort-Large-Set | Large Set]]
| 2851
| 1566 (a)(c)
| [[SolverStatistics-Festival-GroupEffort-Large-Set | '''2766''' (b) ]]
| [[SolverStatistics-Sokolution-GroupEffort-Large-Set | 2471 (a) ]]
| [[SolverStatistics-Takaken-GroupEffort-Large-Set | 2462 (b) ]]
| [[SolverStatistics-JSoko-GroupEffort-Large-Set | 2165 (b) ]]
| [[SolverStatistics-YASS-GroupEffort-Large-Set | 2260 (b) ]]




|- align="right" bgcolor="#f1efe5" align="right"
| align="left"| Total
| 3051 
| 1762 
| '''2966''' 
| 2671 
| 2654 
| 2345 
| 2471 
|}

(a): Time limit: 5 minutes     (b): Time limit: 10 minutes     (c): Best GroupEffort solver configuration

== Disclaimer ==

Sokoban
=======
 
Sokoban (R) Registered Trademark of Falcon Co., Ltd., Japan
Sokoban Copyright (C) 1982-2004 by Hiroyuki Imabayashi, Japan
Sokoban Copyright (C) 1989, 1990, 2001-2004 by Falcon Co., Ltd., Japan
 
License
=======
 
Redistribution and use of the Sokoban Solver statistics are permitted
provided that the following conditions are met:
 
* Redistributions may not be sold, nor may they be used in a commercial product or activity.
* Redistributions must reproduce the entire content of this disclaimer in the documentation
and/or other materials provided with the distribution.
 
Credits (in no particular order)
================================
 
CosmoNotes, CosmoPoly, LOMA, MicroCosmos, MiniCosmos,
NaboCosmos and PicoCosmos level sets
Copyright (C) by Aymeric du Peloux.
E-mail: aymeric.du-peloux@gadz.org
Web site: unknown
 
dh1, dh2, bagatelle, cantrip, and maelstrom level sets
Copyright (C) by David Holland
E-mail: unknown
Web site: unknown
 
Microban, Mas Microban, Sasquatch, Mas Sasquatch, Sasquatch III,
Sasquatch IV, Sasquatch V, Sasquatch VI and Sasquatch VII level sets.
Copyright (C) by David W. Skinner
E-mail: sasquatch@bentonrea.com
Web site: unknown
 
Grigr2001, Grigr2002 and GrigrSpecial level sets
Copyright (C) by Evgeny Grigoriev (aka GRIGoRusha)
E-mail: grigr@yandex.ru
Web site: http://grigr.narod.ru/
 
SokEvo and SokHard level sets
Copyright (C) 2002-2006 by Lee J Haywood
E-mail: fruise@gmail.com
Web Site: http://leehaywood.org/games/sokoban/
 
Sven level set
Copyright (C) 2007 by Sven Egevad
E-mail: sven.egevad@telia.com
Web Site: unknown
 
Sokoban Perfect and Sokoban Revenge level sets
Copyright (C) 1989, 1991 by Thinking Rabbit
Web Site: https://sokoban.jp/
 
Y.M. Auto and Y.M. Handmade level sets
Copyright (C) Yoshio Murase
E-mail: yoshio@asahi.email.ne.jp
Web Site: http://www.ne.jp/asahi/ai/yoshio/sokoban/index.html
 
JSoko
Copyright (C) by Matthias Meger
E-mail: JSoko@mail.de
Web site: https://jsokoapplet.sourceforge.io/
 
Sokoban Automatic Solver (aka Takaken solver)
Copyright (C) 2002 - 200X Ken'ichiro Takahashi (aka takaken)
E-mail: takaken@ic-net.or.jp
Web site: https://computerpuzzle.net/english/sokoban/ver740/index.html
 
YASS - Yet Another Sokoban Solver and Optimizer - For Small Levels
Copyright (C) 2007 by Brian Damgaard
Web site: https://sourceforge.net/projects/sokobanyasc/
 
Sokoban YASC
Copyright (C) 2001-2020 by Brian Damgaard
Web Site: https://sourceforge.net/projects/sokobanyasc/
 
YSokoban
Copyright (C) 2007 by George Petrov
E-mail: YGP@WriteMe.com
Web Site: http://ygp.orgfree.com/sokoban.html
 
XSokoban
Copyright (C) by Andrew Myers
E-mail: andru@cs.cornell.edu
Web Site: http://www.cs.cornell.edu/andru/xsokoban.html
 
Sokofan
Copyright (C) 2005-2007 by Eric Leung
E-mail: eleung2@hotpop.com
Web Site: unknown
 
Sokolution
Copyright (C) 2016-2019 by Florent DIEDLER
E-mail: fdiedler2000@gmail.com
Web site: http://sokolution.infinityfreeapp.com/
 
Festival
Copyright (C) 2020 by Yaron Shoham
Web site: https://festival-solver.site/
 
Trademarks
==========
 
Company names, brand names and product names are trademarks
or registered trademarks of their respective holders.
 
Contact
=======
 
E-mail: sokobanowiki@gmail.com

Solver Statistics - Revision history

Matthias Meger: /* General benchmarking information */ new JSoko version

Matthias Meger: /* General benchmarking information */ New JSoko version

Matthias Meger: /* Disclaimer */ fixed URL

Matthias Meger: /* Open Test Suite */ added BoxSearch to Open Test Suite and removed a duplicate collection from all results