Comments by SamTrenholme
- Gave each side 40 seconds to do 40 moves (too slow for what we're doing)
- Disabled pondering with -xponder
#!/bin/sh SECS=10 while : ; do winboard.exe -noGUI -xanimate -cp -fcp joker80.exe -scp joker80.exe \ -boardSize middling -xponder \ -variant capablanca -lpf schoolbook.fen -tc 0:${SECS} -mps 40 \ -autosave -sgf game-${SECS}sec-40moves-$(date +%s).pgn -mg 10 sleep 1 doneWhile there still is the occasional game won on time, most of the time the game ends in checkmate or draws by repetition.
Note that I also hacked backend.c to remove the popup that tells you the match results at the end of a match.
Sometimes, one of the sides loses on time. Since it is a little more complicated to adjudicate such a game to see who had the better position when the flag fell, I have simply discarded any game which does not end with a draw or a checkmate (Joker80, unlike TJChess10x8, never resigns).
Once I did this, here is the win/loss/draw statistics I have so far for Joker80 playing Schoolbook Chess against itself with a 40 moves in 10 seconds time control:
e4 Win: 51.6556% Loss: 38.4106% draw 9.93377% Total 151
f4 Win: 51.4019% Loss: 41.1215% draw 7.47664% Total 107
f3 Win: 52.381% Loss: 33.3333% draw 14.2857% Total 63
c4 Win: 44.2623% Loss: 40.9836% draw 14.7541% Total 61
c3 Win: 38.8889% Loss: 50% draw 11.1111% Total 18
h3 Win: 100% Loss: 0% draw 0% Total 2
Ni3 Win: 100% Loss: 0% draw 0% Total 1
Total Win: 50.3722% Loss: 38.9578% draw 10.67% Total 403
#!/bin/sh awk '{ l=$0; sub(/\[.*\]/,'',l); # Get opening move if(match(l,/^1\./)) { sub(/{.*/,'',l); sub(/^1\.[ \t]*/,'',l); sub(/[ ]+.*$/,'',l); opening=l } # Tally up wins and losses if(match(l,/Checkmate/) || match(l,/mates/) || match(l,/resign/)) { sub(/{.*}[ ]*/,'',l); if(match(l,/0\-1/)) { loss[opening]++; } else if(match(l,/1\-0/)) { win[opening]++; } opening = 'invalid' } # Tally up draws if(match(l,/1\/2\-1\/2/)) { draw[opening]++ opening = 'invalid' } } END { for(a in win) { print 'W Opening ' a ' won ' win[a] ' times.' } for(a in loss) { print 'L Opening ' a ' lost ' loss[a] ' times.' } for(a in draw) { print 'D Opening ' a ' drew ' draw[a] ' times.' } }' | awk '# Tally total results by opening {result = $1; opening = $3; times = $5; tally[opening] = tally[opening] ' ' result ' ' times; total[opening] += times; } END { for(o in tally) { print o ' ' tally[o] ' T ' total[o] } }' | awk '# Make percentage win/los/draw { opening = $1 for(a in z) { delete z[a]; } for(a=2;a<=NF;a++) { if(a % 2 == 0) { d = $a } else { z[d] = $a } } if(z['T'] > 0) { t = z['T']; if(t != 0) { w = z['W'] / t; l = z['L'] / t; d = z['D'] / t; w *= 100; l *= 100; d *= 100; tt += z['T']; tw += z['W']; tl += z['L']; td += z['D']; } print opening ' Win: ' w '% Loss: ' l '% draw ' d '% Total ' t } } END { if(tt != 0) { tw = tw / tt; tw *= 100; tl = tl / tt; tl *= 100; td = td / tt; td *= 100; } print 'Total Win: ' tw '% Loss: ' tl '% draw ' td '% Total ' tt }'
Here is the final ChessV-0.9.0-generated list (with human tweaking) of Schoolbook openings:
1. c3 Ng6
1. c3 e5
1. c3 e6
1. Nd3 f5
1. Nd3 Nd6
1. Nd3 Ng6
1. e4 e5
1. e4 Ng6
1. e4 d5
1. f4 f5
1. f4 Nd6
1. f4 c6
1. Ng3 Ng6
1. Ng3 e5
1. Ng3 c6
Next: Run 100 40-moves-in-10-second games for each of these openings to see which ones make Joker80 have more wins as white at this time control. Should I point out, at this point, that Joker80’s wins as White decrease when I increase its time control?
Timeline: There is no timeline. Whenever I feel like doing it.
Note that games lost on time do not count; only games with a definite resolution were counted in these statistics (if we played 100 games and 10 were lost on time, I only counted the other 90):
Nd3-Nd6 57%
Nd3-Ng6 56%
Nd3-f5 55%
e4-Ng6 57%
e4-d5 52%
e4-e5 51%
f4-Nd6 60%
f4-f5 52%
f4-c6 47%
c3-e5 58%
c3-Ng6 47%
c3-e6 46%
Ng3-e5 49%
Ng3-Ng6 48%
Ng3-c6 45%
Here, the first column is the first two moves played; the second column is the percent figure as described above. I’ve grouped all of whites moved together in the above list, and have sorted them based on White’s winning percentage if Black makes his best reply.
Anyway, that was fun but this should be my last work on Schoolbook until 2010, when I start preparing for the Schoolbook 2010 tournament. I have a DNS server that I want to finish up that I’ve been neglecting this last week.
Win: 47.2527% Loss: 35.1648% draw 17.5824% score 56.044%
So, then, looking at the games I saw Black wasn't defending as well as he should, so I ran 100 games with three times the time: 40 moves in 30 seconds. I got considerably different results:
Win: 40.8602% Loss: 45.1613% draw 13.9785% score 47.8495%
('score' above is White wins + Draws/2)
Here, we see Black defends a lot better and White doesn't win nearly as often.
Point being, on my system (Core 2 dual, 1.5 ghz), the 40-moves-in-10-seconds games does not give us enough time to play well enough to determine whether a given opening is any good. 1. e4 d5 2. exd5 Nb6 is a fine defense for Black, but we need more time for each game to see that.
As an aside, one of the 30-seconds-for-40-moves games has this pretty checkmate:
1. e4 d5 2. exd5 Nb6 3. Ne2 Nxd5 4. f3 c6 5. c4 Nb6 6. c5 Nd5 7. Bc2 g6 8. Cj3 Ad7 9. Qd1 h5 10. Nhg3 Ke8 11. Bf2 Nb4 12. Ba4 b5 13. cxb6 axb6 14. O-O Ni6 15. d4 Qd6 16. Ne4 Qe6 17. Ci5 j6 18. Nf4 Qc4 19. b3 Qa6 20. Axb4 jxi5 21. Bxc6 Kf8 22. Bxd7 Nh4 23. g3 Cj6 24. Axa6 Cxj2 25. Nxg6+ fxg6 26. gxh4 Cj1# 0-1
I believe his next chess engine (which can already play FIDE chess) does not have this issue.
So, I started my computer making a bunch of 40-moves-in-4-second games last night. There's another issue: About once every 1,000 games, Joker80 crashes, and this stops WinBoard from generating games unattended. I'll see if removing the offending pop-up from WinBoard allows Winboard to restart and continue making games when Joker80 crashes. It may also be necessary to suppress the 'send bug report to Microsoft' window.
The nice thing about games this quickly is that I already have +/- 2% numbers for White's wins with the 1. e4 f5 2. exf5 Nb6 Schoolbook opening:
Win: 45.3718% Loss: 42.868% draw 11.7602% score 51.2519% Total 1318
(Win: Number of White wins. Loss: Number of White losses. Draw: Number of draws. Score: White wins + draws/2 Total: Number of games played)
Compare this to the +/- 5% numbers I got from 100 40-moves-in-60-second games for the same Schoolbook opening yesterday:
Win: 46% Loss: 45% draw 9% score 50.5% Total 100
As we can see, the numbers are almost the same.
Also, can you combine 'wild' with 'board's like Capablanca?
Finally, is the source code to this server available?
In the 2000s, we have had many discussions about what makes for the best Capablanca setup. It has been asserted, for example, that all of the pawns need to be defended in the opening setup, and that any opening setup with undefended pawns gives White a sizable advantage.
What I have done is research a subset of Capablanca setups to see if this is true. Since there are some 126,000 setups where the Queen is to the left of the King and the bishops are on opposite colors with the Capablanca pieces, I have reduced the number of setups to six thusly:
- The rooks must be in the corners. The king must be in the F file. This is to accommodate the software I used for my testing, which only allows castling if these conditions are met.
- The knights must be in the C and H files, and the bishops must be in in the D and G files. The thinking is this: Since the knights are weaker in Capablanca setups on a 10x8 board than they are in FIDE chess, placing the knights relatively near the center makes them more relevant in the opening and midgame. The bishops are placed near the center just like they are placed in FIDE chess; this is done so that natural pawn development moves (e4, f4) do not hamper the development of the bishops.
- The Archbishop (Knight + Bishop), Marshall (Rook + Bishop), and Queen may be placed anywhere in the three remaining files.
- RMNBAKBNQR (I call this arrangement “Notetaker’s Chess”; all pawns are defended in the opening array)
- RQNBAKBNQR (“Schoolbook Chess”; again, all pawns are defended)
- RANBMKBNQR (I call this arrangement “Narcotic Chess”; the B pawn is undefended in the opening array)
- RQNBMKBNAR (I call this arrangement “Opiate Chess”; the D and I pawns are undefended in the opening array and White can threaten mate on the first move with Md3)
- RANBQKBNMR (“Aberg Chess”; the B pawns are undefended)
- RMNBQKBNAR (“Carrera Chess”; the original 10x8 setup from the 1600s; the I pawns are undefneded)
Setup | Wins | Losses | Draws | Games played |
ranbqkbnmr | 46% | 43% | 12% | 1010 |
rmnbqkbnar | 47% | 42% | 12% | 1017 |
ranbmkbnqr | 49% | 41% | 10% | 1002 |
rqnbakbnmr | 48% | 40% | 13% | 1006 |
rqnbmkbnar | 50% | 38% | 11% | 1004 |
rmnbakbnqr | 53% | 37% | 10% | 1011 |
(Numbers may not add up to 100% because of rounding). Since we played 1,000 games with each variant, the scores may be off by as much as 2% or so.
In conclusion, we can see that whether or not all pawns are defended is not a relevant factor in White having more of an advantage. The setup with the lowest White advantage (ranbqkbnmr, White advantage 3%) has one undefended pawn for each side in the opening array, and the setup with the highest White advantage (rmnbakbnqr, White advantage 16%) has all pawns defended in the opening array.
ranbqkbnmr appears to be the most balanced setup (3% White advantage); this is followed by rmnbqkbnar (5% White advantage). ranbmkbnqr and rqnbakbnmr both have an 8% White advantage; ranbmkbnqr has fewer draws, however. Both rqnbmkbnar and rmnbakbnqr give White a considerable advantage (12% and 16%, respectively); I am not surprised rqnbmkbnar has a considerable White advantage, since the D pawn near the King is unprotected, and since White can threaten mate on the first move with Md3. However, it is a mystery why White has such an advantage with the rmnbakbnqr, since all pawns are defended in this setup. It would appear undefended flank pawns don’t give White any significant advantage.
For people interested in this research, I have made a 4-megabyte archive with all of the games played in the course of this research here:
http://www.samiam.org/schoolbook/I would like to thank H.G. Muller for modifying Winboard and creating Joker80; this software made this research reasonably straightforward to perform.
Edit: The Aberg and Carrera setups only have one undefended pawn.
You know, I used to be a Linux fanatic. Then I started using Ubuntu. Now my computer, which I bought as a Linux laptop, runs Windows XP. The problem with Ubuntu is that it’s, as you have observed too unstable: The drivers are too unstable, the software API is too unstable, etc. Since the majority of Linux developers are unpaid, there is no central command and control that can keep the programmers disciplined enough to make a commercial grade desktop product like Windows or MacOS.
The only version of Linux I use today is a free RedHat enterprise Linux clone called CentOS. Hardware support comes slowly; my laptop’s hardware is not fully supported in RHEL/CentOS 5 so I have to wait until CentOS 6 (should be early to mid 2010) before considering running Linux on my computer again [1].
The advantage of RHEL is that, once a given piece of hardware is supported, the OS is supported with bug fixes and security enhancments for a period of seven years. At the start of 2014 Windows XP will still be supported by Microsoft (at least in theory; there is a known bug with XP’s TCP/IP stack if you run a TCP server like Apache or FICS that Microsoft will not fix, since XP isn’t designed to run TCP/IP services); RHEL 5 will still be supported by RedHat. This kind of EOL schedule allows me to upgrade my OS on my timeframe, not the timeframe of some random Open-Source developer on the internet.
Looking at the server, it looks incomplete. Variants do not appear to have “Examine” support and the help files do not describe all of the supported variants.
I think the type of server people are looking for is a server where one can define their own rules (or, at least your own setup and have both players agree on the rules) and move pieces the way you want to. I think play-by-mail makes more sense for Chess variants because, since the game is new, people need more time to think about the next move they will make.
Most Chess variant players like making new Chess variant rules and trying out the new variant for only a few games before moving on to the next variant. I think we’re somewhat unusual in that we would probably be happy playing different Capablanca Chess and FIDE Chess opening setups for the rest of our lives. I even stay with a given variant long enough to develop opening theory for the variant—I think Schoolbook is unique in having an opening book.
[1] In the meantime, I do run CentOS 5 in a virtual machine, mainly for software development purposes.
I am of the opinion that criticisms against legitimate research have no real merit unless backed up by their own research. For example, can you find an opening book for Black to better defend against White’s attacks in the RMNBAKBNQR opening array? It’s one thing to assert “Joker80 doesn’t play the opening very well” (I’m not sure this is what you’re asserting; however, Joker80 was used to play all games, and Zillions was never used); however, this assertion is unsupported unless it is a claim like “Joker80 doesn’t play the RMNBAKBNQR opening very well. Black most commonly replies to 1. e4 with 1. ... e5 (Which, indeed, is the most common opening for this setup); however my research shows that Black gets much better results if he instead responds to 1. e4 with...”
Is there a reasonable Black defense with the RMNBAKBNQR array that Joker80 missed? If so, what is it?
And, yes, the Aberg setup does look to be the best setup of the Capablanca pieces.
Zillions is a general-purpose engine which is excellent for prototyping variants, but it does not play Chess all that well. I use it for basic testing to make sure a given game is sane (no forced mate in the opening, reasonable White-Black balance, not too drawish, etc.), but not for serious opening analysis.
ChessV, with some heuristics to evaluate opening moves, actually comes up with a reasonable opening book for FIDE chess. You should download and try it; I have a copy of it at samiam.org/chessv; to say it plays the opening better than Zillions is a vast understatement. It can’t find the Sicilian defense, but besides that its replies to 1. e4 in FIDE chess (when its opening book is removed) are fine; keep in mind people played FIDE chess for centuries before deciding the Sicilian was a really good reply to 1. e4.
The reason why many Chess engines designed to play FIDE Chess can’t come up with very good opening moves is because they don’t have to. It’s a lot simpler to just have a really big opening book and play moves from the opening book until a novelty is finally played.
Joker80, the engine I used (which, as it turns out, I didn’t write), actually plays the opening quite well, since it’s an engine designed to play Chess without an opening book.
In terms of the opening research, right now I'm researching the White win/Draw ratio of the Schoolbook setup. One concern brought up by Charles Daniel is how well Joker80 is playing the opening. Indeed, earlier I did some opening research by comparing how well Joker80 plays 15 different openings suggested by ChessV (see this posting for the openings). Joker80 was given 10 seconds to play 40 moves, and 100 games of each of the 15 openings was played (1500 games total). Here are the results:
Win: 46% Loss: 42% draw 11% Total 1389 (we played 1,500 games but only tallied games ending in a checkmate or draw)
Here, we see that the White advantage, which is 8% without any Schoolbook opening book, is only 4% when both White and Black play openings suggested by ChessV. Indeed, with the opening 1. e4 d5 2. exd5 Nb6, which appears to be a strong defense to 1. e4 in Schoolbook chess, we get the following results when playing at a time control of 60 seconds for 40 moves:
Win: 44% Loss: 43% draw 13% Total 204
The research I'm currently doing is the White win/loss/draw ratio when playing Schoolbook Chess at a pace of 90 seconds for 40 moves. I will have better results early next week, but here is the results I have so far:
Win: 52% Loss: 35% draw 13% Total 162
Here, we see White does a lot better when given more time to think about his move (but these results may change once I get 1,000 games or so at this slower time control).
I think the biggest conclusion we have gotten from all those 40-move-in-4-second games done with six different Capablanca opening setups is that having undefended flank pawns does not make the game an automatic win for White. It has been proposed (without empirical evidence backing the proposal) that having undefended pawns in a Capablanca opening setup strongly biases the game in White's favor, but I didn't see that at all in my experiment.
In terms of this being a popular variant, one advantage of Capablanca Chess is that there are a number of engines (Zillions, of course, but also ChessV, Joker80, TJchess10x8, etc.) that can play this variant. This allows us to do a lot of research; no Chess variant is popular enough to have tournaments with even dozens of participants, but we can simulate a huge tournament with very strong players and thousands of games played by having computer chess engines play each other over and over and over.
There seem to be two different communities of Chess Variant players: Chess Variant players who don't want to stick around with a given variant for very long; the majority of the community here is like this, which is why, for example, no one has played Schoolbook here for a while (people have since moved on to new variants). Then there's a community only interested in modest variants to Chess and analyzing those modest variants including building up an opening library and have a computer program play the variant in question as well as possible. For example, people making computer engines talk about having the engines play simple variants like Capablanca Chess in places like this forum.
Examining a game means you move pieces around on a board without the game being rated. It would be a great way for people to get a sense of what a variant is like without having to play a rated game with the variant.
There are 31 such pieces; of those 31 pieces, 16 can traverse the entire board (if the board is not a cylinder or toric board; more can traverse the entire board if it is toric depending on the board’s dimensions):
X - X - X - X X X X - X - # - - # - - # - X # X - X - X - X - X - - X - X - X X X X - X - - X - - # - - # - X # X - # - X X X X - X X - X X X X - X - X - X X - X - X - X # X X # X X # X X # X - X - X - X X X X X X X X X X X X X X X X X X X - # - X # X X # X X # X X X X - X - X - X X X XHere, an ‘X’ indicates that the piece can move one or more squares in the indicated direction, as long as the path in question is not blocked by a friendly or enemy piece. This piece can capture an enemy piece by going to the square the enemy piece is at, just like in FIDE Chess.
In the above table, the piece on the third row in the left column is the Rook, and the piece in the lower right corner is the FIDE Queen. The rest of the pieces are fairy pieces; the second piece in the second row is a rider version of Shogi’s Silver; the second piece in the fourth row is a rider version of Shogi’s Gold.
Let’s take an 8x8 board and add FIDE Chess’ pawns and king to the board, putting the pawns on the player’s second row and the King in the E file (just like in FIDE Chess). We then, for the seven remaining positions in the player’s back row, randomly choose one of the 16 above pieces. This results in about 250 million possible opening setups.
How powerful are these pieces? While I haven’t done a full analysis of these pieces, the majority of the pieces are at least as powerful as a rook. This means that White will probably have a strong advantage in the majority of setups; to compensate for this, I would implement the “Pie rule”: Player one chooses White’s first move, then player two chooses whether to play white or black.
Castling could be handled by allowing the king to, once in the game, swap pieces with another piece, as long as the king and the other piece have not yet moved.
If 250 million is not enough possible opening setups, or if people find the pieces too powerful, we can have each piece, instead of being able to move any number of unobstructed squares, randomly choose whether a piece can only move one square in a given direction, be able to move square or leap two squares in that direction, or be able to move any number of unobstructed squares in the direction. This results in 15,120 different possible pieces. If we have seven such pieces, we have some 180,660,221,412,287,006,638,080,000,000 possible opening setups.
If we insist on left-right symmetry in all aspects, we have 882 different pieces to play with. If we make the game compatible with a FIDE chess board, where the pieces representing the rooks, knights, bishops, and queens are in their FIDE opening position, there are 605,165,749,776 possible opening setups.
Have I tested these variants? Nope; all it is so far is an interesting thought experiment.
Personally, I prefer the term 'defence density' for this heuristic, since King safety is more difficult in Capablanca Chess than in FIDE Chess.
I could not find the download on your web page nor in the Winboard forum.
This will make Schoolbook 2010 a lot easier to implement.
It is.
I had my computer spend all weekend playing over 1,000 games of the Schoolbook setup giving Joker80 90 seconds to make 40 moves (each side got 90 seconds to make these 40 moves). Here are the win/loss/draw ratio for White when given all this time to make a move:
Win: 48% Loss: 39% draw 13% Total 1002
Compare this to the results we got when we play much faster 40-moves-in-4-seconds games:
Win: 48% Loss: 40% Draw: 13% Total 1006
The numbers are identical, within the +-2% margin of error because we used 1,000 games in our study.
Conclusion: Giving a game different time controls does not appear to affect the Win/Loss/Draw ratio for Joker80.
My next bit of research is whether fine-tuning the opening by hand affects the Win/Loss/Draw ratios. In other words, can we come up with a better opening book than the openings Joker80 comes up with during the game?
It looks like I can. I don't have over 1,000 games for a given opening, but my preliminary findings show that Black has about as many wins as White after 1. e4 d5 2. exd5 Nb6 and Black creams White after what I call the “Winther Defense”, 1. f4 c5 (named thusly because Zillions, when using a Capablanca rules file tweaked by Winther to play the opening better, came up with this defense); it looks like the tactical complications after 1. f4 c5 2. Bxc5 Qxf4+ 3. Af3 strongly favor Black.
FIDE pawn: Moves forward, captures diagonally forward
Shogi pawn: Moves forward, captures forward
Berolina pawn: Moves diagonally forward, captures forward
Beroshogi pawn: Moves diagonally forward, captures diagonally forward
If you want five more pawns, we can have the following fairy pawns:
Moves forward and diagonally forward, captures forward.
Moves forward and diagonally forward, captures diagonally forward.
Moves forward and diagonally forward, captures forward and diagonally forward.
Moves forward, captures forward and diagonally forward
Moves diagonally forward, captures forward and diagonally forward
OK, is that not enough types of pawns. We can add Winther’s “Scorpion Pawn” movement: Moves but doesn’t capture one square forward and two squares to the left, and one square forward and two squares to the right:
. . . . . * . . . * . . # . . . . . . . . . . . .Here ‘#’ is the pawn and ‘*’ is the squares the Scorpion move allows the pawn to move to.
This gives us 18 pawn types. If that’s not enough, we can add a move I call a “caltrop” move, which is the other two forward knight moves to a pawn:
. * . * . . . . . . . . # . . . . . . . . . . . .This gives us 36 pawn types...but the pawns are getting pretty powerful at this point. I start worrying about White having an unfair advantage when the pawns get too powerful...
As an aside, what is a Ninja pawn?
Betza didn't use these pieces together because his goal was to make a variant using the FIDE pieces that was as strong as the FIDE army. You can't have the RN and BN with the Queen if you want to do that. However, in CWDA, Betza added the RN to the “Remarkable Rookies” army and the BN to the “Colorbound Cobblers” army, so he obviously had no problem with the pieces, just with having too many of them on the board at once. Also, look at “Almost Chess”.
Speaking of Betza, I wonder about a Capablanca variant where the knights are different; one is a Betza Crab + Ferz; the other knight is Crab + Wazir. Instead of an Archbishop, we have a Crab + Wazir + Bishop; instead of a Marshall/Cardinal, we have a Crab + Ferz + Rook. Should be about as powerful as the Capablanca army, and allows 252,000 opening setups instead of “only” 126,000 opening setups.
25 comments displayed
Permalink to the exact comments currently displayed.
With the version of WinBoard I'm using, there is no option for animation (and no animation in the game) nor an option for turning off the GUI (-noGUI doesn't work, nor does /noGUI, and the help files don't seem to have anything). I'm using Winboard 4.3.15m.
Which version of Winboard are you using? It's in the GUI under Help->about.