Comments/Ratings for a Single Item

Reinhard, that is not relevant. It will happen on the average as often for
the other side. It is in the nature of Chess. Every game that is won, is
won by an error, that might not have been made on longer thinking. As the
initial position is not a won position for eaiter side. But most games are
won by either side, and if they are allowed to think longer, most games are
still won by either side.

What is so hard to understand about the statement 'the win probability
(score fraction, if you allow for draws) obtained from a given quiet, but
complex (many pieces) position between equal opponents does not depend on
time control' that it prompt people to come up with irrelevancies? Why do
you think that saying anything at all that does not mention an observed
probability would have any bearing on this statement whatsoever?

I don't think the ever more hollow sounding selfdeclared superiority of
Derek need much comment. He obviously doesn't know zilch about
probability theory and statistics. Shouting that he does won't make it
so, and won't fool anyone.

This discussion is too silly for words anyway. Because even if it were true that the winning probability for a given material imbalance would be different at 1 hour per move than it would be at 10 sec/move, it would merely mean that piece values are different for different quality players. And although that is unprecedented, that revelation in itself would not make the piece values at 1 hour per move of any use, as that is a time control that no one wants to play anyway.

So the whole endeavor is doomed from the start: by testing at 1 hour per move, either you measure the same piece values as you would at 10  sec/move, and wasted 99.7% of your time, or you find different values, and then you have wrong values, which cannot be used at any time control you would actually want to play...

Here is another approach I would suggest for strength of pieces.  How about
we pick 100 and people order them from strongest to weakest?  Work on a
scoring system for position, and then at least get an idea of order of
strength.

Anyone think this might be a sound approach?

Rich Hutnik:
| Anyone think this might be a sound approach?

Well, not me! Science is not a democracy. We don't interview people in
the street to determine if a neutron is heavier than a proton, or what the 100th decimal of the number pi is.

At best, you could use this method to determine the CV rating of the
interviewed people. But even if a million people would think that piece A
is worth more than piece B, and none the other way around, that doesn't
make it so. The only thing that counts is if A makes you win more often
than B would. If it doesn't, than it is of lower value. No matter what people say, or how many say it.

To anyone who was interested ...

My playtesting efforts using SMIRF have been suspended indefinitely due to a serious checkmate bug which tainted the first game at 30 minutes per move between Scharnagl's and Muller's sets of CRC piece values.

Since Muller's Joker80 has recently established itself via 'The Battle Of
The (Unspeakables)' tournament as the best free CRC program in the world,
I checked it out.  I must report that setting-up Winboard F (also written
by Muller) to use it was straight-forward with helpful documentation. 
Generally, I am finding the features of Joker80 to be versatile and
capable for any reasonable uses.

Muller:

I would like to conduct two focused playtests using Joker80 at very long
time controls (e.g., 30 minutes per move) to investigate these important questions-

1.  Is Muller's rook value within the CRC set too low?
2.  Is Scharnagl's archbishop value within the CRC set too low?

I would need for you to compile special versions of Joker80 for me using
significantly different values for those CRC pieces as well as
Scharnagl's CRC piece set.  To isolate the target variable, these games would be Muller (standard values) vs. Muller (test values) and Scharnagl (standard values) vs. Scharnagl (test values) via symmetrical playtesting.  Anyway, we can discuss the details if you are interested or willing.  Please let me know.

This sounds like an interesting proposition.

Muller:

Please investigate this potentially serious bug I may have discovered
while testing Joker80 under Winboard F ...

Bugs, Bugs, Bugs!
http://www.symmetryperfect.com/pass

I am having a hard time with software today.

'Human vs. engine play is virtually untested. 
Did you at any point of the game use 'undo'
(through the WinBoard 'retract move')?'

Yes.
Many of us error-prone humans use it frequently.
________________________________________________

'This is indeed something I should fix but
the current work-around would be not to use 'undo'.'

Makes sense to me.
I can avoid using the 'retract move' command altogether.
________________________________________________________

'I could make a Joker80 version that reads the piece base values from a
file 'joker.ini' at startup. Then you could change them to anything you
want to test, without the need to re-compile. Would that satisfy your
needs?'

Yes, better than I ever imagined.
Thank you!

OK, I replaced the joker80.exe on my website by one with adjustable piece
values. (If you run it from the command line, it should say version 1.1.14
(h).) I also tried to fix the bug in undo (which I discoverd was disabled
altogether in the previous version), and although it seemed to work, it
might remain a weak spot. (I foresee problems if the game contained a
promotion, for instance, as it might not remember the correct promotion
piece on replay.) So try to avoid using the undo.

I decided to make the piece values adjustable through a command-line
option, rather than from a file, to avoid problems if you want to run two
different sets of piece values (where you then would have to keep the
files separate somehow). The way it works now is that for the engine name
(that WinBoard asks in the startup dialog, or that you can put in the
winboard.ini file to appear in the selectable engines there), you should
write:

joker80.exe P85=300=350=475=875=900=950

The whole thing should be put between double quotes, so that WinBoard
knows the P... is an option to the engine, and not to WinBoard. The
numerical values are those of P, N, B, R, A, C and Q, respectively, in
centiPawn. You can replace them by any value you like. If you don't give
the P argument, it uses the default values. If you give a P argument with
not enough values, the engine exits.

Note that these are base values, for the positionally average piece. For N
and B this would be on c3, in the presence (for B) of ~ 6 own Pawns, half
of them on the color of the Bishop. A Bishop pair further gets 40cP bonus.
For the Rook it is the value for one in the absence of (half-)open files.
The Pawn value will be heavily modified by positional effects
(centralization, support by own Pawns, blocking by enemy Pawns), which on
the average will be positive.

Note that you can play two different versions against each other
automatically. The first engine plays white, in two-machines mode. (You
won't be able to recognize them from their name...)

One small refinement:

If the command-line argument was used to modify the piece values, Joker80
will give its own name to WinBoard as 'Joker80.xp', in stead of
'Joker80.np', so that it becomes less hard to figure out which engine
was winning (e.g. from the PGN file).

Note also that at very long time control you might want to enlarge the
hash table; default is 128MB, but if you invoke Joker80 as

'joker80.exe 22 P100=300=....'

it will use 256MB (and with 23 in stead of 22 it will use 512MB, etc.)

Everything is working fine.
Thank you!

I now have 12 instances of the Joker80 program running in various
sub-directories of Winboard F with the 'winboard.ini' file set to
conveniently initiate any desired standard or special material values for
the CRC models by Muller, Scharnagl and Nalls.

In the first test, I am going to attempt to find a playtesting time where
a distinct seperation in playing strength occurs between the standard
Muller model wherein the rook is 1 pawn more valuable than the bishop and
a special Muller model wherein the rook is 2 pawns more valuable than the
bishop.  If I successfully find a playtesting time that is survivable by
humans, then we can hopefully establish a tentative probability as to
which CRC model plays decisively better after a few-several games.

At par 100 (for the pawn), the bishop is at 459 under both models with the
rook at 559 under the standard Muller model and 659 under the special
Muller model.

I want to playtest a special Muller model with a rook value 2.00 pawns higher than the bishop because the Nalls model has a rook value 2.19 pawns higher than the bishop and the Scharnagl model has a rook value 1.94 pawns higher than the bishop (for an average of 2.06 pawns).

Since I am attempting to test for such a small difference in the material value of only one type of piece (the rook), I have doubts that I will be able to obtain conclusive results.  In any case ... If I obtain conclusive results, then very long time controls will surely be required to produce them.

Well, to get an impression at what you can expect: In my first versions of
Joker80 I still used the Larry-Kaufman piece values of 8x8 Chess. So the
Bishop was half a Pawn too low, nearly equal to the Knight (as with more
than 5 Pawns, Kaufman has a Knight worth more than a lone Bishop,
neutraling a large part of the pair bonus.) Now unlike a Rook, a Bishop is
very easy to trade for a Knight, as both get into play early. Making the
trade usually wrecks the opponent's pawn structure by creating a doubled
Pawn, giving enough compensation to make it attractive.

So in almost all games Joker played with two Knights against two Bishops
after 12 moves or so. Fixing that did increase the playing strength by
~100 Elo points. So where the old version would score 50%, the improved
version would score 57%.

Now a similarly bad value for the Rook would manifest itself much more
difficultly: the Rooks get into play late, there is no nearly equal piece
for which a 1:1 trade changes sign, and you would need 1:3 trades (R vs
B+2P) or 2:2 trades (R+P for N+N), which are much more difficult to set
up. So I would expect that being half a Pawn off on the Rook value would
only reduce your score by about 3%, rather than 7% as with the Bishop.
After playing 100 games, the score differs by more than 3% from the true
win probability more often than not. So you would need at least 400 games
to show with minimal confidence that there was a difference.

Beware that the result of the games are stochastic quantities. Replay the
game at the same time control, and the game Joker80 plays will be
different. And often the result will be different. This is true at 1 sec
per move, but it is equally true at 1 year per move. The games that will
be played, are just a sample from the myriads of games Joker80 could play
with non-zero probability. And with fewer than 400 games, the difference
between the actually measured score percentage and the probability you
want to determine will be in most cases larger than the effect of the
piece values, if they are not extremey wrong (e.g. setting Q < B).

Of course, I would bet anything that there are no 1:1 exchanges supported
under the standard Muller CRC model that could cause material losses.  If
that were the case, yours would not be one of the three most credible CRC
models under close consideration.  In fact, even your excellent Joker80
program would play poorly if stuck with using faulty CRC piece values.

Obviously, the longer the exchange, the rarer its occurrence during
gameplay.  The predominance of simple 1:1 exchanges over even the least
complicated, 1:2 or 2:1 exchanges, in gameplay is large although I do not
know the stats.

In fact, there is a certain 1:2 or 2:1 exchange I am hoping to see that is
likely to support my contention that the Muller rook value should be
higher: the 1 queen for 2 rooks or 2 rooks for 1 queen exchange.  Please
recall that under the standard Muller model, this is an equal exchange. 
However, under asymmetrical playtesting of comparable quality to and
similar to that I used to confirm the correctness of your higher
archbishop value, I played numerous CRC games at various moderate time
controls where the player without 1 queen (yet with 2 rooks) defeated the
player without 2 rooks (yet with 1 queen).  Ultimately, a key mechanism to conclusive results is that while the standard Muller model is neutral toward a 2 rook : 1 queen or 1 queen : 2 rook exchange, the special Muller model regards its 1 queen as significantly less valuable than 2 rooks of its opponent.  Consequently, this contrast in valuation could be played into ... and we would see who wins.

I am actually pleased that you are a realist who shares my pessimism in
this experiment.  In any case, low odds do not deter a best effort to
succeed.  The main difference between us is that you calculate your
pessimism by extreme statistical methods whereas I calculate my pessimism
by moderate probabilistic methods.  I remain hopeful that eventually I
will prove to you that the method Scharnagl & I developed is occasionally
productive.

Muller:

Please confirm that these are legal values for the 'winboard.ini' file.

/firstChessProgramNames={'C:\winboard-F\Joker80\w\M-st\w-M-st 22
P100=353=459=559=1029=1059=1118'
'C:\winboard-F\Joker80\w\M-sp\w-M-sp 22
P100=353=459=659=1029=1059=1118'
'C:\winboard-F\Joker80\w\S-st\w-S-st 22
P100=306=363=557=702=912=960'
'C:\winboard-F\Joker80\w\S-sp\w-S-sp 22
P100=306=363=557=866=912=960'
'C:\winboard-F\Joker80\w\N-st\w-N-st 22
P100=308=376=594=940=958=1031'
'C:\winboard-F\Joker80\w\N-sp\w-N-sp 22
P100=308=376=594=940=958=1031'
'C:\winboard-F\TJchess\TJChess10x8'
}
/secondChessProgramNames={'C:\winboard-F\Joker80\b\M-st\b-M-st 22
P100=353=459=559=1029=1059=1118'
'C:\winboard-F\Joker80\b\M-sp\b-M-sp 22
P100=353=459=659=1029=1059=1118'
'C:\winboard-F\Joker80\b\S-st\b-S-st 22
P100=306=363=557=702=912=960'
'C:\winboard-F\Joker80\b\S-sp\b-S-sp 22
P100=306=363=557=866=912=960'
'C:\winboard-F\Joker80\b\N-st\b-N-st 22
P100=308=376=594=940=958=1031'
'C:\winboard-F\Joker80\b\N-sp\b-N-sp 22
P100=308=376=594=940=958=1031'
'C:\winboard-F\TJchess\TJChess10x8'
}

It looks OK to me.

One caveat: the normalization (e.g. Pawn = 100) is not completely
arbitrary, as the engine weights material against positional terms, and
doubling all piece values would effectively scale down the importance of
passers and King Safety.

In addition, the engine also uses some heavily rounded 'quick' piece
values internally, where B=N=3, R=5, A=C=8 and Q=9, to make a rough guess
if certain branches stand any chance to recoup the material it gave
earlier in the branche. So in certain situations, when it is behind 800
cP, it won't consider capturing a Rook, because it expects that to be
worth about 500 cP, and thus falls 300 cP below the target. Such a large
deficit would be beyond the safety margin for pruning the move. But if the
piece values where scaled up such that the 800 merely represented being a
Bishop behind, this obviously would be an unjustified pruning.

The safety margin is large enough to allow some leeway here, but don't
overdo it. It would be safest to keep the value of Q close to 950.

I am indeed skeptical to the possibility to do enough games to measure the
difference you want to see in the total score percentage. But perhaps some
sound conclusions could be drawn by not merely looking at the result, but
at the actual games, and single out the Q vs 2R trades. (Or actually any
Rook versus other material trade before the end-game. Rooks capturing
Pawns to prevent their promotion probably should not count, though.) These
could then be used to separately extracting the probability for such a
trade for the two sets of piece values, and determine the winning
probability for each of the piece values once such a trade would have
occurred. By filtering the raw data this way, we get rid of the stochastic
noise produced by the (majority of) games whwre the event we want to
determine the effect of would not have occurred.

As I moved to renormalize all of the values used in Joker80 (written into
the 'winboard.ini' file) with the pawn at a par of 85 points, I looked
at my notes again.  They reminded me that your use of the 'bishop pair'
refinement (with a bonus of 40 points) ramifies that the material value of
the rook is either 1.00 pawns or 1.47 pawns greater than the material value
of the bishop in CRC, depending upon whether or not only one bishop or both
bishops, respectively, remain in the game.  At that point, I realized that
I would be attempting to playtest for a discrepancy that I know from
experience is just too small to detect even at very long time controls. 
So, this planned test has been cancelled.

I am not implying that this matter is unimportant, though.  I remain
concerned for the standard Muller model whenever it allows the exchange of
its 2 rooks for 1 queen belonging to its opponent.

Well, I share that concern. But note that the low Rook value was not only
based on the result of Q-2R assymetric testing. I also played R-BP and
NN-RP, which ended unexpectedly bad for the Rook, and sets the value of
the Rook compared to that of the minor pieces. While the value of the
Queen was independently tested against that of the minor pieces by playing
Q-BNN.

The low difference between R and B does make sense to me now, as the wider
board should upgrade the Bishop a lot more than the Rook. The Bishop gets
extra forward moves, and forward moves are worth a lot more than lateral
moves. I have seen that in testing cylindrical pieces, (indicated by *),
where the periodic boundary condition w.r.t. the side edges effectifely
simulates an infinitely wide board. In a context of normal Chess pieces,
B* = B+P, while R* = R + 0.25P. OTOH, Q* = Q+2P. So it doesn't surprise
me that on wider boards R loses compared to Q and B.

I can think of several systematic errors that lead to unrealistically poor
performance of the Rook in asymmetric playtesting from an opening position.
One is that Capablanca Chess is a very violent game, where the three
super-pieces are often involved in inflicting an early chekmate (or nearly
so, where the opponent has to sacrifice so much material to prevent the
mate, that he is lost anyway). The Rooks initially offer not much defense
against that. But your chances for such an early victory would be strongly
reduced if you were missing a super-piece. So perhaps two Rooks would do
better against Q after A and C are traded. This explanation would do
nothing for explaining poor Rook performance of R vs B, but perhaps it is
B that is strong (it is also strong compared to N). The problem then would
be not so much low R value, but high Q value, due to cooperativity between
superpieces. So perhaps the observed scores should not be entirely
interpreted as high base values for Q, C and A, but might be partly due to
super-piece pair bonuses similar to that for the Bishop pair. Which I would
then (mistakenly) include in the base value, as the other super-pieces are
always present in my test positions.

Another possible source of error is that the engine plays a strategy that
is not well suited for playing 2R vs Q. Joker80's evaluation does not
place a lot of importance to keeping all its pieces defended. In general
this might be a winning strategy, giving the engine more freedom in using
its pieces in daring attacks. But 2R vs Q might be a case where this
backfires, and where you can only manifest the superiority of your Rook
force by very careful and meticulous, nearly allergic defense of your
troops, slowly but surely pushing them forward. This is not really the
style of Joker's play. So it would be interesting to do the asymmetreic
playtesting for Q vs 2R also with other engines. But TJchess10x8 only
became available long after I started my piece value project, TSCP-G does
not allow setting up positions (although now I know a work-around for
that, forcing initial moves with both ArchBishops to capture all pieces to
delete, and then retreating them before letting the engine play). And Smirf
initially could not play automatically at all, and when I finally made a WB
adapter for it so that it could, fast games by it where more decided by
timing issues than by play quality (many losses on time with scores like
+12!). And Fairy-Max is really a bit too simplistic for this, not knowing
the concept of a Bishop pair or passed pawns, besides being a slower
searcher.

Muller:

Please have another look at this except from my 'winboard.ini' file. 
There are standard and special versions of piece values by Muller,
Scharnagl & Nalls for the white and black players renormalized to pawn =
85 points.

The special version of the Muller model has a rook value exactly 85 points
or 1.00 pawn higher than the standard version.

The special version of the Scharnagl model has an archbishop value (736
points) at appr. 95% of the archbishop value (775 points) instead of 597
points at appr. 77% for the standard version.

The special version of the Nalls model is identical to the standard
version until some test is needed and planned.

Since I assume that the 'bishop pairs bonus' is hardwired into Joker80,
40 points has been subtracted from the model-independant, material values
of the bishop under all three models.  Is this correct?
_____________________________________________________

/firstChessProgramNames={'C:\winboard-F\Joker80\w\M-st\w-M-st 22
P85=300=350=475=875=900=950'
'C:\winboard-F\Joker80\w\M-sp\w-M-sp 22
P85=300=350=560=875=900=950'
'C:\winboard-F\Joker80\w\S-st\w-S-st 22
P85=260=269=474=597=775=816'
'C:\winboard-F\Joker80\w\S-sp\w-S-sp 22
P85=260=269=474=736=775=816'
'C:\winboard-F\Joker80\w\N-st\w-N-st 22
P85=262=279=505=799=815=876'
'C:\winboard-F\Joker80\w\N-sp\w-N-sp 22
P85=262=279=505=799=815=876'
'C:\winboard-F\TJchess\TJChess10x8'
}
/secondChessProgramNames={'C:\winboard-F\Joker80\b\M-st\b-M-st 22
P85=300=350=475=875=900=950'
'C:\winboard-F\Joker80\b\M-sp\b-M-sp 22
P85=300=350=560=875=900=950'
'C:\winboard-F\Joker80\b\S-st\b-S-st 22
P85=260=269=474=597=775=816'
'C:\winboard-F\Joker80\b\S-sp\b-S-sp 22
P85=260=269=474=736=775=816'
'C:\winboard-F\Joker80\b\N-st\b-N-st 22
P85=262=279=505=799=815=876'
'C:\winboard-F\Joker80\b\N-sp\b-N-sp 22
P85=262=279=505=799=815=876'
'C:\winboard-F\TJchess\TJChess10x8'
}

Is there any special reason you want to keep the Pawn value equal in all
trial versions, rather than, say, the total value of the army, or the
value of the Queen? Especially in the Scharnagl settings it makes almost
every piece rather light compared to the quick guesses used for pruning.

Note that there are so many positional modifiers on the value of a pawn
(not only determined by its own position, but also by the relation to
other friendly and enemy pawns) that I am not sure what the base value
really means. Even if I say that it represents the value of a Pawn at g2,
the evaluation points lost on deleting a pawn on g2 will depend on if
there are pawns on e- and i-file, and how far they are advanced, and on
the presence of pawns on the f- and h-file (which mighht become backward
or isolated), and of course if losing the pawn would create a passer for
the opponent.

If I were you, I would normalize all models to Q=950, but then replace
the
pawn value everywhere by 85 (I think the standard value used in Joker is
even 75). I don't think you could say then that you deviate from the
model, as the models do not really specify which type of Pawn they use as
a standard. My value refers to the g2 pawn in an opening setup. Perhaps
Reinhard's value refers to an 'average' pawn, in a typical pawn chain
occurring in the early middle game, or a Pawn on d4/e4 (which is the most
likely to be traded).

As to the B-pair: tricky question. The way you did it now would make the
first Bishop to be traded of the value the model prescribes, but would
make the second much lighter. If you would subtract half the bonus, then
on the average they would be what the model prescribes. The value is
indeed hard-wired in Joker, but if you really want, I could make it
adjustable through a 8th parameter.

'If I were you, I would normalize all models to Q=950 but then replace
the pawn value everywhere by 85.'

Since this is what you (the developer of Joker80) recommend as optimum, 
this is what I will do.

Are you sure that replacing any pawn values different than 85 points
after renormalization to queen = 950 points still renders an accurate 
and complete representation, more or less, of the Scharnagl and Nalls 
models?

At par of queen = 950 points, the pawn value in the Nalls model
is not represented as being only 92.19% as high as that in the Muller 
model and the pawn value in the Scharnagl model is not represented
as being only 98.95% as high as that in the Muller model.

Thru it all ... If a perfect representation is not quite possible, 
I can accept that without reservation.
__________________________________

'I don't think you could say then that you deviate from the
model as the models do not really specify which type of Pawn they use as
a standard.'

Correctly calculating pawn values at the start of the game (much less, 
throughout the game) requires finesse as it is indeed a complex issue.
In fact, its excessively complexity is the reason my 66-page paper on
material values of pieces is silent in the case of calculating pawn values
in FRC & CRC.  Instead, someone needs to read an entire book from an 
outside source about calculating the material values of the pieces in 
Chess to sufficiently understand it.

Personally, I am content with the test situation as long as Joker80 
handles all pawns under all three models initially valued at 85 points
as fairly and equally as realistically possible.

I cannot speak for Reinhard Scharnagl at all, though.
________________________________________________

'The way you did it now would make the first Bishop to be traded of the 
value the model prescribes, but would make the second much lighter. 
If you would subtract half the bonus, then on the average they would 
be what the model prescribes.'

Now, I understand better.
It makes sense.
[I am glad I asked you.]

Yes, I will subtract 20 points (1/2 of the 'bishop pair bonus') from the
model-independant, material values for the bishop under the 
Scharnagl & Nalls models.

Muller:

Here is my latest revision to my 'winboard.ini' file.
Are these piece values acceptable to you?
Do you think these piece values will work smoothly with Joker80 running
under Winboard F yet remain true to all three models?
______________________________________________________

/firstChessProgramNames={'C:\winboard-F\Joker80\w\M-st\w-M-st 22
P85=300=350=475=875=900=950'
'C:\winboard-F\Joker80\w\M-sp\w-M-sp 22
P85=300=350=560=875=900=950'
'C:\winboard-F\Joker80\w\S-st\w-S-st 22
P85=302=339=551=694=902=950'
'C:\winboard-F\Joker80\w\S-sp\w-S-sp 22
P85=302=339=551=857=902=950'
'C:\winboard-F\Joker80\w\N-st\w-N-st 22
P85=284=326=548=866=884=950'
'C:\winboard-F\Joker80\w\N-sp\w-N-sp 22
P85=284=326=548=866=884=950'
'C:\winboard-F\TJchess\TJChess10x8'
}
/secondChessProgramNames={'C:\winboard-F\Joker80\b\M-st\b-M-st 22
P85=300=350=475=875=900=950'
'C:\winboard-F\Joker80\b\M-sp\b-M-sp 22
P85=300=350=560=875=900=950'
'C:\winboard-F\Joker80\b\S-st\b-S-st 22
P85=302=339=551=694=902=950'
'C:\winboard-F\Joker80\b\S-sp\b-S-sp 22
P85=302=339=551=857=902=950'
'C:\winboard-F\Joker80\b\N-st\b-N-st 22
P85=284=326=548=866=884=950'
'C:\winboard-F\Joker80\b\N-sp\b-N-sp 22
P85=284=326=548=866=884=950'
'C:\winboard-F\TJchess\TJChess10x8'
}