Comments by DerekNalls

The proof Nalls' 50-page program is comparatively inadequate is that he
will not likewise use it for such estimatations on command.  Since his
system is only good for three sets now (FRC,CRC and the one of his own),
it apparently does not even achieve hit-or-miss or willynilly import.
_____________________________________________________________________

You are too gracious.  In fact, I only know my model to be reliable for
two sets now- FRC & CRC.  It could be way off track for one set- Hex Chess SS.

To be sure, I appreciate your efforts on behalf of Joe Joyce. 
Unfortunately, despite your very good intentions, I also regard your
efforts warily as borderline irresponsible.

Contrary to naive intuition, with relative piece values, guesses can be
worse than using no values at all.

If you have no relative piece values to play by, then you will naturally
use caution in forcing exchanges that are not obviously advantageous (or
allowing your opponent to force exchanges upon you).

If you have relative piece values that you hope are instructive yet are,
in reality, too inaccurate, then you will feel justified in forcing
exchanges that are allegedly advantageous (or allowing your opponent to force exchanges upon you).

This doomed course of action will cause you to lose repeatedly.  The
material nature of the loss will not be immediately clear since you will
assume this could not possibly be the problem.  Instead, you will
incorrectly attribute the loss to a positional shortcoming or poor move at a critical junction and for instance, analyze every move in the opening
game in detail.  Of course, all of these efforts will fail to solve the
problem and you will still continue to lose.

The moral of the story is that if you know nothing, admit it since other
courses of action can be disastrous.  Be honest, realistic and
responsible to the greatest extent possible.

In the total absence of feedback, you have no means of obtaining any
vitally needed experimental knowledge regardless of how high your
intelligence may be.  You are like a man target shooting blindfolded or a
man hunting gold with a metal detector wearing earplugs.

Please remember that reliable relative piece values have only been
established, to date, for FRC & CRC.  These are the only two testbeds
available for ANY model.  Forays into other games, if you must make them,
should at least carry a strong 'use-at-your-own-risk warning'.

It is contradictory and arrogant on your part to hold no confidence in my
model yet hold irrationally too much confidence in your model.  As far as
your model goes, you are as confident as you are reckless.  Good luck!

It is useful to classify inaccuracies and try to define how much inaccuracy
is too much with relative piece values.

The first, most dangerous inaccuracy is what I classify as a 'direct
inversion'.  A direct inversion is where two pieces with significantly
different values have their order of value reversed from its true
existence.

I am referring to more than a trivial case of, for example, mistakenly
defining the knight (30.00- DN model) as more valuable than the bishop
(32.42- DN model) upon an 8 x 8 board IF the reverse is actually true
since the values of these two pieces are truly very close.

Instead, I am referring to a non-trivial case of, for example, defining
the rook (59.43- DN model) to be more valuable than the archbishop (70.61-
DN model) upon a 10 x 8 board where the reverse is actually true.  Under
such a mistaken belief, a player willfully enters disadvantageous, simple
1-to-1 piece exchanges involving his/her archbishop for the opponent's
rook.  If any game is won where this exchange has occurred, it is against
the odds.  Incidentally, such simple exchanges are realistically likely to
occur in typical games.

I think most of us would agree this is too much inaccuracy.

The second, potentially-dangerous inaccuracy is what I classify as an
'indirect inversion'.  An indirect inversion is where, despite the
hierarchy of values for the lineup of pieces being correct, the numerical
erraticities within it are great enough to cause incorrect conclusions in
evaluating complex exchanges involving more than one piece per player.

Derek Nalls
relevant FRC pieces upon the 8 x 8 board
material values

knight-  3.000
bishop-  3.242
rook-    5.088
queen-   9.371
_____________

Reinhard Scharnagl
relevant FRC pieces upon the 8 x 8 board
material values

knight-   3.0000
bishop-   3.4488
rook-     5.3030
queen-    9.0001
______________

Note that under the RS model, 1 queen + 1 knight (2 pieces) is valued at a
total of 12.0001 and 2 bishops + 1 rook (3 pieces) is valued at a total of
12.2006.  It values the 3 pieces 0.2005 higher than the 2 pieces- a
marginal amount.  In practice, it would probably be indifferent to this
exchange.

Note that under the DN model, 1 queen + 1 knight (2 pieces) is valued at a
total of 12.371 and 2 bishops + 1 rook (3 pieces) is valued at a total of
11.572.  It values the 2 pieces 0.799 higher than the 3 pieces- a
significant amount.  In practice, it would probably aggressively pursue
this exchange.

Due to their contrasting evaluations of this complex 2-to-3 pieces
exchange, both players (RS & DN) would willfully enter opposite sides of
this exchange as being advantageous.  Unless both models are inaccurate so
that, in fact, this exchange is absolutely neutral to the interests of both
players, one player who willfully enters this exchange will get harmed by
it and probably, eventually lose the game.

Predictably, it is my contention that a player who trades 1 queen + 1
knight for 2 bishops + 1 rook will probably, eventually lose the game for
a reason, albeit indirect and less effectual, based upon the fact that a
player who trades 1 queen for 1 bishop + 1 rook will probably, eventually
lose the game.  However, such complex exchanges occur rarely in typical
games.  In fact, the example exchange never occurred between 2 versions of
SMIRF that Reinhard Scharnagl compiled for playtesting- 1 using his piece
values, 1 using my piece values.  So, I was never had the opportunity to
see my point proven.

Still, I am discontent with this type of subtle inaccuracy.  How do the
rest of you regard it?

We all agree that the relative piece values for FRC and CRC (to a lesser
extent) are reasonably well-established.  I think we should reaffirm WHY
(even if it seems too obvious to some of us) in order to pinpoint what
important steps need to be taken to bring other desirable chess variants
into our realm of understanding.

Thru much human effort, relative piece values for Chess (FRC) were
understood with only a little less accuracy than today long before chess
computers and programs attained impressive playing strength.
Notwithstanding, powerful computers and AI programs are now available and
affordable even to individuals in the modern era.  Accordingly, I think
this great resource should never be neglected and furthermore, should be
regarded as indispensible to our future endeavors.

Even in the absence of any predictive theory, a powerful program,
custom-written to play a single chess variant as well as possible, can
determine the correct relative piece values for an entire lineup of
pieces.  The greater the depth (in plies), time or number of positions
searched per move throughout a playtested game, the more narrowly it can
define the range of correct values for each unique piece (although a
tantalizingly-large, range of values remains with any game playtested at
survivable times using today's state-of-the-art technology).

Since FRC & CRC are fairly, closely related, it seems probable that no
predictive, universal model for relative piece values will mature until
additional reliable, experimental testbeds involving less-related chess
variants have been created to test results against.

Forget about the Zillions Of Games program.  It only plays chess variants
that are closely related to Chess reasonably well- NOT great!- when given
a lot of time per move.  The less related a given chess variant is to
Chess, the worse the ZOG program plays to the point of taking an enormous
amount of time to make poor moves.

The recent development of achieving within-range relative piece values for
CRC is a useful roadmap.  How did it happen?  Out of appr. 8 billion people
worldwide, an adequate number of individuals took an interest in learning
to play one of a few popular Capablanca Chess variants very well.  A
minority of these chess variant players succeeded at their goal.  For
whatever reasons, three programs were written and made available for free
for the worldwide popular IBM-compatible, MS Windows configuration that
the best human players confirmed to be strong.  In the course of making
each of these three programs as strong as possible at playing one another
and some of the best human players, the relative piece values for CRC were
refined to the point that improvements in playing strength no longer came
easily and quickly with adjustments.

How many efforts of this magnitude is the worldwide chess variant
community capable of?  In any case, we need at least a few more.

'If Archbishop and Chancellor have equal value, it DOES NOT IMPLY ANYTHING
for the value difference of Rook vs Bishop. They are all different pieces,
and have nothing to do with each other.'

YES it does according to my model and every quality, holistic model built upon a proper foundation I have ever seen.  Contrary to your statement, I think it obvious to any logical person that the component pieces have at least SOMETHING to do with their composite pieces.

Some computer chess programmers are notorious for achieving useful relative
piece values that are within decent range of their optimums based purely upon AI playing strength without creating any coherent, fully-developed theory that is logically explained, justified and consistent.  Unfortunately, such people contribute little to the understanding of
relative piece values for themselves or other interested parties.

I have appr. two years of experience working with Reinhard Scharnagl's 
excellent SMIRF program, my fast dual-CPU server and choice Capablanca chess variants.

Reinhard Scharnagl would compiled two, otherwise-identical versions of his 
program using his and my favorite sets of relative piece values (at that time) which would played against one another using a great amount of time per move.  Eventually, we carefully completed many games this way.  We would both analyze the game results and discuss conclusions.  Sometimes we would agree.  Sometimes we would disagree.  Subsequent tests would settle disagreements ... sometimes.  In this manner, we both improved our models over time until we reached a point where any further minor improvements became prohibitively difficult to achieve within a survivable time frame.

'In real life the value of a piece is not the sum of the value of each of its individual moves ...'

YES it is although not exactly.  The moves of component pieces of a composite piece have far more effect upon determining its relative piece value than ALL other factors added together.

'... but also depends critically on properties like mating potential,
color-boundedness, forwardness, speed, manoeuvrability, concentration,
sensitivity to blocking.'

I have also read ALL of the pioneering works of Betza on the subject.
Essentially, my model mathematicized a subject (to the extent possible
present day) he had only speculatively verbalized.  Rest assured, my model 
makes quantitative adjustments for all non-trivial, effecacious factors to relative piece values that I know of with certainty.

You need to read and thoroughly understand my 58-page paper on the subject.

universal calculation of piece values
http://www.symmetryperfect.com/shots/calc.pdf

'Theoretical considerations like you refer to are just nonsense, with no
connection to real life.'

The connection of my model to 'real life' is very strong.  My theory was
adjusted and refined numerous times to comply with game results over
different piece sets and game boards.  Experience dictated the details
of the theory in accordance with the scientific method.

'What would you rather have (if you can choose to make a trade or
not), a piece that is more 'valuable' according to some contrived
reasoning, or a piece that gives you a larger probability to win the game?'

Both.

Under a proper model, they should not be mutually exclusive at all.
In fact, they should be in agreement ... until a point in the endgame
where checkmate becomes possible.  Be mindful that significant
differences in relative piece values between the opening game, 
mid-game and endgame (to the limited extent that they are applicable)
are accommodated under sophisticated models.
____________________________________________

See the published values of Ed Trice and Reinhard Scharnagl for
CRC pieces upon the 10 x 8 board.

http://www.gothicchess.com/piece_values.html

In addition to the published values of Hans Aberg and Derek Nalls, this verifies that it is beyond dispute that your published values for the archbishop and chancellor are radical.  Your radical contention that an archbishop and a chancellor have appr. equal relative piece values requires an especially sound theoretical framework to be convincing.  Instead, all I am receiving from you is piecemeal descriptions of endgame scenarios where material values are likely to disappear and become meaningless compared to positional values (i.e., checkmate achievable regardless of material sacrifices) and consequently, conclusions drawn are likely to be faulty.

I prefer to await Scharnagl's expert opinion on why SMIRF does not win every game under computer tournament conditions where time allowances per move are extremely small.  Of course, this precariously presumes that he wishes to comment after your wholesale insulting remarks toward SMIRF.  

This 'Battle Of The Goths Championship 2008' is a strange exercise with AI results that are virtually worthless theoretically due to the 'virtually instantaneous' execution of moves for each program involved.  I suspect that the entire purpose of this 'tournament' is contrived to seem to demonstrate the universal superiority of Joker80 where, in fact, the limited superiority at 'speed chess' only- a trivial achievement- is being demonstrated.
______________

'The ones who determine the facts through accurate measurement thus contribute in an absolutely essential way to our understanding, as without such facts the theoreticians cannot even start their work.'

Exactly!  That is how I constructed my theory.  How did you construct your theory?  Let's discuss 'accurate measurement' in a bit more detail since you claim to know so much about it.

Except for opening books, endgame tablebases and some 'very obvious, checkmate or no choice moves' that may arise within the midgame, it is generally true that there is a direct relation between the maximum search time a computer is allowed per move [Ply depth completion is actually the more important criterion but it is a function of time.] and the quality of the moves found or generated.

Since 

every move within a game is important and potentially, critically so
(although the importance of the first move of the game is greatest and the
importance of the last move of the game is least)

AND

every move during the game depends critically upon all previous moves by both players for its best chance of being a successful step toward the goal of victory,

it is critically important that every move generated via computer AI be of the highest quality possible for the results to have the highest chances of being theoretically instructive, relevant and valuable instead of mostly-purely random.

Otherwise, you have not adequately distilled each side to play as resourcefully as possible to definitively determine which side probably possesses the ultimate advantage or disadvantage via your gametests with different armies.  This is a vital prerequisite to enable you to derive relative piece values that are reliable at all.

This is true to the extreme for chess variants related the Chess such as 
Capablanca chess variants for which the game-winning objective is to capture a single royal piece (i.e., king) regardless of material sacrifice.  Consequently, the levels of depth and irony inherent to chess variants of this type of design are very high.

The effectiveness of traps is based upon the fact that, upon naive inspection, what looks like 'the best move available upon the board' can, in reality, be 'the worst move available upon the board'.  Obviously, it is critical to correctly distinguish between the two wherever they arise within a game.  It is not just humans that are susceptible to falling into traps.  Chess supercomputers have made similar mistakes. [See Kasparov vs. Deep Blue I.]  

For example, an 8-ply search completion may lead a computer to recommend 
a very bad move that it would never recommend if allowed a 10-ply search 
completion.  However, the deeper the search ply completion, the less likely for a 'dramatic irony' of this type to exist and remain dangerously undiscovered.

So ... what do you think you have accomplished by generating 20,000+ very badly played games (obviously) via ultra-fast, ultra-shallow depth moves?
This monumental exercise in 'mindless woodpushing' can only have a 
statistically random effect reflecting the tendencies of the individual chess programs involved to spit-out moves when forced to do so before being given adequate time to explore enough plies to play any better than the moron level.  This could have some minimal value if chess variants related to chess were games well-suited for morons to play competently.  However, they are well-suited only for genii to play competently ... albeit usually and only with extensive training, effort and experience.

In summary, you might as well blow the pieces across the gameboard with strong fans.  The results of this type of 'mindless woodpushing' would be only slightly less significant to your misguided effort to devise the 'most accurate relative piece values for CRC in existence' (by your claim) than this method you are presently using.
________________________________________________

'Why would I read a 58-page monologue from someone adhering to such flawed logic?  It can only be a waste of time.'

I wish to echo Scharnagl's remark that (paraphrased) 'my published model is still a work in progress'.  Nonetheless ... If you fail to read anything, then you fail to learn anything.  You should be able to learn something from my mis-steps as well as my correct steps.

The figures within everyone else's published models for the relative piece values of CRC pieces upon the 10 x 8 board implicitly agree with mine that the archbishop is significantly less valuable than the chancellor.  So, you are not just characterizing my published work as 'worthless nonsense'.  Logically, you must also be characterizing the published works of everyone else of note (namely, Aberg, Trice, Scharnagl) likewise for the single reason that we do not share your radical view that the archbishop and the chancellor have appr. equal value.

Furthermore, under your model, the archbishop is only a little less valuable than the queen which is another radical contention on your part that demands much defense.

'... the Battle-of-the-Goths tournament was played at 1 hour per game per side (55'+5'/move, the time on the clocks is displayed in the viewer). And
you call it speed Chess. Poof, there goes half your argument up in smoke.'

Sorry, I could not find the time per move on your crude web page.

Nonetheless, less than 1 minute per move is much too short to yield quality moves ... at least by anything better than low standards.
_________________________________________________________

'Not that it was any good to begin with: it is well known and amply tested
that the quality of computer play only is a very weak function of time
control.'

WRONG!

The quality of computer play correlates strongly as a function of ply depth completion which, in turn, is a function of time where exponentially greater time is generally required to complete each successive ply.
___________________________________________________________________

'The fact that you ask how 'my theory was constructed' is shocking.
Didn't you notice I did not present any theory at all?'

In fact, I have noticed that you have failed to present a theory to date.
I apologize for politely yet incorrectly giving you the benefit of the doubt that you had developed any theory at all unpublished but somewhere within your mind.  Do you actually prefer for me to state or imply that you are clueless even as you claim to be the world's foremost authority on the subject and claim the rest of us are stupid?  Fine then.
____________________________________________________________

'I just reported my OBSERVATION that quiet positions with C instead of A 
do not have a larger probability to win the game, and that in my opinion 
thus any concept of 'piece value' that does not ascribe nearly equal value to A and C is worse than useless.'

When you speak of what is needed to 'win the game' you are fixating upon the mating power of pieces which translates to endgame relative piece values- NOT opening game or midgame relative piece values.  Incidentally, relative piece values during the opening game are more important than during the midgame which, in turn, are more important than during the endgame.  Furthermore, I am particularly wary about the use of relative piece values at all during the endgame since any theoretically deep possibility to achieve checkmate (regardless of material sacrifices), discovered or undiscovered, renders relative piece values an absolutely non-applicable and false concept.

I strongly recommend that you shift your attention oppositely to the supremely-important opening game to derive more useful relative piece values.
_______

'So what have I think I proved by the battle-of-the-Goths long TC tourney
about the value of A and C? Nothing of course! Did I claim I did? No, 
that was just a figment of your imagination!'

I did not claim that I knew exactly how your ridiculous idea that an
archbishop is appr. equally valuable to a chancellor originated.  This 'tournament' of yours that I criticized just seems to be a part of your 'delusion maintenance' belief system.
__________________________________________

'It might be of interest to know that prof. Hyatt develops Crafty 
(one of the best open-source Chess engines) based on 40/1' games, 
as he has found that this is as accurate as using longer TC for relative 
performance measurement, and that Rybka (the best engine in the World)
is tuned through games of 40 moves per second.'

Now, you are completely confusing a method for QUICKLY and easily testing a computer hardware and software system to make sure it is operating properly with a method for achieving AI games consisting of highest quality moves of theoretical value to expert analysts of a given chess variant.

I have already explained some of this to you.  Gawd!
____________________________________________________

'The method you used (testing the effect of changing the piece values,
rather than the effect of changing the pieces) is highly inferior, and
needs about 100 times as many games to get the statistical noise down to
the same level as my method. (Because in most games, the mis-evaluated
pieces would still be traded against each other.)'

First, you are falsely inventing stats out of thin air!

If you really were competent with statistics, then you would know the
difference between their proper and improper application within your 
own work attempting to derive accurate relative piece values.

Second, you do not recognize (due to having no experience) the surprisingly great frequency with which a typical game between two otherwise-identical versions running a quality program with contrasting relative piece values will play into each other's most significant differences in the values of a piece.

Here is a hypothetical example ...

If white (incorrectly) values a rook significantly higher than an archbishop

AND

If black (correctly) values an archbishop significantly higher than a rook,

then the trade of white archbishop for a black rook will be readily
permitted by both programs and is very likely to actually occur at some point during a single game or a couple-few games at most.

Consequently, all things otherwise equal, white will probably lose most 
games which is indicative of a problem somewhere within its set of
relative piece values (compared to black).
__________________________________________

'If you are not prepared to face the facts, this discussion is pointless.'

When I reflect your remark back to you, I agree completely.
___________________________________________________________

'Play a few dozen games with Smirf, at any time control you feel
trustworthy, where one side lacks A and the other B+N, and see who is
crushed.'

relative piece values
opening game
(bishop pairs intact)

Muller

pawn  10.00
knight  35.29
bishop  45.88
rook  55.88
archbishop  102.94
chancellor  105.88
queen  111.76

Nalls

pawn  10.00
knight  30.77
bishop  37.56 
rook  59.43
archbishop  70.61
chancellor  94.18
queen  101.60

So, what is your problem?  Both of our models are in basic agreement on this issue.  There is no dispute between us.  [I hate to disappoint you.]

What you failed to take into account (since you refuse to educate yourself via my paper) is the 'supreme piece(s) enhancement' within my model.  My published start-of-the-game relative piece values are not the final word for a simplistic model.  My model is more sophisticated and adaptable with some adjustments required during the game.

For CRC, the 3 most powerful pieces in the game (i.e., archbishop, 
chancellor, queen) share, by a weighted formula, a 12.5% bonus which
contributes to 'practical attack values' (a component of material values 
under my model).  Moreover, the shares for each piece of the 12.5% bonus 
typically increase, by a weighted formula, during the game as some of the 
3 most powerful pieces are captured and their share(s) is inherited by the
remaining piece(s).  Thus, if the archbishop becomes the only remaining, 
most powerful piece, then it becomes much more valuable than the 
combined values of the bishop and knight.

Notwithstanding, I'll bet you still think my model is 'worthless nonsense'.
Right?

In the future, please do the minimal fact finding prerequisite to making 
sense in what you are arguing about?
____________________________________

'... the rest of the World beware that your theory of piece values 
sucks in the extreme!'

No, it does not.  Your self-described 'far less than a theory, only an 
observation' comes close, though.

'So the ply depth depends only logarithmically on search time,
which is VERY WEAKLY.

So if you had wanted to show any understanding of the matter at hand, you
should have written RIGHT! instead of WRONG! above it...'
______________________________________________________

'... it is well known and amply tested that the quality of computer play only is a very weak function of time control.'
____________________________________

I disagreed with your previous remark only because it was misleadingly,
poorly expressed.  You made it sound as if you barely realized at all that
the quality of computer play is a function of search time.  Obviously, you do.  So, here is the correction you demand and deserve ....

RIGHT!
_______

'Absolute nonsense. Most Capablanca Chess games are won by annihilation of
the opponents Piece army, after which the winning side can easily push as
many Pawns to promotion as he needs to perform a quick mate.
Closely-matched end-games are relatively rare, and mating power almost
plays no role at all. As long as the Pawns can promote to pieces with
mating power, like Queens.'

Very well.  I spoke incorrectly when I creditted you with foolishly assigning the archbishop nearly equal value to the chancellor due mainly to its decent mating power, relevant mainly in endgames ... sometimes.
You are even more foolish than that.  You actually think the archbishop 
has nearly equal value to the chancellor throughout the game- in the opening game and mid-game as well.  Wow!

By the way, please add IM Larry Kaufmann to your dubious list of 
'insufferably stupid people' who disagree with your relative piece values
in CRC:

http://en.wikipedia.org/wiki/Gothic_Chess
___________________________________________________

'... But let's cut the beating around the bush ...'

Good idea!

I have now completely run out of patience with your endless inept, 
amateurish attempts to discredit my work.  Not because you disagree.
Not even because you are unnecessarily rude and disrespectful.  Instead, 
strictly because you have NOT done your homework!  You refuse to 
read the same 58-page paper you are confidently grading with an 'F'.  

Consequently, virtually all of your criticisms to date about my model 
for calculating relative piece values have been incorrect, irrelevant
and/or irrational.  When/If you ever address concerns about my method
that I can identify as making sense and knowing at least what you are
talking about, then I will politely answer them.  Until then, my side of 
this conversation is closed.

A substantial revision and expansion has recently occurred.

universal calculation of piece values
http://www.symmetryperfect.com/shots/calc.pdf
66 pages

Only three games have relative piece values calculated using this complex
model:  FRC, CRC and Hex Chess SS (my own invention).  Furthermore, I only
confidently consider my figures somewhat reliable for two of these games, FRC (including Chess) and Capablanca Random Chess, because much work has been done by many talented individuals (hopefully, including myself) as well as computers to isolate reliable material values.  This dovetails into the reason that I do not take requests.  I have absolutely no assurance that the effort spent outside these two established testbeds is productive at all.  If it is important to you to know the material values for the pieces within your favorite chess variant (according to this model), then you must calculate them yourself.

Under the recent changes to this model, the material values for FRC pieces
and Hex Chess SS pieces remained exactly the same.  However, the material
values for a few CRC pieces changed significantly:

Capablanca Random Chess
material values for pieces
http://www.symmetryperfect.com/shots/values-capa.pdf

pawn  10.00
knight  30.77
bishop  37.56 
rook  59.43
archbishop  93.95
chancellor  95.84
queen  103.05

Focused, intensive playtesting on my part has proven Muller to be correct
in his radical, new contention that the accurate material value of the
archbishop is extraordinarily, counter-intuitively high.  I think I have
successfully discovered a theoretical basis which is now explained within
my 66-page paper.

All of the problems (that I am presently aware of) within my set of CRC
material values have now been solved.  Some problems remain within
Muller's set.  I leave it to him whether or not to maturely discuss them.

As far as playtesting goes ...

Admittedly, my initial intention was just to amuse myself by 
disproving the consistency of Muller's unusually-high archbishop 
material value in relation to other piece values within his CRC set.
If indeed his archbishop material value had been as fictitious as it 
was radical, then this would have been readily-achievable 
using any high-quality chess variant program such as SMIRF.
No matter what test I threw at it, this never happened.

Previously, I have only used 'symmetrical playtesting'.
By this I mean that the material and positions of the pieces
of both players have been identical relative to one another.
This is effective when playing one entire set of CRC piece values
against another entire set as, for example, Reinhard Scharnagl & I
have done on numerous occasions.  The player that consistently 
wins all deep-ply (long time per move) games, alternatively playing 
white and black, can be safely concluded to be the player using 
the better of the two sets of CRC piece values since this single 
variable has been effectively isolated.  However, this playtesting
method cannot isolate which individual pieces within the set 
carry the most or least accurate material values.

In fact, I had no problem with Muller's set of CRC piece values
as a whole.  The order of the material values of all of the CRC 
pieces was-is correct.  However, I had a large problem with his
material value for the archbishop being nearly as high as for
the chancellor.  

To pinpoint an unreasonably-high material value for only one 
piece within a CRC set required 'asymmetrical playtesting'.  
By this I mean that the material and positions of the pieces 
of both players had to be different in an appropriate manner to
test the upper and lower limits of the material value for a certain 
piece (e.g., archbishop).  This was achieved by removing select
pieces from both players within the Embassy Chess setup so that 
BOTH players had a significant material advantage consistent
with different models (i.e., Scharnagl set vs. Muller set).  
This was possible strictly because of the sharp contrast between the 
'normal, average' and 'very high', respectively, material values 
for the archbishop assigned by Scharnagl and Muller.  The fact
that the SMIRF program implicitly uses the Scharnagl set to play
both players is a control variable- not a problem- since it is 
insures equality in the playing strength with which both players
are handled.  The player using the Scharnagl set lost every game 
using SMIRF MS-173h-X ... regardless of time controls, 
white or black player choice and all variations in excluded pieces 
that I could devise.

I thought it was remotely possible that an intransigent, positional 
advantage for the Muller set somehow happened to exist within the 
modified Embassy Chess setup that was larger than its material 
disadvantage.  This type of catastrophe can be the curse of 
'asymmetrical playtesting'.  So, I experimented likewise using a 
few other CRC variants.  Same result!  The Scharnagl set lost every 
game.

I seriously doubt that all CRC variants (or at least, the games I tested)
are realistically likely to carry an intransigent, positional advantage 
for the Muller set.  If this is true, then the Muller set is provably, 
ideally suited to CRC, notwithstanding- just for a different reason.

Finally, I reconsidered my position and revised my model.

For the reasons you describe (which I mostly agree with), I do not ever use
'asymmetrical playtesting' unless that method is unavoidable.  However,
you should know that I used many permutations of positions within my
'missing pieces' test games to try to average-out positions that may
have pre-set a significant positional advantage for either player.  

Yes, the fact that SMIRF currently uses your (Scharnagl) material values
with a 'normal, average' material value for the archbishop instead of a
'very high' material value (as well as the interrelated positional value
given to the archbishop with SMIRF) means that both players will place
greater effort than I think is appropriate into avoiding being forced into
disadvantageous exchanges where they would trade their chancellor or queen
for the archbishop of the opponent.  Still, the order of your material
values for CRC pieces agrees with the Muller model (although an
archbishop-chancellor exchange is considered only slightly harmful to the
chancellor player under his model).  So, I think tests using SMIRF are
meaningful even if I disagree substantially with the material value for
one piece within your model (i.e., the archbishop).

Due to apprehension over boring my audience with irrelevant details, I did
not even mention within my previous post that I also invented a variety of
10 x 8 test games using the 10 x 8 editor available in SMIRF that were
unrelated to CRC.  

For example, one game consisted of 1 king & 10 pawns per player with 9
archbishops for one player and 8 chancellors or queens for another player.
 Under the Muller model, the player with the 9 archbishops had a
significant material advantage.  Under the Scharnagl model, the player
with the 8 chancellors or 8 queens had a significant material advantage. 
The player with the 9 archbishops won every game.

For example, one game consisted of 1 king & 20 pawns per player with 9
archbishops for one player and 8 chancellors or queens for another player.
 Under the Muller model, the player with the 9 archbishops had a
significant material advantage.  Under the Scharnagl model, the player
with the 8 chancellors or 8 queens had a significant material advantage. 
The player with the 9 archbishops won every game.

For example, one game consisted of 1 king & 10 pawns per player with 18
archbishops for one player and 16 chancellors or queens for another
player.  Under the Muller model, the player with the 18 archbishops had a
significant material advantage.  Under the Scharnagl model, the player
with the 16 chancellors or 16 queens had a significant material advantage.
 The player with the 18 archbishops won every game.

I have seen it demonstrated many times how resilient positionally the
archbishop is against the chancellor and/or the queen in virtually any
game you can create using SMIRF with a 10 x 8 board and a CRC piece set.

When Muller assures us that he is responsibly using statistical methods
similar to those employeed by Larry Kaufmann, a widely-respected
researcher of Chess piece values, I think we should take his word for it. 
Of course, I remain concerned about the reliability of his stats generated
via using fast time controls.  However, it has now been proven to me that
his method is at least sensitive enough to detect 'elephants' (i.e.,
large discrepancies in material values) such as exist between contrasting
CRC models for the archbishop even if it is not sensitive enough to detect
'mice' (i.e., small discrepancies in material values) so to speak.

Yes, your test example yields a result totally inconsistent with
everyone's models for CRC piece values.  [I did not run any playtest
games of it since I trust you completely.]  Yes, your test example could
cause someone who placed too much trust in it to draw the wrong conclusion
about the material values of knights vs. archbishops.  The reason your test
example is unreliable (and we both agree it must be) is due to its 2:1 ratio of knights to archbishops.  The game is a victory for the knights player simply because he/she can overrun the archbishops player and force materially-disadvantageous exchanges despite the fact that 4 archbishops indisputably have a material value significantly greater than 8 knights.

In all three of my test examples from my previous post, the ratios of
archbishops to chancellors and archbishops to queens were only 9:8.  Note
the sharp contrast.  Although I agree that a 1:1 ratio is the ideal goal, it was impossible to achieve for the purposes of the tests.  I do not believe a slight disparity (1 piece) in the total number of test pieces per player is enough to make the test results highly unreliable.  [Yes, feel free to invalidate my test example with 18 archbishops vs. 16 chancellors and 18 archbishops vs. 16 queens since a 2 piece advantage existed.]  Although surely imperfect and slightly unreliable, I think the test results achieved thru 'asymmetrical playtesting' or 'games with different armies' can be instructive as long as the test conditions are not pushed to the extreme.  Your test example was extreme.  Two out of three of my test examples were not extreme.

Feel free to invalidate my other two test examples I (reluctantly)
mentioned as well.  

My reason is that having ranks nearly full of archbishops, chancellors or
queens in test games does not even resemble a proper CRC variant setup
with its variety and placement of pieces.  Therefore, those test results
cannot safely be concluded to have any bearing upon the material values of
pieces in any CRC variant.   

Your reason is well-expressed.

The feasibility of using identical armies to calculate piece values

It has been a long time since our sets of CRC piece values have played one
another (on my dual 2.4 Ghz CPU server) using otherwise-identical versions
of SMIRF.  Obviously, the reason is that it has been a long time since
there existed a large disparity within our material values for any one of
the CRC pieces.  Recently, that has changed in the case of the
archbishop.

I already have the standard version of SMIRF MS-174b-O which uses
Scharnagl CRC piece values.  Would you be willing to compile a special
version of SMIRF MS-174b-O for me which uses Nalls CRC piece values?

Capablanca Random Chess
material values of pieces
http://www.symmetryperfect.com/shots/values-capa.pdf

Back on safe ground using 'symmetrical playtesting', the results of who
wins the test games should be indicative of who is using a better set of
CRC piece values.

I understand.  I wondered what the 'X' & 'O' designations for recent
SMIRF versions meant.  Do you still possess an older version of SMIRF (of
satisfactory quality to you) that uses your current CRC material values?

Since there is appr. 2-1/2 pawns difference between our models in our 
material values for the archbishop, I predict that my playtesting results
would probably be worthwhile and decisive.