lichess.org
Donate

Evaluating Sharpness using LC0's WDL

#11
Interesting how clearly you show the complementarity of novel measure using a different species of engine output.

Why is the LC0 measure, metric, or scale having the word contempt in it? Maybe I forgot what contempt was, is that human chess theory, or was it engine theory first? The human natural language of it kinds of might be throwing me off a bit.
#13
Very interesting!

I think another component of sharpness is the degree to which only-lines or only-moves are needed to maintain the evaluation of a position. For example, one player might have a +4.0 evaluation but there's only one line in the position that maintains it, and every other move leads to equality or worse. Typically, the more concrete the position, the sharper the position is. I wonder if there's a way to incorporate this metric into the overall 'sharpness formula'?
#14
@tackyshrimp said in #13:
> Very interesting!
>
> I think another component of sharpness is the degree to which only-lines or only-moves are needed to maintain the evaluation of a position. For example, one player might have a +4.0 evaluation but there's only one line in the position that maintains it, and every other move leads to equality or worse. Typically, the more concrete the position, the sharper the position is. I wonder if there's a way to incorporate this metric into the overall 'sharpness formula'?

I have thought about this quite a bit, but my main problem is that it's difficult to deal with only moves that are easy to find. A recapture is a perfect example: if you don't retake the piece you are just lost, but every somewhat experienced player sees the move and the position is therefore not more difficult than other positions with more possibilities.
#16
@dboing said in #11:
> Interesting how clearly you show the complementarity of novel measure using a different species of engine output.
>
> Why is the LC0 measure, metric, or scale having the word contempt in it? Maybe I forgot what contempt was, is that human chess theory, or was it engine theory first? The human natural language of it kinds of might be throwing me off a bit.

Many chess engines use a so-called contempt factor (www.chessprogramming.org/Contempt_Factor). It's basically a way to overestimate the own position in order to prevent early draws by the engine.
I would guess that the feature for LC0 is called contempt because a higher factor means a bigger rating difference, so LC0 will play like it's facing a weaker opponent.
#17
@jk_182 said in #14:
> I have thought about this quite a bit, but my main problem is that it's difficult to deal with only moves that are easy to find. A recapture is a perfect example: if you don't retake the piece you are just lost, but every somewhat experienced player sees the move and the position is therefore not more difficult than other positions with more possibilities.

Perhaps you could develop a weighting scheme that decreases the weight for easy only-moves? Like you said, recaptures are easy to see. Maybe also certain moves like forward (rather than backward) minor piece movements (since they often say backward knight moves are the hardest to find), or maybe checks as well. I can see this being imperfect but potentially more useful than not.
#18
The problem then becomes to determine which moves are easy. There are many rules of thumb but I think that there are just too many exceptions.
I'm currently working on a way to determine how surprising moves are using neural network engines. This may be a good (but certainly not perfect) option to determine how difficult moves are to find.
#20
Why do you think that this method is more valuable than looking at a grandmaster database and determining which openings are 'sharp' based on drawing percentage? isn't it the case (at least practically) that games with high draw percentage are less sharp?