I've been curious about how good LeelaPieceOdds is, so I downloaded a bunch of data and graphed it.

For context, Leela is a chess bot and this version of it has been trained to play with a handicap.

This is BBNN odds, meaning Leela starts without bishops and knights.

I first heard about LeelaQueenOdds from simplegeometry's comment, and I've been playing against it occasionally since then. LeelaPieceOdds is the version with more varied handicap options, which is more interesting to me because LeelaQueenOdds ~always beats me.

Playing Leela feels like playing chess while tired. I just keep blundering. It's not invincible, and when you beat it it doesn't feel like it was that difficult. But more often than not it manages to overcome the huge disadvantage it started with.

The data

Bishop-Bishop-Knight-Knight

How to read the graphs

Orange means the human player won. The title contains the handicap[1] and the time control. I split the data by time control (Bullet, Blitz, Rapid), which indicates how much time the human is allowed to spend thinking over the whole game. Leela always plays quickly so the time control is essentially only affecting the human.