Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compare weights - to see what is (un)learned #6

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

MelleKoning
Copy link

@MelleKoning MelleKoning commented Jan 9, 2020

This adds some code to compare two weightfiles in detail.

When running a testsuite over two nets as defined in nets.txt, the standard output will contain the positions where the result was different for.

Example output for two nets:

 1000  problems...  C:\lc0\lc0.exe
  nodes:0 nodes
  weight:61826
 {
     "logFile": "C:\\lc0\\logs.txt",
     "Threads": 2,
     "minibatchsize": 32,
     "SmartPruningFactor": 1,
     "VerboseMoveStats": false,
     "HistoryFill": "always",
     "WeightsFile": "C:\\lc0\\nets\\61826"
}
*** if you need to pause all instances runing just create a file named 'pause-Position-tester.txt'

Run 1 of 2: 61826, 0 nodes
107/1000,  Agree:10.70%, End of set: 2020-01-09 18:20, avg npm: 1
Run 2 of 2: 61890, 0 nodes
109/1000,  Agree:10.90%, End of set: 2020-01-09 18:21, avg npm: 1

r1bqk2r/2p1bppp/p1np1n2/1p2p3/3PP3/1B3N2/PPP2PPP/RNBQ1RK1 w kq - bm c3 ; 317
 w:61826 True w: 61890 False
3rkbr1/1b1n1p2/p1p1p1q1/4P2p/PppP1BpP/2N3P1/1P2BP2/R1Q1R1K1 w - - bm Bxc4 ; 400
 w:61826 False w: 61890 True
1qrb1rk1/3b1p1p/p2p2p1/3Pp1P1/2N2n2/P3BP2/KP1Q3P/2R2BR1 w - - bm Bxf4 ; 3303
 w:61826 True w: 61890 False
1r3rk1/2q2pp1/1nn1p1p1/p2p2P1/2pPPP1P/P1P5/3NBR2/R4QK1 w - - bm h5 ; 7681
 w:61826 False w: 61890 True
1rb2rk1/1p2bp2/pN2pn1p/4N1q1/7p/3B4/P1P3PP/1R2QR1K w - - bm Nf3 Qc3 ; 10955
 w:61826 False w: 61890 True
2b4k/6pp/4pb2/p1q1p3/1pBrP2P/1P3P2/PNP4Q/1K4R1 w - - bm h5 ; 13795
 w:61826 True w: 61890 False
r1bq1r1k/p2nbpp1/2p1p2p/3nP3/PppPN1Q1/6P1/1PN2PBP/R1B2RK1 w - - bm Qh5 ; 13946
 w:61826 True w: 61890 False
4r1k1/1p3p2/p2b2p1/5nP1/3pBP2/2PP1K2/1P1B4/7R w - - bm c4 ; 18668
 w:61826 True w: 61890 False
2r1r1k1/1bqn1pbp/p1p2np1/1p4N1/1P1P4/P1NB3P/2QB1PP1/2RR2K1 w - - bm Qa2 ; 18732
 w:61826 False w: 61890 True
2rq1rk1/p2nbppb/Ppp3np/3pP3/3P4/1N2BN2/1P2BPPP/R2Q1RK1 w - - bm Ne1 ; 23524
 w:61826 False w: 61890 True
5b2/1N3Qpk/p7/1pp1p3/P7/1P2q2P/1B4P1/6K1 w - - bm Qf2 ; 32131
 w:61826 True w: 61890 False
3rr1k1/1pp2pp1/2npqb1p/pB6/P3b1P1/2P1B2P/1PPQ1P2/1K1RN1R1 w - - bm g5 ; 34782
 w:61826 True w: 61890 False
r1bq1rk1/4bppp/pnn1p3/3pP3/1ppP1PP1/4BN2/PPPQBN1P/R4RK1 w - - bm g5 ; 37750
 w:61826 True w: 61890 False
5rk1/p5p1/P4r1p/1Np1R1n1/3P2qN/1P1Q1bP1/5P1P/2R1K3 w - - bm Rcxc5 ; 43780
 w:61826 False w: 61890 True
r3kb1r/1p1b2pp/2nq1p2/8/p1BN4/3Q3P/PB3PP1/R4K1R w kq - bm Re1+ ; 49844
 w:61826 True w: 61890 False
q7/2nnb1kp/6p1/3pp1P1/1p2P2P/1P2B1N1/1PK2Q2/5B2 w - - bm Kb1 ; 52702
 w:61826 False w: 61890 True
r1b1r1k1/1pq2pp1/p2pp2p/2n5/3NN1Pb/1B2Q3/PPP2P1P/2KR2R1 w - - bm f4 ; 53457
 w:61826 False w: 61890 True
1kr5/8/3Pp3/2np4/p2N4/1p6/1PP3PP/1K1R4 b - - bm e5 ; 62903
 w:61826 True w: 61890 False
2r1k1r1/1q2bpP1/p2p4/1p6/4N2P/PP2B3/2P1N1bK/R3QR2 b - - bm Bxe4 ; 76208
 w:61826 False w: 61890 True
r6r/1pq2pk1/p2p1np1/3Pn2p/5Q2/1N4PP/PPP5/1K1R1B1R b - - bm Rac8 ; 77092
 w:61826 False w: 61890 True
3r1rk1/pb2q1p1/2p4p/1p4p1/1P2P3/2N2BP1/P1QB1RKP/8 b - - bm Qxb4 ; 87576
 w:61826 False w: 61890 True
4k3/8/1p5R/p4N2/n4PP1/5K2/8/6r1 b - - bm Rf1+ ; 89788
 w:61826 False w: 61890 True
r1bqk2r/2p1bppp/p1np1n2/1p2p3/3PP3/1B3N2/PPP2PPP/RNBQ1RK1 w kq - bm c3 ; 113059
 w:61826 True w: 61890 False
3rkbr1/1b1n1p2/p1p1p1q1/4P2p/PppP1BpP/2N3P1/1P2BP2/R1Q1R1K1 w - - bm Bxc4 ; 113092
 w:61826 False w: 61890 True
 positions learned:13 positions unlearned:11

I just tested on 200 nodes instead of 0 nodes as well, although the number of found moves is higher (of course) the difference between nets has also increased:

Run 1 of 2: 61826, 200 nodes
141/1000,  Agree:14.10%, End of set: 2020-01-09 18:57, avg npm: 172
Run 2 of 2: 61890, 200 nodes
142/1000,  Agree:14.20%, End of set: 2020-01-09 19:22, avg npm: 172
 positions learned:26 positions unlearned:25

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant