Performance

[LMBDA-design]

Hello, i found that the auc in your codes in beyond 97% when using Res50 features, so how do you think it's 93%?Anything wrong with mine ?

[Dootmaan]

Hi @LMBDA-design and sorry for the late reply. Your result is really amazing since the SOTA methods today is also nearly 97% AUC. Can you please tell us your exact training settings and what ResNet50 pertained weights did you use? If possible, can you please upload the weights of your 97% AUC DTFD-MIL?

[LMBDA-design]

I just followed your default configuration..... exactly the same...
maybe it's because that there are some several images encountered some problem when preprocessing , so there isn't enough images to be tested. 126 images are successfully generated to be tested.
Still confused about the way you get the performance metric.....Could you please introduce it ? Or just take the max auc in 200epochs ? Honestly ,it may reach 1.0 in your original code......

[Dootmaan]

We use sklearn.metrics.roc_auc_score/roc_curve/f1_score for the metrics, which is basically the same of the official DTFD-MIL implementation. When running the official DTFD-MIL code we also found that AUC on VALIDATION set may reach 1.0, but when it comes to the official CAMELYON16 test set we only achieved 93% AUC in the end. Same 93% AUC results are also reported in other recent papers, such as https://arxiv.org/pdf/2301.08125 and https://arxiv.org/pdf/2303.08446(CVPR2023).

If you really find AUC reaching 1.0 on the TEST set with our default split, would you please upload your saved weights of DTFD-MIL so we can look into it?

[Dootmaan]

Hi @LMBDA-design. Seems like you have missed our reply yesterday. We really would like to conduct some test on the 97% AUC model, so could you please upload your trained weights? Thank you for your help!😃

[LMBDA-design]

OK sure i would be glad to upload the weights soon. But where to put the weights?

[Dootmaan]

Hi @LMBDA-design and thank you for your help! You can upload the weights to BaiduYunpan, Aliyun Drive or Google Drive, etc. Or you can simply email us the weights (email address whongyi@zju.edu.cn)!

[Dootmaan]

Hi @LMBDA-design! Sorry to bother you again. Seems like you have missed our last reply, so we wonder if it is possible for you to directly send the trained weights (97% AUC one) to us by email (whongyi@zju.edu.cn)? That would greatly help our research. Thank you!

[Dootmaan]

Hi @LMBDA-design! Sorry to bother. If you have any problem emailing/uploading your weights, please do not hesitate to tell us. We really would like to help and look into this performance issue.