A question on the value of losses

[SorenGioro]

Hello JiaoR,
Thanks for your great work!
I’m currently using some slabs (nearly 3000) in the OpenCatalyst dataset training a model, and both val_lattice_loss and val_coord_loss are nearly 0.6 with default hyperparameters
I wonder whether this value is reliable, could you please give me some advice?

[jiaor17]

Hi,

Thanks for your interest!

Your reported validation losses (val_lattice_loss and val_coord_loss around 0.6) are indeed in line with what we have observed on MPTS-52. This suggests that your model is training as expected and the loss values are reliable.

However, it's important to note that validation loss values alone might not fully capture the model's performance in practical scenarios. To get a more comprehensive understanding of how well your model is performing, I recommend conducting a generation test. You can do this by running the generation process on a subset of your validation set and then calculating the match rate. This will provide a clearer picture of how your model performs on this dataset.

[SorenGioro]

Thank you for your reply! I will try it out later