The direct training on egenvalues

[ChenQianA]

Hi, Dr Zhong Yang. The training of HamGNN consists of two steps: 1). training on matrix elements, 2). then training the model afterwards simultaneously on the matrix elements and eigenvalues. Since dealing with matrix elements involves lots of disk storage, it would be more friendly if the model could be directly trained on eigenvalues without matrix elements for HamGNN. However, the numerical stability of CGTP is very tricky, which might lead to the infeasibility of directly training solely on eigenvalues using HamGNN. Have you done tests on that, and is that feasible? Thank you so much, and I look forward to your response.

[QuantumLab-ZY]

Hi @ChenQianA,

Thank you very much for your thoughtful question and for your interest in HamGNN. You raised an excellent point regarding the feasibility of training the model directly on eigenvalues without explicitly involving the Hamiltonian matrix elements.

Directly incorporating band energy errors into the loss function can lead to optimization instabilities, because the gradients derived from predicted Hamiltonians with insufficient accuracy may diverge when back-propagated through the eigenvalues. To address the problem, we therefore adopted a two-stage training protocol. In the first stage, the model focuses on accurately reproducing the Hamiltonian matrix elements, ensuring stable and physically meaningful intermediate representations. In the second stage, we fine-tune the model jointly on both matrix elements and eigenvalues.

Practically, since the second stage initializes from the pre-trained Hamiltonian model of the first stage, it can start with a much lower learning rate, and only a few epochs are typically needed for convergence to satisfactory accuracy in band structures and energy predictions. The second round of training does not need to wait for the model to fully converge before it can be used, unless it is for benchmark comparison.

Best regards,
Zhong Yang