IceNet for Anomalous Events Forecasting Discussion from Qien's MPhil Project

[qiencai]

Hi everyone, towards the end of my MPhil project in predicting break-up events at emperor penguin colonies, I would like to thank Ellie and Bryn in our lab, and Peter Fretwell for guiding me throughout the project, as well as the people who provided feedback through my practice presentation. I would like to share some results that I have found in terms of the performance of IceNet and also raise some questions, some by myself and some given by professors at DAMTP Cambridge, when I gave my project presentation this Tuesday. Hopefully, people might use them as some inspiration for future model improvements in Antarctica.
I used the 25 km daily forecast from IceNet to study the break-up events in the Bellingshausen Sea region. While the local SIC does not show a clear signal of break-up events (potentially due to resolution and coastal data limitations), I discovered that break-up happens when the distance from colonies to the nearest ice edge drops below 200 km. I therefore used this as a proxy for break-up events and tried to forecast the date when it drops below 200 km using IceNet.
Taking Smyley Island as an example, below is the result of attempting to predict the date when the distance drops below 200 km, over 2021–2024.
Despite some results that are intuitive, better predictions as we approach the events (not entirely true, and I will discuss it later), it was observed that earlier events are not captured very well by IceNet, especially those in October. This raised the first question from my professor: how well can AI capture anomalies? I think for IceNet, with its built in long term trend as an input, and an important one (at least from the importance analysis that Tom has done in his original paper) it is prone to overfitting to the past ice regime.
Another observation, which potentially could have caused some predictions to diverge further from the truth as lead time decreases, is that predictions affected by the long term trend would return to a high concentration trajectory then fall, especially if the predictions started from very low SIC at earlier times in the year. This was observed for single grid cell SIC in the 2022 Bellingshausen Sea regions (blue plot on the left). Sometimes, it is observed that earlier predictions would show almost the same decreasing trend as later forecasts, but since they were started earlier, they predict lower SIC later on (which often can generate better predictions by luck, therefore earlier predictions show better results sometimes).
A few initial thoughts that I have include: 1. adding sea ice thickness as input; 2. training a regional model for regional risk assessment work; 3. higher resolution or more advanced AI models, such as diffusion, which Andrew is working on; 4. removing the statistical trend as input.
I wonder if people would have more valuable input or feedback, happy to discuss more!

[aranas]

@qiencai this is super cool work! I was wondering:

1. Are all the years displayed in the top plot considered anomalous since break-up happens before the end of the fledging period or are only some of those anomalous?
   I ask because to answer the question of "how well can AI capture anomalies?", I'd first want to see how well the forecasts capture the norm. Have you looked at other more average years than the ones plotted?

2. On the trend bias:

   predictions affected by the long term trend would return to a high concentration trajectory then fall,

   What do you think might be causing this 'jump'?