This repository is the official implementation of SANSformers: Self-Supervised Forecasting in ELECTRONIC HEALTH RECORDS with Attention-Free Models .
Presentation regarding the experiments here: https://drive.google.com/file/d/1iLqgO9OZVRsf4BbA-P2kD6WM0Ew0nuVO/view?usp=sharing
- You need to have access to the MIMC-IV v1.0 dataset
- Download the ZIP file and mount it to the Jupyter Notebook file attached.
- Clone this repository and make sure all the files in here are downloaded, including the YAML and Pickle files, then mount them onto the drive.
https://bitbucket.org/dlh169/dlh169pretrainedmodel/src
Download these pretrained models if you would like to replicate what was created in my experiment
These two models achieves the following performance:
| Model name | loss | bin_loss | auc_bin |
|---|---|---|---|
| Additive Sansformer | 66% | 66% | 66% |
| Axial Sansformer | 66% | 66% | 66% |
📋 Include a table of results from your paper, and link back to the leaderboard for clarity and context. If your main result is a figure, include that figure and link to the command or notebook to reproduce it.
📋 Refer to LICENSE.txt on our GitHub repository for licensing information. We recommend cloning the provided Google Colab notebooks for your own use if you plan on making any contributions.
Y. Kumar, A. Ilin, H. Salo, S. Kulathinal, M. K. Leinonen and P. Marttinen, "Self-Supervised Forecasting in Electronic Health Records with Attention-Free Models," in IEEE Transactions on Artificial Intelligence, doi: 10.1109/TAI.2024.3353164.