|
| 1 | +## Dealing with massive image files |
| 2 | + |
| 3 | +### If a single image can fit into GPU memory |
| 4 | +- Use distributed processing to load 1 image on each GPU, use multiple GPUs (at least, TensorFlow supports this). [link](https://www.tensorflow.org/guide/distributed_training) |
| 5 | +- Fit an autoencoder and train using the internal representation. |
| 6 | + - Potentially interesting if a single image modality fits, but not all 4 at once |
| 7 | + - I tried this before and it didn't take that long even with batch size=1 |
| 8 | +- Use early strided convolution layers to reduce dimensionality. Used in U-net. [link](https://arxiv.org/abs/1505.04597) |
| 9 | +- Image fusion |
| 10 | + - principal component analysis (this also works for image compression if you do it differently) |
| 11 | + - frequency-domain image fusion such as various shearlet transforms (I don't understand these, but here's a paper [link](https://journals.sagepub.com/doi/full/10.1177/1748301817741001)) |
| 12 | + - I guess you could probably also use an autoencoder for this |
| 13 | + - This should reduce our 4-channel (4 neuroimaging types) image to have less channels containing the same information |
| 14 | + |
| 15 | +### Works even if a single image can't fit into GPU memory |
| 16 | +- Cropping |
| 17 | + - This probably works better if the images are registered to approximately the same space |
| 18 | +- Slicing [Cameron's review with some of these](https://www.sciencedirect.com/science/article/pii/S187705091632587X) |
| 19 | + - Use 2-dimensional slices of 3D image, which each definitely fit in memory |
| 20 | + - (probably) can train models for each modality separately and average/use a less-GPU intensive model to combine them? |
| 21 | + - (probably) split image into smaller 3D patches for segmentation |
| 22 | +- Downsampling: [this paper](https://nvlpubs.nist.gov/nistpubs/ir/2013/NIST.IR.7839.pdf) is not about neuroimaging at all but maybe has some insights? |
| 23 | + - Spectral truncation |
| 24 | + - Compute fast Fourier transform, reduce sampling rate, compute inverse FFT |
| 25 | + - I'm going to add wavelet transform here for similar reasons |
| 26 | + - Average pooling (take the average of 2x2x2 voxels) |
| 27 | + - Max pooling (take the maximum of 2x2x2 voxels) |
| 28 | + - Decimation/Gaussian blur with decimation (take every other line) |
| 29 | +- Use a convolutional neural network that works on spectrally compressed images [link](https://www.sciencedirect.com/science/article/abs/pii/S0925231219310148) |
| 30 | + - probably really stupid |
| 31 | + - compute FFT, discrete cosine transform, or whatever |
| 32 | + - clip the spectrum to get rid of irrelevant high frequency noise |
| 33 | + - use a spectral convolutional neural network to compute everything in frequency domain |
| 34 | + - transform back to image domain |
| 35 | + |
0 commit comments