Studying fine tuning and multitask learning in an angle discrimination task

experiment ideas:

adapation to different distribution of angles
hysteresis in adaptation to wide vs narrow distributions
continual learning on different distributions
Compare the results of different trainers, with and without saving the trainer state.

Saving the trainer state prevents initialization pollution, while saving only the network weights does not!

I'm at a point where there are several things that I need to accomplish to push to the next phase of the project

termination conditions for the iteration: based on the variance of the distirbutions, or maybe the KL divergence or something like that
Add mean reversion to the iteration to prevent the growth of noise between iterates. Ideally, we want this to act on a small scale only.

Name		Name	Last commit message	Last commit date
Latest commit History 59 Commits
datageneration		datageneration
experiments		experiments
notebooks_working		notebooks_working
writeups		writeups
.gitignore		.gitignore
Results0.1-InitialResults.ipynb		Results0.1-InitialResults.ipynb
Results0.2-finetuning.ipynb		Results0.2-finetuning.ipynb
Results1.0-FisherInformation.ipynb		Results1.0-FisherInformation.ipynb
Results1.1-FisherInfo2.ipynb		Results1.1-FisherInfo2.ipynb
Results1.2-FisherInfor3.ipynb		Results1.2-FisherInfor3.ipynb
Results1.3-TrainingResults.ipynb		Results1.3-TrainingResults.ipynb
Results1.4-nextPdf.ipynb		Results1.4-nextPdf.ipynb
Results2.0-objective-effects.ipynb		Results2.0-objective-effects.ipynb
Results2.1-retraining.ipynb		Results2.1-retraining.ipynb
Results2.2-backprop_dynamics.ipynb		Results2.2-backprop_dynamics.ipynb
Results2.3-simple_tests_more_exploration.ipynb		Results2.3-simple_tests_more_exploration.ipynb
Results2.4-experiment_analysis.ipynb		Results2.4-experiment_analysis.ipynb
Results2.5-interation_approaches.ipynb		Results2.5-interation_approaches.ipynb
Results3.0-return.ipynb		Results3.0-return.ipynb
adapt_fit_loop.py		adapt_fit_loop.py
adaptableModel.py		adaptableModel.py
basicModel.py		basicModel.py
discriminationAnalysis.py		discriminationAnalysis.py
example_codes.py		example_codes.py
nonlinFisher.py		nonlinFisher.py
old_solutions_analytic.py		old_solutions_analytic.py
readme.md		readme.md
rotatedFaces.py		rotatedFaces.py
scratch2.2-FIspeedup.ipynb		scratch2.2-FIspeedup.ipynb
trainers.py		trainers.py