When an app upgrades from one version of sbt to another, we should remove the old version as to prevent the cache from growing too large.