Build out a proper backward compatibility setup

[IanWoodard]

Currently, we do not have a good process for ensuring backward compatibility across releases of devservices. While we don't have to support every previous version of devservices, we should support some amount of previous versions at any given point. This would likely require better handling for configs such that adding new fields doesn't cause breakages with old versions. Additionally, we will likely want some testing to ensure backward compatibility.