Not a huge issue because they get refreshed whenever a rule is applied but it's annoying. I couldn't figure out how to enforce this in Fabric.js.