Fix windows installer leaving files and error on plover_plugins #1183

Job	Run time
Analyze	13s
Test (Python 3.10)	36s
Test (Windows)	1m 34s
Test (Python 3.11)	43s
Test (Python 3.12)	38s
Test (Python Latest Supported)	45s
Test (Linux)	43s
Test (macOS)	32s
Test (Code Quality)	43s
Test (Qt GUI)	55s
Test (Packaging)	1m 19s
Build (macOS App)	1m 46s
Build (Linux)	4m 17s
Build (Windows)	3m 11s
Build (macOS DMG)	1m 15s
Notarize (macOS App)	0s
Notarize (macOS DMG)	0s
Release	0s
	19m 10s

Provide feedback