In terms of evaluating the status of a specific model during tuning, we should have general objectives. Intel® Neural Compressor Objective supports code-free configuration through a yaml file. With built-in objectives, users can compress models with different objectives easily. In special cases, users can also register their own objective classes.
The objective supported by Intel® Neural Compressor is driven by accuracy. If users want to evaluate a specific model with other objectives, they can realize it with objective in a yaml file. Default value for objective is performance, and the other values are modelsize and footprint.
In some cases, users want to use more than one objective to evaluate the status of a specific model and they can realize it with multi_objectives in a yaml file. Currently multi_objectives supports built-in objectives.
If users use multi_objectives to evaluate the status of the model during tuning, Neural Compressor will return a model with the best score of multi_objectives and meeting accuracy_criterion after tuning ending.
Intel® Neural Compressor will normalize the results of each objective to calculate the final weighted multi_objective score.
Built-in objectives support list:
| Objective | Usage |
|---|---|
| accuracy | Evaluate the accuracy |
| performance | Evaluate the inference time |
| footprint | Evaluate the peak size of memory blocks during inference |
| modelsize | Evaluate the model size |
Users can specify a built-in objective in neural_compressor.config.TuningCriterion as shown below:
from neural_compressor.config import TuningCriterion
tuning_criterion = TuningCriterion(objective='accuracy')Users can specify built-in multiple objectives in neural_compressor.config.TuningCriterion as shown below:
from neural_compressor.config import TuningCriterion
tuning_criterion = TuningCriterion(objective=['performance', 'accuracy'])Refer to example as an example.