Registry
The registry is a collection of input datasets, feature extractors, sample similarity measures, noise sources, and interpolation sources. See Extensibility and API to learn how to add new items to the registry and how to use them for metrics calculation. A number of entities have been pre-registered, and can be resolved in both CLI and API modes by their names:
Preregistered inputs
Can be used as values to input1
and
input2
arguments:
cifar10-train - CIFAR-10 training split with 50000 images
cifar10-val - CIFAR-10 validation split with 10000 images
cifar100-train - CIFAR-100 training split with 50000 images
cifar100-val - CIFAR-100 validation split with 10000 images
stl10-train - STL-10 training split with 500 images
stl10-test - STL-10 testing split with 800 images
stl10-unlabeled - STL-10 unlabeled split with 100000 images
Preregistered feature extractors
Can be used as values to the feature_extractor
argument:
inception-v3-compat - a standard InceptionV3 feature extractor from the original reference implementations of the Inception Score. This feature extractor is carefully ported to reproduce the original extractor’s bilinear interpolation and neural architecture quirks.
vgg16 - a legacy VGG-based feature extractor used in the reference implementation of the Precision and Recall metrics.
clip-rn50, clip-rn101, clip-rn50x4, clip-rn50x16, clip-rn50x64, clip-vit-b-32, clip-vit-b-16, clip-vit-l-14, clip-vit-l-14-336px - a set of modern CLIP-based feature extractors for evaluation of more realistic image generators, such as DDPMs.
dinov2-vit-s-14, dinov2-vit-b-14, dinov2-vit-l-14, dinov2-vit-g-14 - a set of modern self-supervised feature extractors, also suitable for state-of-the-art image generators evaluation.
Preregistered sample similarities
Can be used as values to the ppl_sample_similarity
argument:
lpips-vgg16 - a standard LPIPS sample similarity measure, based on a pre-trained VGG-16 and deep feature aggregation.
Preregistered noise source types
Can be used as values to input1_model_z_type
and
input2_model_z_type
arguments:
normal - standard normal distribution
unit - uniform distribution on a unit sphere
uniform_0_1 - standard uniform distribution
Preregistered interpolation methods
Can be used as values to the ppl_z_interp_mode
argument):
lerp - linear interpolation
slerp_any - spherical interpolation of normal samples
slerp_unit - spherical interpolation of unit samples