skimage2.feature.fisher_vector

skimage2.feature.fisher_vector(descriptors, gmm, *, improved=False, alpha=0.5)

Compute the Fisher vector given some descriptors/vectors, and an associated estimated GMM.

Parameters:

descriptorsnp.ndarray, shape=(n_descriptors, descriptor_length)

NumPy array of the descriptors for which the Fisher vector representation is to be computed.

gmmsklearn.mixture.GaussianMixture

An estimated GMM object, which contains the necessary parameters needed to compute the Fisher vector.

improvedbool, default=False

Flag denoting whether to compute improved Fisher vectors or not. Improved Fisher vectors are L2 and power normalized. Power normalization is simply f(z) = sign(z) pow(abs(z), alpha) for some 0 <= alpha <= 1.

alphafloat, default=0.5

The parameter for the power normalization step. Ignored if improved=False.

Returns:

fisher_vectornp.ndarray

The computation Fisher vector, which is given by a concatenation of the gradients of a GMM with respect to its parameters (mixture weights, means, and covariance matrices). For D-dimensional input descriptors or vectors, and a K-mode GMM, the Fisher vector dimensionality will be 2KD + K. Thus, its dimensionality is invariant to the number of descriptors/vectors.

References

[1]

Perronnin, F. and Dance, C. Fisher kernels on Visual Vocabularies for Image Categorization, IEEE Conference on Computer Vision and Pattern Recognition, 2007

[2]

Perronnin, F. and Sanchez, J. and Mensink T. Improving the Fisher Kernel for Large-Scale Image Classification, ECCV, 2010

Examples

>>> from _skimage2.feature import fisher_vector, learn_gmm
>>> sift_for_images = [np.random.random((10, 128)) for _ in range(10)]
>>> num_modes = 16
>>> # Estimate 16-mode GMM with these synthetic SIFT vectors
>>> gmm = learn_gmm(sift_for_images, n_modes=num_modes)
>>> test_image_descriptors = np.random.random((25, 128))
>>> # Compute the Fisher vector
>>> fv = fisher_vector(test_image_descriptors, gmm)