Utilities

Templates

Template classes for local image feature detection and description algorithms.

class featmf.templates.KptDetDescAlgo

A template for image local keypoint detection and description algorithms.

Rules for the child classes:

- The child classes must implement the function detect_and_describe() that returns a KptDetDescAlgo.Result object. They must sanitize the input types to suit the particular algorithm.
- __call__ function calls detect_and_describe(). The child classes need not implement this.
class Result(keypoints: Optional[Union[ndarray, Tensor]], descriptors: Optional[Union[ndarray, Tensor]], scores: Optional[Union[ndarray, Tensor]])

Result for a keypoint detection and description algorithm. Includes keypoints, descriptors, and scores. Each of these items can contain a numpy array, a torch tensor, or None.

Note

Check the documentation of the algorithm for types, shapes, and descriptions of these items. A generic description is given in this class.

__init__(keypoints: Optional[Union[ndarray, Tensor]], descriptors: Optional[Union[ndarray, Tensor]], scores: Optional[Union[ndarray, Tensor]]) None
copy() Result

Returns a deepcopy of the self item.

Return type:

KptDetDescAlgo.Result

d_dim() int

Returns the descriptor dimension (if descriptors have been loaded). Otherwise, returns 0.

descriptors: Optional[Union[ndarray, Tensor]]

Descriptors for the keypoints detected by the algorithm. It’s shape must be [N, D] where N is the number of keypoints and D is the descriptor dimension.

get(i)
Parameters:

i (Union[int, slice, list[int], str]) – Item to be returned

Depending upon the type of i, returns one of the following

- str: Should be in [‘keypoints’, ‘descriptors’, ‘scores’]. Returns the corresponding item. Type will be the same as the original item.
- int: Returns a new result containing only the i-th keypoint, descriptor, and score (if present). N = 1.
- slice: Returns a new result containing only the keypoints, descriptors, and scores in the specified slice.
- list[int]: Returns a new result containing only the keypoints, descriptors, and scores at the specified indices.

Note

Only in the case of str type, the returned value is a direct reference (editing it will edit the original result). In all other cases, a new result is created.

Raises:
  • IndexError – If keypoints not initialized

  • TypeError – If i is of an invalid type

Tip

This function is also implemented in the __getitem__ method for usual [i] indexing.

Returns:

The requested indices (or item)

Return type:

Union[KptDetDescAlgo.Result, np.ndarray, torch.Tensor, None]

keypoints: Optional[Union[ndarray, Tensor]]

Keypoints detected by the algorithm. Depending upon the algorithm, its shape can be one of the following

- [N, 2] for 2D keypoints (x, y) without size information.
- [N, 3] for 2D keypoints (x, y, s) with size information (diameter of the circle centered at the keypoint).
- [N, 4] for 2D keypoints (x, y, s, ang) with size and angle information. The angles are in radians and are measured clockwise from the +ve x-axis (which is to the right of the image).

Where N is the number of keypoints detected.

Note

Using [N, 3] as (x, y, ang) is discouraged (set size as 1 and use [N, 4] instead).

See the documentation of the particular algorithm for more details.

len()

Returns the number of keypoints (0 if no keypoints).

repr()

Returns a string representation of the object describing the keypoints, descriptors, and scores.

Tip

This function is also implemented in the __repr__ method for usual print calls.

scores: Optional[Union[ndarray, Tensor]]

Scores for the keypoints detected by the algorithm. It’s shape must be [N,] where N is the number of keypoints. This could be None if the algorithm does not provide detection scores (confidence).

sort_scores(top_k: Optional[int] = None, ascending: bool = True) Result

Sort the result based on the detection scores. To use this function, the scores should not be None.

Parameters:
  • top_k (Union[int, None]) – If not None, only the top-k detections are retained (others are discarded). Default is None.

  • ascending (bool) – If True, the scores are sorted in the ascending order. If False, the scores are sorted in the descending order. Default is True.

Raises:
  • AssertionError – If scores are None.

  • TypeError – If scores are not of correct type.

Returns self:

The result object (scores are sorted).

Note

This function changes the data contained in the called object.

Return type:

KptDetDescAlgo.Result

detect_and_describe(img: Union[ndarray, Tensor, Image], *args: Any, **kwargs: Any) Result
Parameters:
  • img (Union[Image.Image, np.ndarray, torch.Tensor]) – A single image to be processed.

  • args – Additional arguments

  • kwargs – Additional keyword arguments

Note

The child classes should enforce type constraints. This is because some algorithms may not work with certain types, or might require preprocessing of the image. See the documentation of the child classes for more information.

Raises:
  • TypeError – If img is of an invalid type (for the particular algorithm).

  • NotImplementedError – If the child class does not implement this method.

Abstract method that child classes should inherit to implement the detection and description algorithm.

Tip

The __call__ method calls this method directly. It has the same inputs and outputs.

Returns:

The result of the detection and description

Return type:

KptDetDescAlgo.Result

repr() str

Returns a string representation of the algorithm (name)

Other utilities

featmf.utilities.draw_keypoints(img: ndarray, pts: ndarray, offset: Tuple[int, int] = (0, 0), color: Optional[Union[ndarray, Tuple[int, int, int]]] = None, sc=1, draw_angle: bool = True) ndarray

Draw keypoints on an image (with provision to add an offset). All keypoints are drawn as circles. If no keypoint scale is specified, the radius of the circle (keypoint neighborhood) is set to 10 pixels. Note that the keypoint size is the diameter of the circle centered at the keypoint. If angles are not provided, no angle lines are drawn.

Parameters:
  • img – The image on which to draw the keypoints. Shape must be [H, W, 3] (RGB image). If a grayscale image is provided, it’s converted to RGB (same value for all channels). The passed image is not modified.

  • pts – The keypoints to draw. If shape is [N, 2], they’re assumed to be [x, y]. If shape is [N, 3], they’re assumed to be [x, y, size] (size is diameter). If shape is [N, 4], they’re assumed to be [x, y, size, angle] (angle in radians, measured clockwise from +ve x-axis/right).

  • offset – The (x, y) offset to add to the keypoints before drawing.

  • color – The color to draw the keypoints. If not provided, keypoints are drawn with random colors. If (R, G, B) tuple is provided, all keypoints are drawn with the same color. It can also be a list of colors.

  • sc – A scalar multiple for the keypoint sizes (for drawing). If size is not provided, this is still used to scale the default radius of 10.

  • draw_angle – If False, keypoint angle lines are not drawn even if keypoints contain angles.

Returns:

An image with the keypoints drawn on it. The shape is [H, W, 3] (RGB image).

featmf.utilities.kpts_cv2np(kpts_cv: List[KeyPoint], parse_size=False, parse_angle=False, angle_conv_rad=True, ret_response=False) Union[ndarray, Tuple[ndarray, ndarray]]

Convert a list of OpenCV keypoints into numpy array(s). By default, only the keypoints (x, y) are extracted and returned.

Parameters:
  • kpts_cv (List[cv.KeyPoint]) – A list of OpenCV keypoint objects.

  • parse_size – If True, the size attribute of each keypoint is also parsed.

  • parse_angle – If True, the angle attribute of each keypoint is also parsed.

  • angle_conv_rad – If True, the angle in KeyPoint is assumed to be in degrees and is subsequently converted to radians.

  • ret_response – If True, the response attribute of each keypoint is returned in a separate array.

Returns:

A numpy array containing the keypoints. If parse_size is True, a tuple is returned, with the second element being the array of scores.

For the shape of keypoint array:

- If parse_size is False and parse_angle is False, the returned array is of shape [N, 2] (each row is [x, y]).
- If parse_size is True, the returned array is of shape [N, 3] (each row is [x, y, size]).
- If parse_angle is also True, the returned array is of shape [N, 4] (each row is [x, y, size, angle]).
- If parse_size is False and parse_angle is True, the returned array is of shape [N, 3] (each row is [x, y, angle]).

The shape of the scores array (returned only if ret_response is True) is [N,].