[mertalev]

Description

This PR addresses some limitations of the ML service design. Currently, detection and recognition models for facial recognition are bundled in the same class and likewise for textual and visual CLIP models. As a result, they duplicate certain shared behaviors for their own set of models. Moreover, there is no good way to choose particular detection and recognition models. This is a big limitation for OCR, as it is common to mix and match unrelated detection and recognition models. Lastly, there is no way to query an individual detection or recognition model. CLIP models have a custom mode option to do this, but this is specific to that model task.

This PR redesigns the ML service such that each model session is its own class, and broader tasks like facial recognition are modeled as dependencies (recognition being dependent on detection, etc.). This lays a solid foundation for a composable set of models with separate settings for each.

As part of this change, the API has also been updated. A given request can look like this:

{
  "facial-recognition": {
    "detection": {
      "modelName": "buffalo_l"
    },
    "recognition": {
      "modelName": "buffalo_l",
      "options": {
        "minScore": 0.5
      }
    }
  }
}

And the response can look like this:

{
  "facial-recognition": [
    {
      "boundingBox": {
        "x1": 463.0,
        "y1": 133.0,
        "x2": 763.0,
        "y2": 526.0
      },
      "embedding": "vector(512)",
      "score": 0.89526224
    }
  ],
  "imageHeight": 1440,
  "imageWidth": 1152
}

Some implementation notes:

• A given request can only be visual or textual; it cannot have multiple modalities at the same time
• Images are only decoded once and the decoded data is shared across all models
• Dependencies can only go one level deep; a model can depend on the output of multiple models, but those models must not have any dependencies
• If a model has a dependency, it is the caller's responsibility to ensure this dependency is included in the request

For facial recognition, a side effect of this change is drastically better performance when there are multiple faces. This is because they are handled in one model pass instead of fed sequentially. With 10 faces in an image, it was 70%+ faster on CPU and 6x faster on GPU. A smaller gain comes from always using Pillow to decode images, as it is several times faster than OpenCV.

While unlikely to be relevant in practice, a nice perk of this change is that it allows one to query any number of tasks at once, like so:

{
  "clip": {
    "visual": {
      "modelName": "ViT-B-32__openai"
    }
  },
  "facial-recognition": {
    "detection": {
      "modelName": "buffalo_l"
    },
    "recognition": {
      "modelName": "buffalo_l",
      "options": {
        "minScore": 0.5
      }
    }
  },
  "ocr": {
    "detection": {
      "modelName": "ch_ppocr_v4",
      "options": {
        "minScore": 0.5
      }
    },
    "recognition": {
      "modelName": "ch_ppocr_v4",
      "options": {
        "minScore": 0.3
      }
    }
  }
}

[zackpollard]

I'm going to approve this, I don't think any of us really are going to be able to properly review this. Imo if you have tested this and are confident with the change, go ahead and merge it in.

[jrasm91]

LGTM