Commit edc776a
Add OWL-ViT base-patch32 to zoo manifest
Added OWL-ViT base model with patch32 configuration to complete the OWL-ViT family:
- `owlvit-base-patch32-torch`
The model:
- Uses existing `FiftyOneZeroShotTransformerForObjectDetection` wrapper
- Supports both CPU and GPU inference
- Performs zero-shot object detection using text queries
- Successfully extracts embeddings for similarity matching
- Leverages HuggingFace hub for automatic model downloads
OWL-ViT base-patch32 completes the model family by providing the 32x32 patch variant alongside the existing patch16 option, giving users choice in the patch size/performance trade-off for their specific use cases.
### What changes are proposed in this pull request?
This PR adds the OWL-ViT base-patch32 model configuration to `fiftyone/zoo/models/manifest-torch.json`, completing the OWL-ViT model offerings. The model uses the same wrapper class as the existing patch16 variant but points to `google/owlvit-base-patch32` on HuggingFace hub.
### How is this patch tested? If it is not, please explain why.
Created and ran tests that:
- Load the model from HuggingFace via `foz.load_zoo_model("owlvit-base-patch32-torch")`
- Perform zero-shot object detection with custom text queries
- Verify detection outputs and bounding box generation
- Test embedding extraction capabilities
- Apply model to FiftyOne datasets
- Confirm compatibility with the existing transformer wrapper
The model successfully detected objects (horses, etc.) using various text prompts and passed all tests.
### Release Notes
**Is this a user-facing change that should be mentioned in the release notes?**
- [x] Yes. Give a description of this change to be included in the release notes for FiftyOne users.
Added OWL-ViT base-patch32 model to the model zoo for zero-shot object detection. Users can now access both patch16 and patch32 variants of OWL-ViT via `foz.load_zoo_model()`, allowing selection based on their requirements.
### What areas of FiftyOne does this PR affect?
- [ ] App: FiftyOne application changes
- [ ] Build: Build and test infrastructure changes
- [ ] Core: Core fiftyone Python library changes
- [ ] Documentation: FiftyOne documentation changes
- [x] Other1 parent 14f60f5 commit edc776a
1 file changed
+34
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2965 | 2965 | | |
2966 | 2966 | | |
2967 | 2967 | | |
| 2968 | + | |
| 2969 | + | |
| 2970 | + | |
| 2971 | + | |
| 2972 | + | |
| 2973 | + | |
| 2974 | + | |
| 2975 | + | |
| 2976 | + | |
| 2977 | + | |
| 2978 | + | |
| 2979 | + | |
| 2980 | + | |
| 2981 | + | |
| 2982 | + | |
| 2983 | + | |
| 2984 | + | |
| 2985 | + | |
| 2986 | + | |
| 2987 | + | |
| 2988 | + | |
| 2989 | + | |
| 2990 | + | |
| 2991 | + | |
| 2992 | + | |
| 2993 | + | |
| 2994 | + | |
| 2995 | + | |
| 2996 | + | |
| 2997 | + | |
| 2998 | + | |
| 2999 | + | |
| 3000 | + | |
| 3001 | + | |
2968 | 3002 | | |
2969 | 3003 | | |
2970 | 3004 | | |
| |||
0 commit comments