Spaces:
Sleeping
Sleeping
| title: PanopticQuality | |
| tags: | |
| - evaluate | |
| - metric | |
| description: >- | |
| PanopticQuality score | |
| sdk: gradio | |
| sdk_version: 3.19.1 | |
| app_file: app.py | |
| pinned: false | |
| emoji: 🕵️ | |
| # SEA-AI/PanopticQuality | |
| This hugging face metric uses `seametrics.segmentation.PanopticQuality` under the hood to compute a panoptic quality score. It is a wrapper class for the torchmetrics class [`torchmetrics.detection.PanopticQuality`](https://lightning.ai/docs/torchmetrics/stable/detection/panoptic_quality.html). | |
| ## Getting Started | |
| To get started with PanopticQuality, make sure you have the necessary dependencies installed. This metric relies on the `evaluate`, `seametrics` and `seametrics[segmentation]`libraries for metric calculation and integration with FiftyOne datasets. | |
| ### Basic Usage | |
| ```python | |
| >>> import evaluate | |
| >>> from seametrics.fo_utils.utils import fo_to_payload | |
| >>> MODEL_FIELD = ["maskformer-27k-100ep"] | |
| >>> payload = fo_to_payload("SAILING_PANOPTIC_DATASET_QA", | |
| >>> gt_field="ground_truth_det", | |
| >>> models=MODEL_FIELD, | |
| >>> sequence_list=["Trip_55_Seq_2", "Trip_197_Seq_1", "Trip_197_Seq_68"], | |
| >>> excluded_classes=[""]) | |
| >>> module = evaluate.load("SEA-AI/PanopticQuality") | |
| >>> module.add_payload(payload, model_name=MODEL_FIELD[0]) | |
| >>> module.compute() | |
| 100%|██████████| 3/3 [00:03<00:00, 1.30s/it] | |
| Added data ... | |
| Start computing ... | |
| Finished! | |
| tensor(0.2082, dtype=torch.float64) | |
| ``` | |
| ## Metric Settings | |
| The metric takes two optional input parameters: __label2id__ and __stuff__. | |
| * `label2id: Dict[str, int]`: this dictionary is used to map string labels to an integer representation. | |
| if not provided a default setting will be used: | |
| `{'WATER': 0, | |
| 'SKY': 1, | |
| 'LAND': 2, | |
| 'MOTORBOAT': 3, | |
| 'FAR_AWAY_OBJECT': 4, | |
| 'SAILING_BOAT_WITH_CLOSED_SAILS': 5, | |
| 'SHIP': 6, | |
| 'WATERCRAFT': 7, | |
| 'SPHERICAL_BUOY': 8, | |
| 'CONSTRUCTION': 9, | |
| 'FLOTSAM': 10, | |
| 'SAILING_BOAT_WITH_OPEN_SAILS': 11, | |
| 'CONTAINER': 12, | |
| 'PILLAR_BUOY': 13} | |
| ` | |
| * `stuff: List[str]`: this list holds all string labels that belong to stuff. | |
| if not provided a default setting will be used: | |
| ` | |
| ["WATER", "SKY", "LAND", "CONSTRUCTION", "ICE", "OWN_BOAT"]` | |
| ## Output Values | |
| A single float number between 0 and 1 is returned, which represents the PQ score. The bigger the number the better the PQ score, and vice versa. | |
| ## Further References | |
| - **seametrics Library**: Explore the [seametrics GitHub repository](https://github.com/SEA-AI/seametrics/tree/main) for more details on the underlying library. | |
| - **Torchmetrics**: https://lightning.ai/docs/torchmetrics/stable/detection/panoptic_quality.html | |
| - **Understanding Metrics**: The Panoptic Segmentation task, as well as Panoptic Quality as the evaluation metric, were introduced [in this paper](https://arxiv.org/pdf/1801.00868.pdf). | |
| ## Contribution | |
| Your contributions are welcome! If you'd like to improve SEA-AI/PanopticQuality or add new features, please feel free to fork the repository, make your changes, and submit a pull request. |