update readme

README.md

@@ -28,7 +28,7 @@ First, ensure you have Python 3.8 or later installed. Then, install det-metrics
 pip install evaluate git+https://github.com/SEA-AI/seametrics@develop
 ```
 
-### Basic Usage
+## Basic Usage
 
 Here's how to quickly evaluate your object detection models using SEA-AI/det-metrics:
 
@@ -60,22 +60,23 @@ results = module.compute()
 print(results)
 ```
 
-This will output the evaluation metrics for your detection model.
-```
-{'all': {'range': [0, 10000000000.0],
-  'iouThr': '0.00',
-  'maxDets': 100,
-  'tp': 2,
-  'fp': 0,
-  'fn': 0,
-  'duplicates': 0,
-  'precision': 1.0,
-  'recall': 1.0,
-  'f1': 1.0,
-  'support': 2,
-  'fpi': 0,
-  'nImgs': 1}
+This will output the following dictionary containing metrics for the detection model. The key of the dictionary will be the model name or "custom" if no model names are available like in this case.
+
+```json
+{
+    "custom": {
+        "metrics": ...,
+        "eval": ...,
+        "params": ...
+    }
+}
 ```
+- `metrics`: A dictionary containing performance metrics for each area range
+- `eval`: Output of COCOeval.accumulate()
+- `params`: COCOeval parameters object
+
+See [Output Values](#output-values) for more detailed information about the returned results structure, which includes metrics, eval, and params fields for each model passed as input.
+
 
 ## FiftyOne Integration
 
@@ -92,35 +93,40 @@ logging.basicConfig(level=logging.WARNING)
 processor = PayloadProcessor(
     dataset_name="SAILING_DATASET_QA",
     gt_field="ground_truth_det",
-    models=["yolov5n6_RGB_D2304-v1_9C"],
+    models=["yolov5n6_RGB_D2304-v1_9C", "tf1zoo_ssd-mobilenet-v2_agnostic_D2207"],
     sequence_list=["Trip_14_Seq_1"],
     data_type="rgb",
 )
 
 # Evaluate using SEA-AI/det-metrics
-module = evaluate.load("SEA-AI/det-metrics")
-module.add_payload(processor.payload)
+module = evaluate.load("det-metrics", payload=processor.payload)
 results = module.compute()
 
 print(results)
 ```
-
-```console
-{'all': {'range': [0, 10000000000.0],
-  'iouThr': '0.00',
-  'maxDets': 100,
-  'tp': 89,
-  'fp': 13,
-  'fn': 15,
-  'duplicates': 1,
-  'precision': 0.8725490196078431,
-  'recall': 0.8557692307692307,
-  'f1': 0.8640776699029126,
-  'support': 104,
-  'fpi': 0,
-  'nImgs': 22}}
+This will output the following dictionary containing metrics for the detection model. The key of the dictionary will be the model name. 
+```json
+{
+    "yolov5n6_RGB_D2304-v1_9C": {
+        "metrics": ...,
+        "eval": ...,
+        "params": ...
+    },
+    "tf1zoo_ssd-mobilenet-v2_agnostic_D2207": {
+        "metrics": ...,
+        "eval": ...,
+        "params": ...
+    }
+}
 ```
 
+- `metrics`: A dictionary containing performance metrics for each area range
+- `eval`: Output of COCOeval.accumulate()
+- `params`: COCOeval parameters object
+
+See [Output Values](#output-values) for more detailed information about the returned results structure, which includes metrics, eval, and params fields for each model passed as input.
+
+
 ## Metric Settings
 
 Customize your evaluation by specifying various parameters when loading SEA-AI/det-metrics:
@@ -146,8 +152,32 @@ module = evaluate.load(
 ```
 
 ## Output Values
+For every model passed as input, the results contain the metrics, eval, and params fields. If no specific model was passed (usage without payload), the default model name “custom” will be used.
+
+```json
+{
+    "model_1": {
+        "metrics": ...,
+        "eval": ...,
+        "params": ...
+    },
+    "model_2": {
+        "metrics": ...,
+        "eval": ...,
+        "params": ...
+    },
+    "model_3": {
+        "metrics": ...,
+        "eval": ...,
+        "params": ...
+    },
+    ...
+}
+```
 
-SEA-AI/det-metrics provides a detailed breakdown of performance metrics for each specified area range:
+### Metrics
+
+SEA-AI/det-metrics metrics dictionary provides a detailed breakdown of performance metrics for each specified area range:
 
 - **range**: The area range considered.
 - **iouThr**: The IOU threshold applied.
@@ -159,6 +189,106 @@ SEA-AI/det-metrics provides a detailed breakdown of performance metrics for each
 - **fpi**: Number of images with predictions but no ground truths.
 - **nImgs**: Total number of images evaluated.
 
+### Eval
+
+The SEA-AI/det-metrics evaluation dictionary provides details about evaluation metrics and results. Below is a description of each field:
+
+- **params**: Parameters used for evaluation, defining settings and conditions.
+  
+- **counts**: Dimensions of parameters used in evaluation, represented as a list [T, R, K, A, M]:
+  - T: IoU threshold (default: [1e-10])
+  - R: Recall threshold (not used)
+  - K: Class index (class-agnostic, so only 0)
+  - A: Area range (0=all, 1=valid_n, 2=valid_w, 3=tiny, 4=small, 5=medium, 6=large)
+  - M: Max detections (default: [100])
+  
+- **date**: The date when the evaluation was performed.
+  
+- **precision**: A multi-dimensional array [TxRxKxAxM] storing precision values for each evaluation setting.
+  
+- **recall**: A multi-dimensional array [TxKxAxM] storing maximum recall values for each evaluation setting.
+  
+- **scores**: Scores for each detection.
+  
+- **TP**: True Positives - correct detections matching ground truth.
+  
+- **FP**: False Positives - incorrect detections not matching ground truth.
+  
+- **FN**: False Negatives - ground truth objects not detected.
+  
+- **duplicates**: Duplicate detections of the same object.
+  
+- **support**: Number of ground truth objects for each category.
+  
+- **FPI**: False Positives per Image.
+  
+- **TPC**: True Positives per Category.
+  
+- **FPC**: False Positives per Category.
+  
+- **sorted_conf**: Confidence scores of detections sorted in descending order.
+
+> Note:
+> **precision** and **recall** are set to -1 for settings with no ground truth objects.
+
+### Params
+
+The params return value of the COCO evaluation parameters in PyCOCO represents a dictionary with various evaluation settings that can be customized. Here’s a breakdown of what each parameter means:
+
+- **imgIds**: List of image IDs to use for evaluation. By default, it evaluates on all images.
+- **catIds**: List of category IDs to use for evaluation. By default, it evaluates on all categories.
+- **iouThrs**: List of IoU (Intersection over Union) thresholds for evaluation. By default, it uses thresholds from 0.5 to 0.95 with a step of 0.05 (i.e., [0.5, 0.55, …, 0.95]).
+- **recThrs**: List of recall thresholds for evaluation. By default, it uses 101 thresholds from 0 to 1 with a step of 0.01 (i.e., [0, 0.01, …, 1]).
+- **areaRng**: Object area ranges for evaluation. This parameter defines the sizes of objects to evaluate. It is specified as a list of tuples, where each tuple represents a range of area in square pixels.
+- **maxDets**: List of thresholds on maximum detections per image for evaluation. By default, it evaluates with thresholds of 1, 10, and 100 detections per image.
+- **iouType**: Type of IoU calculation used for evaluation. It can be ‘segm’ (segmentation), ‘bbox’ (bounding box), or ‘keypoints’.
+- **useCats**: Boolean flag indicating whether to use category labels for evaluation (default is 1, meaning true).
+
+> Note:
+> If useCats=0 category labels are ignored as in proposal scoring.
+> Multiple areaRngs [Ax2] and maxDets [Mx1] can be specified.
+
+## Confidence Curves
+When you use **module.generate_confidence_curves()**, it creates a graph that shows how metrics like **precision, recall, and f1 score** change as you adjust confidence thresholds. This helps you see the trade-offs between precision (how accurate positive predictions are) and recall (how well the model finds all positive instances) at different confidence levels. As confidence scores go up, models usually have higher precision but may find fewer positive instances, reflecting their certainty in making correct predictions.
+
+#### Confidence Config
+The `confidence_config` dictionary is set as `{"T": 0, "R": 0, "K": 0, "A": 0, "M": 0}`, where:
+- `T = 0`: represents the IoU (Intersection over Union) threshold.
+- `R = 0`: is the recall threshold, although it's currently not used.
+- `K = 0`: indicates a class index for class-agnostic mean Average Precision (mAP), with only one class indexed at 0.
+- `A = 0`: signifies that all object sizes are considered for evaluation. (0=all, 1=small, 2=medium, 3=large, ... depending on area ranges) 
+- `M = 0`: sets the default maximum detections (`maxDets`) to 100 in precision_recall_f1_support calculations.
+
+
+```python
+import evaluate
+import logging
+from seametrics.payload.processor import PayloadProcessor
+
+logging.basicConfig(level=logging.WARNING)
+
+# Configure your dataset and model details
+processor = PayloadProcessor(
+    dataset_name="SAILING_DATASET_QA",
+    gt_field="ground_truth_det",
+    models=["yolov5n6_RGB_D2304-v1_9C"],
+    sequence_list=["Trip_14_Seq_1"],
+    data_type="rgb",
+)
+
+# Evaluate using SEA-AI/det-metrics
+module = evaluate.load("det-metrics", payload=processor.payload)
+results = module.compute()
+
+# Plot confidence curves
+confidence_config={"T": 0, "R": 0, "K": 0, "A": 0, "M": 0}
+fig = module.generate_confidence_curves(results, confidence_config)
+fig.show()
+```
+
+![Alt text](assets/example_confidence_curves.png)
+
+
 ## Further References
 
 - **seametrics Library**: Explore the [seametrics GitHub repository](https://github.com/SEA-AI/seametrics/tree/main) for more details on the underlying library.