bowdbeg commited on
Commit
0da91fc
1 Parent(s): fcc706c

update doc

Browse files
Files changed (1) hide show
  1. matching_series.py +32 -34
matching_series.py CHANGED
@@ -25,7 +25,6 @@ import numpy as np
25
  # TODO: Add BibTeX citation
26
  _CITATION = """TBA"""
27
 
28
- # TODO: Add description of the module here
29
  _DESCRIPTION = """\
30
  Matching Series is a metric for evaluating time-series generation models. It is based on the idea of matching the generated time-series with the original time-series. The metric calculates the Mean Squared Error (distance) between the generated time-series and the original time-series between matched instances. The metric outputs a score greater or equal to 0, where 0 indicates a perfect generation.
31
  """
@@ -201,31 +200,6 @@ class matching_series(evaluate.Metric):
201
  pass
202
 
203
  def compute(self, *, predictions=None, references=None, **kwargs) -> Optional[dict]:
204
- """Compute the evaluation module.
205
-
206
- Usage of positional arguments is not allowed to prevent mistakes.
207
-
208
- Args:
209
- predictions (`list/array/tensor`, *optional*):
210
- Predictions.
211
- references (`list/array/tensor`, *optional*):
212
- References.
213
- **kwargs (optional):
214
- Keyword arguments that will be forwarded to the evaluation module [`~evaluate.EvaluationModule.compute`]
215
- method (see details in the docstring).
216
-
217
- Return:
218
- `dict` or `None`
219
-
220
- - Dictionary with the results if this evaluation module is run on the main process (`process_id == 0`).
221
- - `None` if the evaluation module is not run on the main process (`process_id != 0`).
222
-
223
- ```py
224
- >>> import evaluate
225
- >>> accuracy = evaluate.load("accuracy")
226
- >>> accuracy.compute(predictions=[0, 1, 1, 0], references=[0, 1, 0, 1])
227
- ```
228
- """
229
  all_kwargs = {"predictions": predictions, "references": references, **kwargs}
230
  if predictions is None and references is None:
231
  missing_kwargs = {k: None for k in self._feature_names() if k not in all_kwargs}
@@ -259,16 +233,40 @@ class matching_series(evaluate.Metric):
259
  eps: float = 1e-8,
260
  ):
261
  """
262
- Compute the scores of the module given the predictions and references
 
263
  Args:
264
- predictions: list of generated time series.
265
- shape: (num_generation, num_timesteps, num_features)
266
- references: list of reference
267
- shape: (num_reference, num_timesteps, num_features)
268
- batch_size: batch size to use for the computation. If None, the whole dataset is processed at once.
269
- cuc_n_calculation: number of Coverage Under Curve calculate times
270
- cuc_n_samples: number of samples to use for Coverage Under Curve calculation. If "auto", it uses the number of samples of the predictions.
 
 
 
 
 
 
 
 
271
  Returns:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
272
  """
273
  if return_all:
274
  return_distance = True
 
25
  # TODO: Add BibTeX citation
26
  _CITATION = """TBA"""
27
 
 
28
  _DESCRIPTION = """\
29
  Matching Series is a metric for evaluating time-series generation models. It is based on the idea of matching the generated time-series with the original time-series. The metric calculates the Mean Squared Error (distance) between the generated time-series and the original time-series between matched instances. The metric outputs a score greater or equal to 0, where 0 indicates a perfect generation.
30
  """
 
200
  pass
201
 
202
  def compute(self, *, predictions=None, references=None, **kwargs) -> Optional[dict]:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
203
  all_kwargs = {"predictions": predictions, "references": references, **kwargs}
204
  if predictions is None and references is None:
205
  missing_kwargs = {k: None for k in self._feature_names() if k not in all_kwargs}
 
233
  eps: float = 1e-8,
234
  ):
235
  """
236
+ Compute the Matching Series metric
237
+
238
  Args:
239
+ predictions: list of list of list of float or numpy.ndarray: The generated time-series. The shape of the array should be `(num_generation, seq_len, num_features)`.
240
+ references: list of list of list of float or numpy.ndarray: The original time-series. The shape of the array should be `(num_reference, seq_len, num_features)`.
241
+ batch_size: int, optional: The batch size for computing the metric. This affects quadratically. Default is None.
242
+ cuc_n_calculation: int, optional: The number of samples to compute the coverage because sampling exists. Default is 3.
243
+ cuc_n_samples: list of int, optional: The number of samples to compute the coverage. Default is $[2^i \text{for} i \leq \log_2 n] + [n]$.
244
+ metric: str, optional: The metric to measure distance between examples. Default is "mse". Available options are "mse", "mae", "rmse".
245
+ num_processes: int, optional: The number of processes to use for computing the distance. Default is 1.
246
+ instance_normalization: bool, optional: Whether to normalize the instances along the time axis. Default is False.
247
+ return_distance: bool, optional: Whether to return the distance matrix. Default is False.
248
+ return_matching: bool, optional: Whether to return the matching matrix. Default is False.
249
+ return_each_features: bool, optional: Whether to return the results for each feature. Default is False.
250
+ return_coverages: bool, optional: Whether to return the coverages. Default is False.
251
+ return_all: bool, optional: Whether to return all the results. Default is False.
252
+ dtype: str, optional: The data type used for computation. Default is "float32".
253
+ eps: float, optional: The epsilon value to avoid division by zero. Default is 1e-8.
254
  Returns:
255
+ dict: A dictionary containing the following keys:
256
+ precision_distance (float): The precision of the distance.
257
+ recall_distance (float): The recall of the distance.
258
+ mean_distance (float): The mean of the distance.
259
+ index_distance (float): The index of the distance.
260
+ matching_precision (float): The precision of the matching instances.
261
+ matching_recall (float): The recall of the matching instances.
262
+ matching_f1 (float): The F1-score of the matching instances.
263
+ coverages (list of float): The coverages.
264
+ cuc (float): The coverage under the curve.
265
+ macro_.* (float): The macro value of the .*.
266
+ .*_features (list of float): The values computed individually for each feature.
267
+ distance (numpy.ndarray): The distance matrix.
268
+ match (numpy.ndarray): The matching matrix.
269
+ match_inv (numpy.ndarray): The inverse matching matrix.
270
  """
271
  if return_all:
272
  return_distance = True