Update README.md (#15)

- Update README.md (1381bed20ff17a272401064b0446861c04ea7d8b)
- Update README.md (1fde69e3a12aab816ed634106f5c2d357e894e54)
- Update README.md (c89fbf3ddd7779407458652c8ebaca0718a6055f)
- Update README.md (97a16257a945a1d5431a4c1dd732a5d94982866e)
- Update README.md (7da32b4d38136ff2958b91d72d21d1b7d2ae4b6f)
- Update README.md (d37bfc5c00fc41c9c34bd7290b26de7056fa9e47)
- Update README.md (b6ec1baea47216c5ff9ede817d2aa675f22a3679)

README.md

@@ -10,18 +10,22 @@ tags:
 - time-series
 ---
 
-# TinyTimeMixer (TTM) Model Card
+# TinyTimeMixer (TTM) 1M Model Card
 
 <p align="center" width="100%">
 <img src="ttm_image.webp" width="600">
 </p>
 
-TinyTimeMixers (TTMs) are compact pre-trained models for Multivariate Time-Series Forecasting, open-sourced by IBM Research.
+TinyTimeMixers (TTMs) are compact pre-trained models for Multivariate Time-Series Forecasting, open-sourced by IBM Research. 
 **With less than 1 Million parameters, TTM introduces the notion of the first-ever “tiny” pre-trained models for Time-Series Forecasting.** 
 
+
+TTM is accepted in NeurIPS 2024. 
+
+
 TTM outperforms several popular benchmarks demanding billions of parameters in zero-shot and few-shot forecasting. TTMs are lightweight 
 forecasters, pre-trained on publicly available time series data with various augmentations. TTM provides state-of-the-art zero-shot forecasts and can easily be 
-fine-tuned for multi-variate forecasts with just 5% of the training data to be competitive. Refer to our [paper](https://arxiv.org/pdf/2401.03955v5.pdf) for more details.
+fine-tuned for multi-variate forecasts with just 5% of the training data to be competitive. Refer to our [paper](https://arxiv.org/pdf/2401.03955.pdf) for more details.
 
 
 **The current open-source version supports point forecasting use-cases specifically ranging from minutely to hourly resolutions 
@@ -30,18 +34,50 @@ fine-tuned for multi-variate forecasts with just 5% of the training data to be c
 **Note that zeroshot, fine-tuning and inference tasks using TTM can easily be executed in 1 GPU machine or in laptops too!!**
 
 
-**Recent updates:** We have developed more sophisticated variants of TTMs (TTM-B, TTM-E and TTM-A), featuring extended benchmarks that compare them with some of the latest models 
-such as TimesFM, Moirai, Chronos, Lag-llama, and Moment. For full details, please refer to the latest version of our [paper](https://arxiv.org/pdf/2401.03955.pdf). 
-Stay tuned for the release of the model weights for these newer variants.
 
-## How to Get Started with the Model
+## Model Releases (along with the branch name where the models are stored):
+
+
+- **512-96:** Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
+ in future. This model is targeted towards a forecasting setting of context length 512 and forecast length 96 and
+ recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). This model refers to the TTM-Q variant used in the paper. (branch name: main) [[Benchmark Scripts]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_512_96.ipynb)
+
+- **1024-96:** Given the last 1024 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
+ in future. This model is targeted towards a long forecasting setting of context length 1024 and forecast length 96 and
+ recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 1024-96-v1) [[Benchmark Scripts]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
+
+- **New Releases (trained on larger pretraining datasets, released on October 2024)**:
+
+ - **512-96-r2**: Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
+ in future. This model is pre-trained with a larger pretraining dataset for improved accuracy. Recommended for hourly and minutely
+ resolutions (Ex. 10 min, 15 min, 1 hour, etc). This model refers to the TTM-B variant used in the paper (branch name: 512-96-r2) [[Benchmark Scripts]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/tinytimemixer/ttm_v2_benchmarking_512_96.ipynb)
+
+
+ 
+
+## Model Capabilities with example scripts
 
-Please refer to the below scrips for **zero-shot** and **finetuning** support:
-- [colab](https://colab.research.google.com/github/IBM/tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb) 
-- [512-96 Benchmarks](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_512_96.ipynb)
-- [1024-96 Benchmarks](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
-- Script for Exogenous support - to be added soon
+The below model scripts can be used for any of the above TTM models. Please update the HF model URL and branch name in the `from_pretrained` call appropriately to pick the model of your choice.
 
+- Getting Started [[colab]](https://colab.research.google.com/github/IBM/tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb) 
+- Zeroshot Multivariate Forecasting [[Example]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/ttm_getting_started.ipynb)
+- Finetuned Multivariate Forecasting:
+ - Channel-Independent Finetuning [[Example]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/ttm_getting_started.ipynb) [M4-Hourly finetuning](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb)
+ - Channel-Mix Finetuning [[Example]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/tutorial/ttm_channel_mix_finetuning.ipynb)
+- **New Releases (extended features released on October 2024)**
+ - Finetuning and Forecasting with Exogenous/Control Variables [[Example]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
+ - Finetuning and Forecasting with static categorical features [Example: To be added soon]
+ - Rolling Forecasts - Extend forecast lengths beyond 96 via rolling capability [[Example]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/hfdemo/ttm_rolling_prediction_getting_started.ipynb)
+ - Helper scripts for optimal Learning Rate suggestions for Finetuning [[Example]](https://github.com/ibm-granite/granite-tsfm/blob/ttm_v2_release/notebooks/tutorial/ttm_with_exog_tutorial.ipynb)
+ 
+## Benchmarks
+
+TTM outperforms popular benchmarks such as TimesFM, Moirai, Chronos, Lag-Llama, Moment, GPT4TS, TimeLLM, LLMTime in zero/fewshot forecasting while reducing computational requirements significantly. 
+Moreover, TTMs are lightweight and can be executed even on CPU-only machines, enhancing usability and fostering wider
+adoption in resource-constrained environments. For more details, refer to our [paper](https://arxiv.org/pdf/2401.03955.pdf) TTM-Q referred in the paper maps to the `512-96` model
+uploaded in the main branch, and TTM-B referred in the paper maps to the `512-96-r2` model. Please note that the Granite TTM models are pre-trained exclusively on datasets 
+with clear commercial-use licenses that are approved by our legal team. As a result, the pre-training dataset used in this release differs slightly from the one used in the research 
+paper, which may lead to minor variations in model performance as compared to the published results. Please refer to our paper for more details.
 
 ## Recommended Use
 1. Users have to externally standard scale their data independently for every channel before feeding it to the model (Refer to [TSP](https://github.com/IBM/tsfm/blob/main/tsfm_public/toolkit/time_series_preprocessor.py), our data processing utility for data scaling.)
@@ -49,24 +85,6 @@ Please refer to the below scrips for **zero-shot** and **finetuning** support:
 3. Enabling any upsampling or prepending zeros to virtually increase the context length for shorter-length datasets is not recommended and will
  impact the model performance. 
 
-## Benchmark Highlights:
-
-- TTM (with less than 1 Million parameters) outperforms the following popular Pre-trained SOTAs demanding several hundred Million to Billions of parameters [paper](https://arxiv.org/pdf/2401.03955v5.pdf):
- - *GPT4TS (NeurIPS 23) by 7-12% in few-shot forecasting*
- - *LLMTime (NeurIPS 23) by 24% in zero-shot forecasting*.
- - *SimMTM (NeurIPS 23) by 17% in few-shot forecasting*.
- - *Time-LLM (ICLR 24) by 2-8% in few-shot forecasting*
- - *UniTime (WWW 24) by 27% in zero-shot forecasting.*
-- Zero-shot results of TTM surpass the *few-shot results of many popular SOTA approaches* including
- PatchTST (ICLR 23), PatchTSMixer (KDD 23), TimesNet (ICLR 23), DLinear (AAAI 23) and FEDFormer (ICML 22).
-- TTM (1024-96, released in this model card with 1M parameters) outperforms pre-trained MOIRAI-Small (14M parameters) by 10%, MOIRAI-Base (91M parameters) by 2% and
- MOIRAI-Large (311M parameters) by 3% on zero-shot forecasting (horizon = 96). [[notebook]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_benchmarking_1024_96.ipynb)
-- TTM quick fine-tuning also outperforms the competitive statistical baselines (Statistical ensemble and S-Naive) in
- M4-hourly dataset which existing pretrained TS models are finding difficult to outperform. [[notebook]](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb)
-- TTM takes only a *few seconds for zeroshot/inference* and a *few minutes for finetuning* in 1 GPU machine, as
- opposed to long timing-requirements and heavy computing infra needs of other existing pre-trained models.
-
-
 
 ## Model Description
 
@@ -84,21 +102,10 @@ only 3-6 hours using 6 A100 GPUs, as opposed to several days or weeks in traditi
 Each pre-trained model will be released in a different branch name in this model card. Kindly access the required model using our 
 getting started [notebook](https://github.com/IBM/tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb) mentioning the branch name.
 
-## Model Releases (along with the branch name where the models are stored):
-
-- **512-96:** Given the last 512 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
- in future. This model is targeted towards a forecasting setting of context length 512 and forecast length 96 and
- recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: main) 
-
-- **1024-96:** Given the last 1024 time-points (i.e. context length), this model can forecast up to next 96 time-points (i.e. forecast length)
- in future. This model is targeted towards a long forecasting setting of context length 1024 and forecast length 96 and
- recommended for hourly and minutely resolutions (Ex. 10 min, 15 min, 1 hour, etc). (branch name: 1024-96-v1) 
-
-- Stay tuned for more models !
 
 ## Model Details
 
-For more details on TTM architecture and benchmarks, refer to our [paper](https://arxiv.org/pdf/2401.03955v5.pdf).
+For more details on TTM architecture and benchmarks, refer to our [paper](https://arxiv.org/pdf/2401.03955.pdf).
 
 TTM-1 currently supports 2 modes:
 
@@ -113,22 +120,19 @@ The current release supports multivariate forecasting via both channel independe
 Decoder Channel-Mixing can be enabled during fine-tuning for capturing strong channel-correlation patterns across 
 time-series variates, a critical capability lacking in existing counterparts.
 
-In addition, TTM also supports exogenous infusion and categorical data which is not released as part of this version. 
-Stay tuned for these extended features.
+In addition, TTM also supports exogenous infusion and categorical data infusion.
 
 
- 
- 
 ### Model Sources
 
-- **Repository:** https://github.com/IBM/tsfm/tree/main/tsfm_public/models/tinytimemixer
-- **Paper:** https://arxiv.org/pdf/2401.03955v5.pdf
-- **Paper (Newer variants, extended benchmarks):** https://arxiv.org/pdf/2401.03955.pdf
+- **Repository:** https://github.com/ibm-granite/granite-tsfm/tree/main/tsfm_public/models/tinytimemixer
+- **Paper:** https://arxiv.org/pdf/2401.03955.pdf
 
-### External Blogs on TTM
-- https://aihorizonforecast.substack.com/p/tiny-time-mixersttms-powerful-zerofew
-- https://medium.com/@david.proietti_17/predicting-venetian-lagoon-tide-levels-with-multivariate-time-series-modeling-8bafdf229588
 
+### Blogs and articles on TTM:
+- Refer to our [wiki](https://github.com/ibm-granite/granite-tsfm/wiki)
+
+ 
 ## Uses
 
 ```
@@ -183,6 +187,7 @@ The TTM models were trained on a collection of datasets from the Monash Time Ser
  - US Births: https://zenodo.org/records/4656049 
  - Wind Farms Production data: https://zenodo.org/records/4654858 
  - Wind Power: https://zenodo.org/records/4656032
+ - [to be updated]
 
 
 ## Citation [optional]
@@ -192,24 +197,17 @@ work
 **BibTeX:**
 
 ```
-@misc{ekambaram2024tiny,
- title={Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series}, 
+@inproceedings{ekambaram2024tinytimemixersttms,
+ title={Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series},
  author={Vijay Ekambaram and Arindam Jati and Pankaj Dayama and Sumanta Mukherjee and Nam H. Nguyen and Wesley M. Gifford and Chandra Reddy and Jayant Kalagnanam},
+ booktitle={Advances in Neural Information Processing Systems (NeurIPS 2024)},
  year={2024},
- eprint={2401.03955},
- archivePrefix={arXiv},
- primaryClass={cs.LG}
 }
 ```
 
-**APA:**
-
-Ekambaram, V., Jati, A., Dayama, P., Mukherjee, S., Nguyen, N. H., Gifford, W. M., … Kalagnanam, J. (2024). Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series. arXiv [Cs.LG]. Retrieved from http://arxiv.org/abs/2401.03955
-
-
 ## Model Card Authors
 
-Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Nam H. Nguyen, Wesley Gifford and Jayant Kalagnanam
+Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Wesley M. Gifford, Sumanta Mukherjee, Chandra Reddy and Jayant Kalagnanam
 
 
 ## IBM Public Repository Disclosure: