JRosenkranz commited on
Commit
18a0c21
·
verified ·
1 Parent(s): d5dc66d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -9,9 +9,16 @@ pinned: false
9
 
10
  # Foundation Model Stack
11
 
12
- Foundation Model Stack (fms) is a collection of components for development, inference, training, and tuning of foundation models leveraging PyTorch native components.
13
- For inference optimizations we aim to support PyTorch compile, accelerated transformers, and tensor parallelism.
14
- At training time we aim to support FSDP, accelerated transformers, and PyTorch compile.
 
 
 
 
 
 
 
15
 
16
  ## Repositories
17
 
 
9
 
10
  # Foundation Model Stack
11
 
12
+ Foundation Model Stack (fms) is a collection of components developed out of IBM Research used for development, inference, training, and tuning of foundation models leveraging PyTorch native components.
13
+
14
+ ## Optimizations
15
+
16
+ In FMS, we aim to bring the latest optimizations for pre-training/inference/fine-tuning to all of our models. A few of these optimizations include, but are not limited to:
17
+
18
+ - fully compileable models with no graph breaks
19
+ - full tensor-parallel support for all applicable modules developed in fms
20
+ - training scripts leveraging FSDP
21
+ - state of the art light-weight speculators for improving inference performance
22
 
23
  ## Repositories
24