chenwuml commited on
Commit
7e723c3
1 Parent(s): 4b2f55b

Add readme instructions on SageMaker

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -28,11 +28,15 @@ cd falconlite-dev/script
28
  ./start_falconlite.sh
29
  ```
30
  ### Perform inference
31
- ```
32
  # after FalconLite has been completely started
33
  pip install -r requirements-client.txt
34
  python falconlite_client.py
35
  ```
 
 
 
 
36
  **Important** - When using FalconLite for inference for the first time, it may require a brief 'warm-up' period that can take 10s of seconds. However, subsequent inferences should be faster and return results in a more timely manner. This warm-up period is normal and should not affect the overall performance of the system once the initialisation period has been completed.
37
 
38
  ## Evalution Result ##
 
28
  ./start_falconlite.sh
29
  ```
30
  ### Perform inference
31
+ ```bash
32
  # after FalconLite has been completely started
33
  pip install -r requirements-client.txt
34
  python falconlite_client.py
35
  ```
36
+
37
+ ### *New!* Amazon SageMaker Deployment ###
38
+ To deploy FalconLite on SageMaker endpoint, please follow [this notebook](https://github.com/awslabs/extending-the-context-length-of-open-source-llms/blob/main/custom-tgi-ecr/deploy.ipynb).
39
+
40
  **Important** - When using FalconLite for inference for the first time, it may require a brief 'warm-up' period that can take 10s of seconds. However, subsequent inferences should be faster and return results in a more timely manner. This warm-up period is normal and should not affect the overall performance of the system once the initialisation period has been completed.
41
 
42
  ## Evalution Result ##