torch
transformers
datasets
accelerate>=0.21.0
pandas
fastparquet
s3fs
numpy
faiss-gpu
sentence_transformers
cohere
mixedbread_ai