neuron-compile-jobs
Collection
3 items
•
Updated
This repository contains AWS Inferentia2 and neuronx compatible checkpoints for Codestral-22B-v0.1. You can find detailed information about the base model on its Model Card.
This model has been exported to the neuron format using specific input_shapes and compiler parameters detailed in the paragraphs below.
It has been compiled to run on an inf2.24xlarge instance on AWS. Note that while the inf2.24xlarge has 12 cores, this compilation uses 12.
- SEQUENCE_LENGTH = 4096
- BATCH_SIZE = 4
- NUM_CORES = 12
- PRECISION = "bf16"
license: apache-2.0
- Downloads last month
- 9
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for nithiyn/codestral-neuron
Base model
mistralai/Codestral-22B-v0.1