TPU inference
#37
by
Arpit-Bansal
- opened
Does the provided inference code, work for TPU environment, if not?
I think it's better to include it.
If someone else has script, could you share?
Hi @Arpit-Bansal ,
Yes, the inference code works in a TPU environment. I tested it with the google/gemma-3-4b-it model, which suggests it should also be compatible with the google/gemma-3-27b-it model. Please refer to the attached gist file for details.
Thank you.