Hugging Face's logo

microsoft
/
phi-2

Text Generation
Transformers
Safetensors
English
phi
nlp
code
text-generation-inference
Inference Endpoints
Model card Files Community
131

return past hidden states when `output_hidden_states` provided

#59
by noahtren - opened
base: refs/heads/main
←
from: refs/pr/59
Discussion Files changed
+2
-1
noahtren

The model should be able to return past hidden states, as expected in the forward() function for HuggingFace models.

Added this because I needed it, but additional support for output_attentions param could be added.

return past hidden states when `output_hidden_states` providede282cdc0
susnato
•
edited Jan 2, 2024

Hi @noahtren , until this PR is merged you can load the model from susnato/phi-2. 

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("susnato/phi-2")
tokenizer = AutoTokenizer.from_pretrained("susnato/phi-2")

This way the phi model on the transformers library will be loaded which has support for output_attentions.

(BTW make sure you have the latest transformers version installed pip install -U transformers)

gugarosa
 Microsoft org

@noahtren !

Please proceed with what @susnato has mentioned. We will soon integrate everything and default the native transformers implementation to this repository.

gugarosa changed pull request status to closed

Sign up or log in to comment