ssmits commited on
Commit
bde78de
1 Parent(s): bb9232e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -0
README.md CHANGED
@@ -44,6 +44,37 @@ dtype: bfloat16
44
 
45
  ![Layer Similarity Plot](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/k9VKXgqUuUr0EjGZf7Ick.png)
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  ## Direct Use
48
  Research on large language models; as a foundation for further specialization and finetuning for specific usecases (e.g., summarization, text generation, chatbot, etc.)
49
 
 
44
 
45
  ![Layer Similarity Plot](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/k9VKXgqUuUr0EjGZf7Ick.png)
46
 
47
+ ```python
48
+ from transformers import AutoTokenizer, AutoModelForCausalLM
49
+ import transformers
50
+ import torch
51
+
52
+ model = "ssmits/Falcon2-5.5B-German"
53
+
54
+ tokenizer = AutoTokenizer.from_pretrained(model)
55
+ pipeline = transformers.pipeline(
56
+ "text-generation",
57
+ model=model,
58
+ tokenizer=tokenizer,
59
+ torch_dtype=torch.bfloat16,
60
+ )
61
+ sequences = pipeline(
62
+ "Can you explain the concepts of Quantum Computing?",
63
+ max_length=200,
64
+ do_sample=True,
65
+ top_k=10,
66
+ num_return_sequences=1,
67
+ eos_token_id=tokenizer.eos_token_id,
68
+ )
69
+ for seq in sequences:
70
+ print(f"Result: {seq['generated_text']}")
71
+
72
+ ```
73
+
74
+ 💥 **Falcon LLMs require PyTorch 2.0 for use with `transformers`!**
75
+
76
+ For fast inference with Falcon, check-out [Text Generation Inference](https://github.com/huggingface/text-generation-inference)! Read more in this [blogpost]((https://huggingface.co/blog/falcon).
77
+
78
  ## Direct Use
79
  Research on large language models; as a foundation for further specialization and finetuning for specific usecases (e.g., summarization, text generation, chatbot, etc.)
80