Divyasreepat commited on
Commit
72bd8da
1 Parent(s): dbea7f4

Update README.md with new model card content

Browse files
Files changed (1) hide show
  1. README.md +142 -13
README.md CHANGED
@@ -1,16 +1,145 @@
1
  ---
2
  library_name: keras-hub
3
  ---
4
- This is a [`DistilBert` model](https://keras.io/api/keras_hub/models/distil_bert) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
5
- Model config:
6
- * **name:** distil_bert_backbone
7
- * **trainable:** True
8
- * **vocabulary_size:** 30522
9
- * **num_layers:** 6
10
- * **num_heads:** 12
11
- * **hidden_dim:** 768
12
- * **intermediate_dim:** 3072
13
- * **dropout:** 0.1
14
- * **max_sequence_length:** 512
15
-
16
- This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  library_name: keras-hub
3
  ---
4
+ ### Model Overview
5
+ DistilBert is a set of language models published by HuggingFace. They are efficient, distilled version of BERT, and are intended for classification and embedding of text, not for text-generation. See the model card below for benchmarks, data sources, and intended use cases.
6
+
7
+ Weights and Keras model code are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
8
+
9
+ ## Links
10
+
11
+ * [DistilBert Quickstart Notebook](https://www.kaggle.com/code/matthewdwatson/distilbert-quickstart)
12
+ * [DistilBert API Documentation](https://keras.io/api/keras_hub/models/distil_bert/)
13
+ * [DistilBert Model Card](https://huggingface.co/distilbert/distilbert-base-uncased)
14
+ * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
15
+ * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
16
+
17
+ ## Installation
18
+
19
+ Keras and KerasHub can be installed with:
20
+
21
+ ```
22
+ pip install -U -q keras-hub
23
+ pip install -U -q keras>=3
24
+ ```
25
+
26
+ Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
27
+
28
+ ## Presets
29
+
30
+ The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
31
+
32
+ | Preset name | Parameters | Description |
33
+ |-----------------------------|------------|--------------------------------------------------------|
34
+ | distil_bert_base_en_uncased | 66.36M | 6-layer model where all input is lowercased. |
35
+ | distil_bert_base_en | 65.19M | 6-layer model where case is maintained. |
36
+ | distil_bert_base_multi | 134.73M | 6-layer multi-linguage model where case is maintained. |
37
+
38
+ ### Example Usage
39
+ ```python
40
+ import keras
41
+ import keras_hub
42
+ import numpy as np
43
+ ```
44
+
45
+ Raw string data.
46
+ ```python
47
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
48
+ labels = [0, 3]
49
+
50
+ # Use a shorter sequence length.
51
+ preprocessor = keras_hub.models.DistilBertPreprocessor.from_preset(
52
+ "distil_bert_base_en_uncased",
53
+ sequence_length=128,
54
+ )
55
+ # Pretrained classifier.
56
+ classifier = keras_hub.models.DistilBertClassifier.from_preset(
57
+ "distil_bert_base_en_uncased",
58
+ num_classes=4,
59
+ preprocessor=preprocessor,
60
+ )
61
+ classifier.fit(x=features, y=labels, batch_size=2)
62
+
63
+ # Re-compile (e.g., with a new learning rate)
64
+ classifier.compile(
65
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
66
+ optimizer=keras.optimizers.Adam(5e-5),
67
+ jit_compile=True,
68
+ )
69
+ # Access backbone programmatically (e.g., to change `trainable`).
70
+ classifier.backbone.trainable = False
71
+ # Fit again.
72
+ classifier.fit(x=features, y=labels, batch_size=2)
73
+ ```
74
+
75
+ Preprocessed integer data.
76
+ ```python
77
+ features = {
78
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
79
+ "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2)
80
+ }
81
+ labels = [0, 3]
82
+
83
+ # Pretrained classifier without preprocessing.
84
+ classifier = keras_hub.models.DistilBertClassifier.from_preset(
85
+ "distil_bert_base_en_uncased",
86
+ num_classes=4,
87
+ preprocessor=None,
88
+ )
89
+ classifier.fit(x=features, y=labels, batch_size=2)
90
+ ```
91
+
92
+ ## Example Usage with Hugging Face URI
93
+
94
+ ```python
95
+ import keras
96
+ import keras_hub
97
+ import numpy as np
98
+ ```
99
+
100
+ Raw string data.
101
+ ```python
102
+ features = ["The quick brown fox jumped.", "I forgot my homework."]
103
+ labels = [0, 3]
104
+
105
+ # Use a shorter sequence length.
106
+ preprocessor = keras_hub.models.DistilBertPreprocessor.from_preset(
107
+ "hf://keras/distil_bert_base_en_uncased",
108
+ sequence_length=128,
109
+ )
110
+ # Pretrained classifier.
111
+ classifier = keras_hub.models.DistilBertClassifier.from_preset(
112
+ "hf://keras/distil_bert_base_en_uncased",
113
+ num_classes=4,
114
+ preprocessor=preprocessor,
115
+ )
116
+ classifier.fit(x=features, y=labels, batch_size=2)
117
+
118
+ # Re-compile (e.g., with a new learning rate)
119
+ classifier.compile(
120
+ loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
121
+ optimizer=keras.optimizers.Adam(5e-5),
122
+ jit_compile=True,
123
+ )
124
+ # Access backbone programmatically (e.g., to change `trainable`).
125
+ classifier.backbone.trainable = False
126
+ # Fit again.
127
+ classifier.fit(x=features, y=labels, batch_size=2)
128
+ ```
129
+
130
+ Preprocessed integer data.
131
+ ```python
132
+ features = {
133
+ "token_ids": np.ones(shape=(2, 12), dtype="int32"),
134
+ "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2)
135
+ }
136
+ labels = [0, 3]
137
+
138
+ # Pretrained classifier without preprocessing.
139
+ classifier = keras_hub.models.DistilBertClassifier.from_preset(
140
+ "hf://keras/distil_bert_base_en_uncased",
141
+ num_classes=4,
142
+ preprocessor=None,
143
+ )
144
+ classifier.fit(x=features, y=labels, batch_size=2)
145
+ ```