English
pszemraj commited on
Commit
380ff73
1 Parent(s): cd0ea4a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -27,7 +27,7 @@ print(f"Tokens:\n\t{output.input_ids}")
27
 
28
  ## Notes
29
 
30
- - the default tokenizer (on branch `main`) has a vocab size of 32128
31
 
32
 
33
 
@@ -61,7 +61,7 @@ print(f"Tokens:\n\t{output.input_ids}")
61
 
62
  This should result in the following (_Feb '24 version_):
63
 
64
- ```python
65
  >>> print(f"Test string: {test_string}")
66
  Test string: When dealing with Varroa destructor mites, it's crucial to administer the right acaricides during the late autumn months, but only after ensuring that the worker bee population is free from pesticide contamination.
67
  >>>
 
27
 
28
  ## Notes
29
 
30
+ - the default tokenizer (on branch `main`) has a vocab size of 32100
31
 
32
 
33
 
 
61
 
62
  This should result in the following (_Feb '24 version_):
63
 
64
+ ```
65
  >>> print(f"Test string: {test_string}")
66
  Test string: When dealing with Varroa destructor mites, it's crucial to administer the right acaricides during the late autumn months, but only after ensuring that the worker bee population is free from pesticide contamination.
67
  >>>