Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ tags:
|
|
12 |
|
13 |
# im2latex
|
14 |
|
15 |
-
This model is a VisionEncoderDecoderModel fine-tuned on a dataset for generating LaTeX formulas from images.
|
16 |
|
17 |
## Model Details
|
18 |
|
@@ -39,8 +39,8 @@ test_ds = val_test_split["test"]
|
|
39 |
## Evaluation Metrics
|
40 |
|
41 |
The model was evaluated on a test set with the following results:
|
42 |
-
- **Test Loss**: 0.
|
43 |
-
- **Test BLEU Score**: 0.
|
44 |
|
45 |
## Usage
|
46 |
|
@@ -51,16 +51,16 @@ from transformers import VisionEncoderDecoderModel, AutoTokenizer, AutoFeatureEx
|
|
51 |
import torch
|
52 |
from PIL import Image
|
53 |
|
54 |
-
#
|
55 |
-
model = VisionEncoderDecoderModel.from_pretrained("
|
56 |
-
tokenizer = AutoTokenizer.from_pretrained("
|
57 |
-
feature_extractor = AutoFeatureExtractor.from_pretrained("
|
58 |
|
59 |
-
#
|
60 |
image = Image.open("path/to/your/image.png")
|
61 |
pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
|
62 |
|
63 |
-
#
|
64 |
generated_ids = model.generate(pixel_values)
|
65 |
generated_texts = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
|
66 |
|
|
|
12 |
|
13 |
# im2latex
|
14 |
|
15 |
+
This model is a base VisionEncoderDecoderModel fine-tuned on a dataset for generating LaTeX formulas from images.
|
16 |
|
17 |
## Model Details
|
18 |
|
|
|
39 |
## Evaluation Metrics
|
40 |
|
41 |
The model was evaluated on a test set with the following results:
|
42 |
+
- **Test Loss**: 0.10
|
43 |
+
- **Test BLEU Score**: 0.67
|
44 |
|
45 |
## Usage
|
46 |
|
|
|
51 |
import torch
|
52 |
from PIL import Image
|
53 |
|
54 |
+
# load model, tokenizer, and feature extractor
|
55 |
+
model = VisionEncoderDecoderModel.from_pretrained("DGurgurov/im2latex")
|
56 |
+
tokenizer = AutoTokenizer.from_pretrained("DGurgurov/im2latex")
|
57 |
+
feature_extractor = AutoFeatureExtractor.from_pretrained("microsoft/swin-base-patch4-window7-224-in22k") # using the original feature extractor for now
|
58 |
|
59 |
+
# prepare an image
|
60 |
image = Image.open("path/to/your/image.png")
|
61 |
pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
|
62 |
|
63 |
+
# generate LaTeX formula
|
64 |
generated_ids = model.generate(pixel_values)
|
65 |
generated_texts = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
|
66 |
|