DGurgurov commited on
Commit
9a9773d
1 Parent(s): 23c0bfb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -12,7 +12,7 @@ tags:
12
 
13
  # im2latex
14
 
15
- This model is a VisionEncoderDecoderModel fine-tuned on a dataset for generating LaTeX formulas from images.
16
 
17
  ## Model Details
18
 
@@ -39,8 +39,8 @@ test_ds = val_test_split["test"]
39
  ## Evaluation Metrics
40
 
41
  The model was evaluated on a test set with the following results:
42
- - **Test Loss**: 0.10473818009443304
43
- - **Test BLEU Score**: 0.6661951245257148
44
 
45
  ## Usage
46
 
@@ -51,16 +51,16 @@ from transformers import VisionEncoderDecoderModel, AutoTokenizer, AutoFeatureEx
51
  import torch
52
  from PIL import Image
53
 
54
- # Load model, tokenizer, and feature extractor
55
- model = VisionEncoderDecoderModel.from_pretrained("your-username/your-model-name")
56
- tokenizer = AutoTokenizer.from_pretrained("your-username/your-model-name")
57
- feature_extractor = AutoFeatureExtractor.from_pretrained("your-username/your-model-name")
58
 
59
- # Prepare an image
60
  image = Image.open("path/to/your/image.png")
61
  pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
62
 
63
- # Generate LaTeX formula
64
  generated_ids = model.generate(pixel_values)
65
  generated_texts = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
66
 
 
12
 
13
  # im2latex
14
 
15
+ This model is a base VisionEncoderDecoderModel fine-tuned on a dataset for generating LaTeX formulas from images.
16
 
17
  ## Model Details
18
 
 
39
  ## Evaluation Metrics
40
 
41
  The model was evaluated on a test set with the following results:
42
+ - **Test Loss**: 0.10
43
+ - **Test BLEU Score**: 0.67
44
 
45
  ## Usage
46
 
 
51
  import torch
52
  from PIL import Image
53
 
54
+ # load model, tokenizer, and feature extractor
55
+ model = VisionEncoderDecoderModel.from_pretrained("DGurgurov/im2latex")
56
+ tokenizer = AutoTokenizer.from_pretrained("DGurgurov/im2latex")
57
+ feature_extractor = AutoFeatureExtractor.from_pretrained("microsoft/swin-base-patch4-window7-224-in22k") # using the original feature extractor for now
58
 
59
+ # prepare an image
60
  image = Image.open("path/to/your/image.png")
61
  pixel_values = feature_extractor(images=image, return_tensors="pt").pixel_values
62
 
63
+ # generate LaTeX formula
64
  generated_ids = model.generate(pixel_values)
65
  generated_texts = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)
66