Update README.md

README.md

@@ -6,4 +6,58 @@ language:
 pipeline_tag: visual-question-answering
 tags:
 - text2text-generation
----
+base_model: google/deplot
+---
+# **Ko-Deplot**
+
+Ko-Deplot is a korean Visual-QA model based on the Google's Pix2Struct architecture. It was fine-tuned from [Deplot](https://huggingface.co/google/deplot), using korean chart image-text pairs.
+
+Ko-Deplot은 Google의 Pix2Struct 구조를 기반으로 한 한국어 Visual-QA 모델입니다. [Deplot](https://huggingface.co/google/deplot) 모델을 한국어 차트 이미지-텍스트 쌍 데이터셋을 이용하여 파인튜닝하였습니다.
+
+- **Developed by:** [NUUA](https://www.nuua.ai/en/): Yohan Kim, Jiyou Shin, Robin Lee
+- **Model type:** Visual Question Answering
+- **License:** apache-2.0
+- **Finetuned from model:** [google/deplot](https://huggingface.co/google/deplot)
+
+# **Model Usage**
+You can run a prediction by querying an input image together with a question as follows:
+
+```python
+from transformers import Pix2StructProcessor, Pix2StructForConditionalGeneration
+import requests
+from PIL import Image
+
+processor = Pix2StructProcessor.from_pretrained('nuua/Ko-Deplot')
+model = Pix2StructForConditionalGeneration.from_pretrained('nuua/Ko-Deplot')
+
+IMAGE_PATH = "LOCAL_PATH_TO_IMAGE"
+image = Image.open(IMAGE_PATH)
+
+inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
+predictions = model.generate(**inputs, max_new_tokens=512)
+print(processor.decode(predictions[0], skip_special_tokens=True))
+```
+
+# **Training Details**
+
+## Training Data
+
+Synthetic chart data from three libraries were used:
+
+- [GenPlot](https://github.com/brendanartley/genplot)
+- [Chart.js](https://github.com/chartjs/Chart.js)
+- [Plotly](https://github.com/plotly/plotly.py)
+
+## Training Procedure 
+
+The model was first exposed to a short warmup stage, following its [original paper](https://arxiv.org/pdf/2210.03347.pdf). It was then trained using the chart data for 50,000 steps.
+
+# **Technical Specifications**
+
+## Hardware
+
+Ko-Deplot was trained by using A100 80G. 
+
+# **Contact**
+
+Any questions and suggestions, please use the discussion tab. If you want to contact us directly, email [email protected].