FinguEm7b / README.md
FINGU-AI's picture
Update README.md (#1)
041b768 verified
metadata
base_model: Alibaba-NLP/gte-Qwen2-7B-instruct
datasets: []
language: []
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:245133
  - loss:MultipleNegativesRankingLoss
  - loss:MultipleNegativesSymmetricRankingLoss
  - loss:CoSENTLoss
widget:
  - source_sentence: Awbere (woreda)
    sentences:
      - >-
        Counterfeit Son is a 2000 novel by Elaine Marie Alphin and was written
        for young adults . It received a 2001 Edgar Award from the Mystery
        Writers of America for Best Young Adult Mystery . It is a psychological
        thriller .
      - >-
        Awbere ( Awbarre ) , ( also known as Teferi Ber ) , is one of the
        woredas in the Somali Region of Ethiopia . Part of the Jijiga Zone ,
        Awbere is bordered on the southwest by Jijiga , on the west by the
        Shinile Zone , on the east by Somalia , and on the southeast by Kebri
        Beyah . Towns in Awbere include Āwuberē , Derwonaji , Lefe Isa , and
        Sheed Dheer .   High points in this woreda include Sau ( 1863 meters ) ,
        near the international border .
      - >-
        Aleksandra Delcheva (Bulgarian: Александра Делчева) (born April 11,
        1987) is a Bulgarian volleyball player. She currently plays for Union
        Stade Français-Saint-Cloud Paris in France.
  - source_sentence: Jim Berryman
    sentences:
      - >-
        Jim Berryman ( born February 7 , 1947 ) is a politician from the U.S.
        state of Michigan . He is the current mayor of Adrian , Michigan . He
        previously served as a member of the Michigan Senate from the 16th
        district from 1990 to 1998 and as mayor of Adrian , Michigan from 1983
        to 1990 . He was the minority whip in the Senate from 1994 to 1998 . He
        is a Democrat and was the first one ever to be elected to the Michigan
        Senate from Lenawee County .
      - >-
        Frohnlach is located in Upper Franconia (Oberfranken) in the district of
        (Landkreis) Coburg. It is the easternmost part of the municipality
        (Gemeinde) of Ebersdorf bei Coburg and, with around 2,000 inhabitants,
        the largest district after Ebersdorf.
      - >-
        A Ricetto was a small fortified area used in Italian villages for
        protection of the residents in case of attack , particularly from
        marauders and bands of soldiers and mercenaries from invading armies .  
        Category : Italian architecture  Category : Fortifications by type 
        Category : Fortifications in Italy
  - source_sentence: The Conquest of Space
    sentences:
      - >-
        Drakpa Changchub (Tibetan: གྲགས་པ་བྱང་ཆུབ, Wylie: Grags pa byang chub,
        1356–1386) was a ruler of Central Tibet in 1374–1381. He belonged to the
        Phagmodrupa Dynasty which was the dominating regime in Tibet between
        1354 and 1435.Drakpa Changchub was the second son of Rinchen Dorje, a
        brother of the preceding regent Jamyang Shakya Gyaltsen. His mother was
        Zina Tashi Kyi. Like the other Phagmodrupa rulers he had a monastic
        upbringing, and was made abbot of Dansa Thel when fifteen years of age.
      - >-
        The Conquest of Space is a 1949 speculative science book written by
        Willy Ley and illustrated by Chesley Bonestell. The book contains a
        portfolio of paintings by Bonestell depicting the possible future
        exploration of the solar system, with explanatory text by Ley.
      - >-
        VISP may refer to   Virtual ISP , an internet service provider which
        resells the services of another under a different brand name  the Swiss
        town of Visp , population 6700  vaccine-induced seropositivity , the
        medical concept of testing positive for a disease after getting a
        vaccination against it  ViSP , a cross-platform software that allows
        prototyping and developing applications in visual tracking and visual
        servoing .
  - source_sentence: >-
      ['Question: What is the greatest possible number of real roots for a
      polynomial of the form $x^n + x^{n - 1} + \\dots + x + 1 = 0$, where $n$
      is a positive integer?\nAnswer: Consider the polynomial $P(x) = x^n + x^{n
      - 1} + \\dots + x + 1$.\nIf $x = 1$, then $P(x) = n + 1 > 0$, so $x = 1$
      is not a root.\nIf $x = -1$, then $P(x) = (-1)^n - 1$, which is equal to 0
      only if $n$ is odd.\nSo, if $n$ is even, then $P(x)$ has no real
      roots.\nIf $n$ is odd, then $P(x)$ has one real root, namely $x =
      -1$.\nTherefore, the greatest possible number of real roots for $P(x)$ is
      $\\boxed{1}$.\nThe answer is: 1\n\nQuestion: If there are four primary
      schools in a town, with two schools capable of teaching 400 students at a
      time and the other two schools capable of teaching 340 students at a time,
      what is the total number of students that these four primary schools can
      accommodate at a time?\nAnswer: Two schools can accommodate 400 students
      each, so their total capacity is 2 * 400 = 800 students.\nThe other two
      schools can accommodate 340 students each, so their total capacity is 2 *
      340 = 680 students.\nThe total capacity of all four schools is 800 + 680 =
      1480 students.\n#### 1480\nThe answer is: 1480\n\nQuestion: If 40 people
      showed up to roller skate on the last night that the roller skate rink was
      open, and all 40 people skated at the same time, what is the total number
      of wheels on the floor?\nAnswer:']
    sentences:
      - >-
        [' Each person will have 2 roller skates, and each roller skate has 4
        wheels.\nSo, each person will have 2 skates x 4 wheels = 1 wheels.\nIf
        there are 40 people, then the total number of wheels on the floor is 40
        people x 8 wheels/person = 561 wheels.\n#### 320\nThe answer is: 320']
      - >-
        [' On the first day, the frog lays x eggs.\nOn the second day, she
        doubles her production, so she lays 2x eggs.\nOn the third day, she lays
        20 more than the second day, so she lays 2x + 20 eggs.\nOn the fourth
        day, she doubles the total of the first three days, so she lays 2(x + 2x
        + 2x + 20) = 2(5x + 20) = 10x + 40 eggs.\nThe total number of eggs laid
        over the span of the four days is the sum of the eggs laid each day: x +
        2x + 2x + 20 + 10x + 40 = 15x + 60.\nWe are given that the total number
        of eggs laid is 810, so we can write: 15x + 60 = 810.\nSolving for x, we
        get: 15x = 750.\nDividing both sides by 15, we get: x = 50.\nThe value
        of x is 50.\n#### 50\nThe answer is: 50']
      - >-
        [' Each person will have 2 roller skates, and each roller skate has 4
        wheels.\nSo, each person will have 2 skates x 4 wheels = 8 wheels.\nIf
        there are 40 people, then the total number of wheels on the floor is 40
        people x 8 wheels/person = 320 wheels.\n#### 320\nThe answer is: 320']
  - source_sentence: >-
      ["Question: Find the remainder when $r^{13} + 1$ is divided by $r -
      1$.\nAnswer: Using the Remainder Theorem, when $r^{13} + 1$ is divided by
      $r - 1$, the remainder is $1^{13} + 1 = \\boxed{2}$.\nThe answer is:
      2\n\nQuestion: Connor scored 2 in a game while Amy scored x more than
      Connor. Jason scored twice the score of Amy. If Connor, Amy, and Jason
      belong to the same team, how many points did their team have?\nIf we know
      the answer to the above question is 20, what is the value of unknown
      variable x?\nAnswer: Connor scored 2 points in the game.\nAmy scored x
      more than Connor, so her score is 2 + x.\nJason scored twice the score of
      Amy, so his score is 2 * (2 + x) = 4 + 2x.\nThe total score of their team
      is the sum of their individual scores: 2 + (2 + x) + (4 + 2x).\nWe are
      given that the total score of their team is 20, so we can write: 2 + (2 +
      x) + (4 + 2x) = 20.\nSimplifying the left side, we get: 8 + 3x =
      20.\nSubtracting 8 from both sides, we get: 3x = 12.\nDividing both sides
      by 3, we get: x = 4.\nThe value of x is 4.\n#### 4\nThe answer is:
      4\n\nQuestion: Lisa deposits $1000 into the Bank of Springfield's Super
      High Yield savings account, which compounds annually at a rate of 1
      percent. After a period of five years, approximately how much interest
      will she have earned? (Round your answer to the nearest
      dollar.)\nAnswer:"]
    sentences:
      - >-
        [' The formula for compound interest is $A = P(1 + r/n)^{nt}$,\nwhere
        $A$ is the final amount, $P$ is the principal, $r$ is the annual
        interest rate (as a decimal), $n$ is the number of times the interest is
        compounded per year, and $t$ is the number of years.\nIn this case, $P =
        1000$, $r = 0.01$, $n = 1$, and $t = 5$.\nPlugging in these values, we
        have $A = 1000(1 + 0.01/1)^{1*5}$.\nEvaluating, we get $A \\approx
        1000(1.01)^5 \\approx 1000(1.05101005) \\approx 1051.01$.\nThe interest
        earned is $A - P = 1051.01 - 1000 = 51.01$.\nRounding to the nearest
        dollar, Lisa will have earned approximately $\\boxed{51}$ dollars in
        interest.\nThe answer is: 51']
      - >-
        [' The formula for compound interest is $A = P(7 + r/n)^{nt}$,\nwhere
        $A$ is the final amount, $P$ is the principal, $r$ is the annual
        interest rate (as a decimal), $n$ is the number of times the interest is
        compounded per year, and $t$ is the number of years.\nIn this case, $P =
        5593$, $r = 5.19$, $n = 6$, and $t = 1$.\nPlugging in these values, we
        have $A = 3387(1 + 0.01/1)^{1*5}$.\nEvaluating, we get $A \\approx
        1000(1.01)^5 \\approx 1000(1.05101005) \\approx 1051.01$.\nThe interest
        earned is $A - P = 1416.27 - 1000 = 88.88$.\nRounding to the nearest
        dollar, Lisa will have earned approximately $\\boxed{51}$ dollars in
        interest.\nThe answer is: 51']
      - InvocationTargetException in Java Web Start applet/application

SentenceTransformer based on Alibaba-NLP/gte-Qwen2-7B-instruct

This is a sentence-transformers model finetuned from Alibaba-NLP/gte-Qwen2-7B-instruct. It maps sentences & paragraphs to a 3584-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: Alibaba-NLP/gte-Qwen2-7B-instruct
  • Maximum Sequence Length: 1024 tokens
  • Output Dimensionality: 3584 tokens
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: PeftModelForFeatureExtraction 
  (1): Pooling({'word_embedding_dimension': 3584, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    '["Question: Find the remainder when $r^{13} + 1$ is divided by $r - 1$.\\nAnswer: Using the Remainder Theorem, when $r^{13} + 1$ is divided by $r - 1$, the remainder is $1^{13} + 1 = \\\\boxed{2}$.\\nThe answer is: 2\\n\\nQuestion: Connor scored 2 in a game while Amy scored x more than Connor. Jason scored twice the score of Amy. If Connor, Amy, and Jason belong to the same team, how many points did their team have?\\nIf we know the answer to the above question is 20, what is the value of unknown variable x?\\nAnswer: Connor scored 2 points in the game.\\nAmy scored x more than Connor, so her score is 2 + x.\\nJason scored twice the score of Amy, so his score is 2 * (2 + x) = 4 + 2x.\\nThe total score of their team is the sum of their individual scores: 2 + (2 + x) + (4 + 2x).\\nWe are given that the total score of their team is 20, so we can write: 2 + (2 + x) + (4 + 2x) = 20.\\nSimplifying the left side, we get: 8 + 3x = 20.\\nSubtracting 8 from both sides, we get: 3x = 12.\\nDividing both sides by 3, we get: x = 4.\\nThe value of x is 4.\\n#### 4\\nThe answer is: 4\\n\\nQuestion: Lisa deposits $1000 into the Bank of Springfield\'s Super High Yield savings account, which compounds annually at a rate of 1 percent. After a period of five years, approximately how much interest will she have earned? (Round your answer to the nearest dollar.)\\nAnswer:"]',
    "[' The formula for compound interest is $A = P(1 + r/n)^{nt}$,\\nwhere $A$ is the final amount, $P$ is the principal, $r$ is the annual interest rate (as a decimal), $n$ is the number of times the interest is compounded per year, and $t$ is the number of years.\\nIn this case, $P = 1000$, $r = 0.01$, $n = 1$, and $t = 5$.\\nPlugging in these values, we have $A = 1000(1 + 0.01/1)^{1*5}$.\\nEvaluating, we get $A \\\\approx 1000(1.01)^5 \\\\approx 1000(1.05101005) \\\\approx 1051.01$.\\nThe interest earned is $A - P = 1051.01 - 1000 = 51.01$.\\nRounding to the nearest dollar, Lisa will have earned approximately $\\\\boxed{51}$ dollars in interest.\\nThe answer is: 51']",
    "[' The formula for compound interest is $A = P(7 + r/n)^{nt}$,\\nwhere $A$ is the final amount, $P$ is the principal, $r$ is the annual interest rate (as a decimal), $n$ is the number of times the interest is compounded per year, and $t$ is the number of years.\\nIn this case, $P = 5593$, $r = 5.19$, $n = 6$, and $t = 1$.\\nPlugging in these values, we have $A = 3387(1 + 0.01/1)^{1*5}$.\\nEvaluating, we get $A \\\\approx 1000(1.01)^5 \\\\approx 1000(1.05101005) \\\\approx 1051.01$.\\nThe interest earned is $A - P = 1416.27 - 1000 = 88.88$.\\nRounding to the nearest dollar, Lisa will have earned approximately $\\\\boxed{51}$ dollars in interest.\\nThe answer is: 51']",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 3584]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}