Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,557 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:305
+- loss:MultipleNegativesRankingLoss
+base_model: nomic-ai/modernbert-embed-base
+widget:
+- source_sentence: According to the text, when is an approach permitted?
+  sentences:
+  - "25.1.2\tAdjudicate on the Rules of the game;\n25.1.3\tImpose any sanction necessary\
+    \ to control the match;\n25.1.4\tAward Tries and record the progressive score;\n\
+    25.1.5\tMaintain a count of Touches during each Possession;\n25.1.6\tAward Penalties\
+    \ for Infringements against the Rules; and\n25.1.7\tReport to the relevant competition\
+    \ administration any Sin Bins, \nDismissals or injuries to any participant sustained\
+    \ during a Match.25.2\tOnly Team captains are permitted to seek clarification\
+    \ of a decision directly \nfrom the Referee.An approach may only be made during\
+    \ a break in play or at \nthe discretion of the Referee."
+  - "11.2\tA player in Possession  may not intentionally kick, pass, flick, knock,\
+    \ throw, \nhand-off or otherwise propel the ball in a Forward direction over an\
+    \ opposition \nplayer and regain Possession.Ruling = A Penalty will be awarded\
+    \ to the Defending Team at the Mark where the ball was \npropelled Forward.12 \
+    \ Ball Touched in Flight  \n  \n12.1\tIf a player from the Defending Team deliberately\
+    \ makes contact with the ball in \nflight and the ball goes to ground, the Attacking\
+    \ Team retains the ball and the \nTouch Count restarts as zero (0) Touch.12.2\t\
+    If a player from the Defending Team deliberately makes contact with the ball \n\
+    in flight and the ball is retrieved by an attacking player, without touching the\
+    \ \nground, play continues and the next Touch is zero (0) Touch."
+  - "Ruling = A Penalty to the Attacking Team at the point of the Infringement or\
+    \ on the seven (7) \nmetre line directly Forward of the Infringement.15.4\tWhen\
+    \ a Rollball occurs within Defending Team’s Seven Metre Zone or a Penalty \nTap\
+    \ within ten (10) metres of the Defending Team’s Try Line, all players from the\
+    \ \nDefending Team must have both feet on or behind their Try Line and no other\
+    \ \npart of the body in contact with the ground Forward of their Try Line.Ruling\
+    \ = A Penalty to the Attacking Team at the seven (7) metre line directly Forward\
+    \ of the \npoint of the Infringement.15.5\tAfter effecting the Touch, the defending\
+    \ player must retire the required seven \n(7) metres or to the Defending Try Line\
+    \ as indicated by the Referee without \ninterfering with the Attacking Team.Ruling\
+    \ = A Penalty to the Attacking Team ten (10) metres Forward of the Infringement\
+    \ or if \non the Defensive Try Line, on the seven (7) metre line."
+- source_sentence: If a player intentionally propels the ball forward over an opponent
+    and regains possession, what is the ruling?
+  sentences:
+  - "24.5\tFor the avoidance of doubt for clauses 24.3 and 24.4 the non-offending\
+    \ Team \nwill retain a numerical advantage on the Field of Play during the Drop-Off.25 \
+    \ Match Officials  \n25.1\tThe Referee is the sole judge on all match related\
+    \ matters inside the Perimeter \nfor the Duration of a match, has jurisdiction\
+    \ over all players, coaches and \nofficials and is required to:\n25.1.1\tInspect\
+    \ the Field of Play, Line Markings and Markers prior to the \ncommencement of\
+    \ the Match to ensure the safety of all participants.25.1.2\tAdjudicate on the\
+    \ Rules of the game;\n25.1.3\tImpose any sanction necessary to control the match;\n\
+    25.1.4\tAward Tries and record the progressive score;\n25.1.5\tMaintain a count\
+    \ of Touches during each Possession;\n25.1.6\tAward Penalties for Infringements\
+    \ against the Rules; and\n25.1.7\tReport to the relevant competition administration\
+    \ any Sin Bins, \nDismissals or injuries to any participant sustained during a\
+    \ Match."
+  - "See Appendix 1.Forced Interchange\nWhen a player is required to undertake a compulsory\
+    \ Interchange for \nan Infringement ruled more serious than a Penalty but less\
+    \ serious \nthan a Permanent Interchange, Sin Bin or Dismissal.Forward\nA position\
+    \ or direction towards the Dead Ball Line beyond the Team’s \nAttacking Try Line.Full\
+    \ Time\nThe expiration of the second period of time allowed for play.Half\nThe\
+    \ player who takes Possession following a Rollball.Half Time\nThe break in play\
+    \ between the two halves of a match.Imminent\nAbout to occur, it is almost certain\
+    \ to occur.Infringement\nThe action of a player contrary to the Rules of the game.In-Goal\
+    \ Area\nThe area in the Field of Play bounded by the Sidelines, the Try Lines\
+    \ \nand the Dead Ball Lines.There are two (2), one (1) at each end of the \nField\
+    \ of Play.See Appendix 1.Interchange\nThe act of an on-field player leaving the\
+    \ Field of Play to be replaced \nby an off-field player entering the Field of\
+    \ Play."
+  - "Ruling = A Change of Possession occurs at the point of the Touch, or if In-Goal\
+    \ the nearest \npoint on the seven (7) metre line.10.8\tIf a Touch is made in\
+    \ the In-Goal Area before the ball is grounded, the player in \nPossession is\
+    \ to perform a Rollball seven (7) metres from the Team’s Attacking \nTry Line,\
+    \ provided it is not the sixth  Touch and the player is not Half.10.9\tIf a player\
+    \ in Possession is Touched while on or behind their Defending Try Line, \nthe\
+    \ Touch counts and once the Referee sets the Mark seven (7) metres directly \n\
+    Forward of the contact point from the Defending Team’s Try Line, a Rollball is\
+    \ \nperformed.10.10\tIf a player in Possession intentionally makes a Touch on\
+    \ an Offside defender \nwho is making every effort to retire and remain out of\
+    \ play, the Touch counts."
+- source_sentence: How large is the Ruck/Rollball Area?
+  sentences:
+  - "13.5\tA player may only perform a Rollball at the Mark under the following \n\
+    circumstances:\n13.5.1\twhen a Touch has been made; or\n13.5.2\twhen Possession\
+    \ changes following the sixth Touch; or\n13.5.3\twhen Possession changes due to\
+    \ the ball being dropped or passed and \ngoes to the ground; or\n13.5.4\twhen\
+    \ Possession changes due to an Infringement by an attacking player \nat a Penalty,\
+    \ a Tap or a Rollball; or\nFIT Playing Rules - 5th Edition\nCOPYRIGHT © Touch\
+    \ Football Australia 2020\n11\n13.5.5\twhen Possession changes after the Half\
+    \ is Touched or when the Half \nplaces the ball on or over the Try Line; or\n\
+    13.5.6\tin replacement of a Penalty Tap; or\n13.5.7\twhen so directed by the Referee.13.6\t\
+    A player is to perform a Rollball seven (7) metres in-field under the following\
+    \ \ncircumstances:\n13.6.1\twhen a Change of Possession takes place due to a player\
+    \ in Possession \nmaking contact with the Sideline or any ground outside the Field\
+    \ of Play, \nprior to a Touch being made; or\n13.6.2\twhen the ball not in Possession\
+    \ of a player makes contact with the \nSideline or any ground outside the Field\
+    \ of Play."
+  - "13.5\tA player may only perform a Rollball at the Mark under the following \n\
+    circumstances:\n13.5.1\twhen a Touch has been made; or\n13.5.2\twhen Possession\
+    \ changes following the sixth Touch; or\n13.5.3\twhen Possession changes due to\
+    \ the ball being dropped or passed and \ngoes to the ground; or\n13.5.4\twhen\
+    \ Possession changes due to an Infringement by an attacking player \nat a Penalty,\
+    \ a Tap or a Rollball; or\nFIT Playing Rules - 5th Edition\nCOPYRIGHT © Touch\
+    \ Football Australia 2020\n11\n13.5.5\twhen Possession changes after the Half\
+    \ is Touched or when the Half \nplaces the ball on or over the Try Line; or\n\
+    13.5.6\tin replacement of a Penalty Tap; or\n13.5.7\twhen so directed by the Referee.13.6\t\
+    A player is to perform a Rollball seven (7) metres in-field under the following\
+    \ \ncircumstances:\n13.6.1\twhen a Change of Possession takes place due to a player\
+    \ in Possession \nmaking contact with the Sideline or any ground outside the Field\
+    \ of Play, \nprior to a Touch being made; or\n13.6.2\twhen the ball not in Possession\
+    \ of a player makes contact with the \nSideline or any ground outside the Field\
+    \ of Play."
+  - "See Appendix 1.Forced Interchange\nWhen a player is required to undertake a compulsory\
+    \ Interchange for \nan Infringement ruled more serious than a Penalty but less\
+    \ serious \nthan a Permanent Interchange, Sin Bin or Dismissal.Forward\nA position\
+    \ or direction towards the Dead Ball Line beyond the Team’s \nAttacking Try Line.Full\
+    \ Time\nThe expiration of the second period of time allowed for play.Half\nThe\
+    \ player who takes Possession following a Rollball.Half Time\nThe break in play\
+    \ between the two halves of a match.Imminent\nAbout to occur, it is almost certain\
+    \ to occur.Infringement\nThe action of a player contrary to the Rules of the game.In-Goal\
+    \ Area\nThe area in the Field of Play bounded by the Sidelines, the Try Lines\
+    \ \nand the Dead Ball Lines.There are two (2), one (1) at each end of the \nField\
+    \ of Play.See Appendix 1.Interchange\nThe act of an on-field player leaving the\
+    \ Field of Play to be replaced \nby an off-field player entering the Field of\
+    \ Play."
+- source_sentence: When may players interchange following a try in Touch Rugby?
+  sentences:
+  - "5.2\tA Team must have a minimum of four (4) players on the field for a match\
+    \ to \ncommence or continue, except during a Drop-Off.5.3\tWhere the number of\
+    \ players on the field from one Team falls below four (4), \nthe match is to be\
+    \ abandoned and the non-offending Team is to be declared the \nWinner.5.3.1\t\
+    This does not apply for players sent to the Sin Bin Area.5.4\tIn mixed gender\
+    \ competitions, the maximum number of males allowed on the \nField of Play is\
+    \ three (3), the minimum male requirement is one (1) and the \nminimum female\
+    \ requirement is one (1).6  Team Coach and Team Officials  \n6.1\tThe Team coach(s)\
+    \ and Team officials may be permitted inside the Perimeter \nbut shall be required\
+    \ to be positioned either in the Interchange Area or at the \nend of the Field\
+    \ of Play for the duration of the match.6.2\tThe Team coach(s) and Team officials\
+    \ may move from one position to the other \nbut shall do so without delay."
+  - "17.8\tFollowing a Try, players may Interchange at will, without having to wait\
+    \ for the \nplayer to enter the Interchange Area, but must do so prior to the\
+    \ Tap being taken \nto recommence play.18  Penalty  \n18.1\tThe Tap must be performed\
+    \ in accordance with the Definitions.Ruling = The Referee will instruct the offending\
+    \ Team to return to the Mark and perform the \nTap again.18.2\tFor Infringements\
+    \ that occur between seven (7) metre lines, the Mark for the \nPenalty Tap is\
+    \ at the point of Infringement unless otherwise indicated by the \nReferee.18.3\t\
+    For Infringements that occur within the Seven Metre Zone the Tap must be \ntaken\
+    \ at the nearest seven (7) metre line."
+  - "4.4\tHats or caps are permitted to be worn during a match provided they are safe\
+    \ \nand meet any NTA regulations.4.5\tSafe footwear must be worn with exceptions\
+    \ allowed for game variants such as \nBeach Touch.4.6\tLight leather or synthetic\
+    \ boots with soft moulded soles are permitted.4.6.1\tShoes with screw-in studs\
+    \ are not to be worn by any player or Referee.4.7\tPlayers are not to participate\
+    \ in any match wearing any item of jewellery, \nchain, identification band/bracelet\
+    \ or similar item that may prove dangerous.Any jewellery or other items that cannot\
+    \ be removed are to be taped to the \nsatisfaction of the Referee.4.8\tLong (extend\
+    \ beyond the finger flesh when viewed from the palm) or sharp \nfingernails are\
+    \ not allowed.4.9\tReferees and players may wear spectacles or sunglasses provided\
+    \ they are safe \nand securely attached."
+- source_sentence: Is there a time-out for injuries during a standard Touch Rugby
+    match?
+  sentences:
+  - "Line \nMarkings are to be laid out as shown in Appendix 1 - The Field of Play.Sidelines\
+    \ \nextend seven (7) metres beyond the Try Lines to join the Dead Ball Lines and\
+    \ \ndefine the In-Goal Areas which measure fifty (50) metres wide by seven (7)\
+    \ \nmetres in length.1.3\tThe Interchange Areas are located no closer than one\
+    \ (1) metre from each \nSideline.1.4\tSuitably sized markers, cones or corner\
+    \ posts of a distinguishing colour and \nmade from safe and pliable material should\
+    \ be positioned at the intersections of \nthe Sideline and Halfway line and the\
+    \ Sideline and the Try Line.1.4.1\tMarkers, cones or corner posts placed on the\
+    \ junction of the Sideline and \nTry Line are deemed to be in the Field of Play.1.4.2\t\
+    All other markers or cones are deemed to be out of the Field of Play.1.5\tThe\
+    \ standard playing surface is grass.Other surfaces including synthetic grass \n\
+    may be used but shall be subject to NTA approved standards."
+  - "10.10\tIf a player in Possession intentionally makes a Touch on an Offside defender\
+    \ \nwho is making every effort to retire and remain out of play, the Touch counts.FIT\
+    \ Playing Rules - 5th Edition\nCOPYRIGHT © Touch Football Australia 2020\n9\n\
+    10.11\tIf a Touch is made on a player in Possession while the player is juggling\
+    \ the ball \nin an attempt to maintain control of it, the Touch counts if the\
+    \ attacking player \nfollowing the Touch retains Possession.10.12\tIf a player\
+    \ in Possession is Touched and subsequently makes contact with \neither the Sideline,\
+    \ a field marker or the ground outside the Field of Play, the \nTouch counts and\
+    \ play continues with a Rollball at the Mark where the Touch \noccurred.10.13\t\
+    When a player from the Defending Team enters its defensive Seven Metre Zone, \n\
+    the Defending Team must move Forward at a reasonable pace until a Touch is \n\
+    Imminent or made.Ruling = A Penalty to the Attacking Team at the point of the\
+    \ Infringement."
+  - "4.9\tReferees and players may wear spectacles or sunglasses provided they are\
+    \ safe \nand securely attached.4.10\tReferees and players may wear sport monitoring\
+    \ equipment and medical \nsupports such as knee or ankle braces provided, at the\
+    \ sole discretion of \ncompetition’s controlling body, the items are not dangerous.5 \
+    \ Team Composition  \n5.1\tA Team consists of a maximum of 14 players, no more\
+    \ than six (6) of whom are \nallowed on the field at any time.FIT Playing Rules\
+    \ - 5th Edition\n6\nCOPYRIGHT © Touch Football Australia 2020\nRuling = A Penalty\
+    \ awarded to the non-offending Team at the time the offence is identified \nseven\
+    \ (7) metres infield on the Halfway Line or the position of the ball, whichever\
+    \ is the \ngreater Advantage.5.2\tA Team must have a minimum of four (4) players\
+    \ on the field for a match to \ncommence or continue, except during a Drop-Off."
+datasets:
+- Trelis/touch-rugby-modernbert-pairs
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+---
+
+# SentenceTransformer based on nomic-ai/modernbert-embed-base
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) on the [touch-rugby-modernbert-pairs](https://huggingface.co/datasets/Trelis/touch-rugby-modernbert-pairs) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [nomic-ai/modernbert-embed-base](https://huggingface.co/nomic-ai/modernbert-embed-base) <!-- at revision 92168cbee600b1abbfc10842aba988aa69572291 -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [touch-rugby-modernbert-pairs](https://huggingface.co/datasets/Trelis/touch-rugby-modernbert-pairs)
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("Trelis/modernbert-embed-base-touch-rugby-ft")
+# Run inference
+sentences = [
+    'Is there a time-out for injuries during a standard Touch Rugby match?',
+    '4.9\tReferees and players may wear spectacles or sunglasses provided they are safe \nand securely attached.4.10\tReferees and players may wear sport monitoring equipment and medical \nsupports such as knee or ankle braces provided, at the sole discretion of \ncompetition’s controlling body, the items are not dangerous.5\u2002 Team Composition  \n5.1\tA Team consists of a maximum of 14 players, no more than six (6) of whom are \nallowed on the field at any time.FIT Playing Rules - 5th Edition\n6\nCOPYRIGHT © Touch Football Australia 2020\nRuling = A Penalty awarded to the non-offending Team at the time the offence is identified \nseven (7) metres infield on the Halfway Line or the position of the ball, whichever is the \ngreater Advantage.5.2\tA Team must have a minimum of four (4) players on the field for a match to \ncommence or continue, except during a Drop-Off.',
+    '10.10\tIf a player in Possession intentionally makes a Touch on an Offside defender \nwho is making every effort to retire and remain out of play, the Touch counts.FIT Playing Rules - 5th Edition\nCOPYRIGHT © Touch Football Australia 2020\n9\n10.11\tIf a Touch is made on a player in Possession while the player is juggling the ball \nin an attempt to maintain control of it, the Touch counts if the attacking player \nfollowing the Touch retains Possession.10.12\tIf a player in Possession is Touched and subsequently makes contact with \neither the Sideline, a field marker or the ground outside the Field of Play, the \nTouch counts and play continues with a Rollball at the Mark where the Touch \noccurred.10.13\tWhen a player from the Defending Team enters its defensive Seven Metre Zone, \nthe Defending Team must move Forward at a reasonable pace until a Touch is \nImminent or made.Ruling = A Penalty to the Attacking Team at the point of the Infringement.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### touch-rugby-modernbert-pairs
+
+* Dataset: [touch-rugby-modernbert-pairs](https://huggingface.co/datasets/Trelis/touch-rugby-modernbert-pairs) at [7cb0ae2](https://huggingface.co/datasets/Trelis/touch-rugby-modernbert-pairs/tree/7cb0ae2222504ad98d6ca368b68a657ba2b33e22)
+* Size: 305 training samples
+* Columns: <code>question</code> and <code>related_chunk</code>
+* Approximate statistics based on the first 305 samples:
+  |         | question                                                                           | related_chunk                                                                        |
+  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                               |
+  | details | <ul><li>min: 10 tokens</li><li>mean: 18.73 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 147 tokens</li><li>mean: 231.2 tokens</li><li>max: 319 tokens</li></ul> |
+* Samples:
+  | question                                                                                                                                         | related_chunk                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+  |:-------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>What is defined as 'Forward' in the context of Touch Rugby?</code>                                                                         | <code>Referee<br>The match official(s) appointed to make Rulings during the conduct <br>of a match.Rollball<br>The act of bringing the ball into play following a Touch or a Change <br>of Possession.Ruck/Rollball Area<br>The area, not exceeding one (1) metre in distance, between the <br>player performing a Rollball and the Half.Ruling<br>The decision made by a Referee as a result of particular <br>circumstance and may result in a Play On, a Tap Penalty, a discipline <br>option, Change of Possession or a Try.Seven Metre Zone<br>The area between the seven (7) metre line and the Try Line.See <br>Appendix 1.Sidelines<br>The side boundaries of the Field of Play.See Appendix 1.Sin Bin<br>A player sent to the Sin-Bin Area for a period of four (4) completed <br>Possessions.The player is counted as a player on the Field of Play <br>and cannot be replaced or Interchanged.</code>                                                                                                     |
+  | <code>What is the numerical difference between the teams on the field of play during a Drop-Off if a player has been sent to the sin bin?</code> | <code>24.5	For the avoidance of doubt for clauses 24.3 and 24.4 the non-offending Team <br>will retain a numerical advantage on the Field of Play during the Drop-Off.25  Match Officials  <br>25.1	The Referee is the sole judge on all match related matters inside the Perimeter <br>for the Duration of a match, has jurisdiction over all players, coaches and <br>officials and is required to:<br>25.1.1	Inspect the Field of Play, Line Markings and Markers prior to the <br>commencement of the Match to ensure the safety of all participants.25.1.2	Adjudicate on the Rules of the game;<br>25.1.3	Impose any sanction necessary to control the match;<br>25.1.4	Award Tries and record the progressive score;<br>25.1.5	Maintain a count of Touches during each Possession;<br>25.1.6	Award Penalties for Infringements against the Rules; and<br>25.1.7	Report to the relevant competition administration any Sin Bins, <br>Dismissals or injuries to any participant sustained during a Match.</code> |
+  | <code>What happens if neither team is leading after two minutes of play in a Drop-Off?</code>                                                    | <code>24.1.5	Should neither Team be leading at the expiration of two (2) minutes, a <br>signal is given and the match will pause at the next Touch or Dead Ball.Each Team will then remove another player from the Field of Play.24.1.6	The Match will recommence immediately after the players have left the <br>field at the same place where it paused (i.e.the Team retains Possession <br>at the designated number of Touches, or at Change of Possession due to <br>some Infringement or the sixth Touch) and the Match will continue until a <br>Try is scored.24.1.7	There is no time off during the Drop-Off and the clock does not stop at <br>the two (2) minute interval.24.1.8	Substitution during the Drop-Off is permitted in accordance with normal <br>Interchange Rules.24.2	Mixed gender Teams may have no more than (2) males on the field during the <br>Drop-Off.</code>                                                                                                                       |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### touch-rugby-modernbert-pairs
+
+* Dataset: [touch-rugby-modernbert-pairs](https://huggingface.co/datasets/Trelis/touch-rugby-modernbert-pairs) at [7cb0ae2](https://huggingface.co/datasets/Trelis/touch-rugby-modernbert-pairs/tree/7cb0ae2222504ad98d6ca368b68a657ba2b33e22)
+* Size: 305 evaluation samples
+* Columns: <code>question</code> and <code>related_chunk</code>
+* Approximate statistics based on the first 305 samples:
+  |         | question                                                                           | related_chunk                                                                         |
+  |:--------|:-----------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                                |
+  | details | <ul><li>min: 13 tokens</li><li>mean: 17.61 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 147 tokens</li><li>mean: 230.39 tokens</li><li>max: 319 tokens</li></ul> |
+* Samples:
+  | question                                                                                        | related_chunk                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+  |:------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>What is the penalty for an attacking player obstructing a defender in Touch Rugby?</code> | <code>Ruling = A Penalty to the Attacking Team at the point of the Infringement or on the seven (7) <br>metre line directly Forward of the Infringement.15.4	When a Rollball occurs within Defending Team’s Seven Metre Zone or a Penalty <br>Tap within ten (10) metres of the Defending Team’s Try Line, all players from the <br>Defending Team must have both feet on or behind their Try Line and no other <br>part of the body in contact with the ground Forward of their Try Line.Ruling = A Penalty to the Attacking Team at the seven (7) metre line directly Forward of the <br>point of the Infringement.15.5	After effecting the Touch, the defending player must retire the required seven <br>(7) metres or to the Defending Try Line as indicated by the Referee without <br>interfering with the Attacking Team.Ruling = A Penalty to the Attacking Team ten (10) metres Forward of the Infringement or if <br>on the Defensive Try Line, on the seven (7) metre line.</code>                                                                                                                    |
+  | <code>When must a player perform a Rollball seven metres in-field?</code>                       | <code>13.5	A player may only perform a Rollball at the Mark under the following <br>circumstances:<br>13.5.1	when a Touch has been made; or<br>13.5.2	when Possession changes following the sixth Touch; or<br>13.5.3	when Possession changes due to the ball being dropped or passed and <br>goes to the ground; or<br>13.5.4	when Possession changes due to an Infringement by an attacking player <br>at a Penalty, a Tap or a Rollball; or<br>FIT Playing Rules - 5th Edition<br>COPYRIGHT © Touch Football Australia 2020<br>11<br>13.5.5	when Possession changes after the Half is Touched or when the Half <br>places the ball on or over the Try Line; or<br>13.5.6	in replacement of a Penalty Tap; or<br>13.5.7	when so directed by the Referee.13.6	A player is to perform a Rollball seven (7) metres in-field under the following <br>circumstances:<br>13.6.1	when a Change of Possession takes place due to a player in Possession <br>making contact with the Sideline or any ground outside the Field of Play, <br>prior to a Touch being made; or<br>13.6.2	when the ball not in Poss...</code> |
+  | <code>What is the ruling if a player uses excessive force when making a touch?</code>           | <code>FIT Playing Rules - 5th Edition<br>8<br>COPYRIGHT © Touch Football Australia 2020<br>9.6	If a player mishandles the ball and even if in an effort to gain control, the ball <br>is accidentally knocked Forward into any other Player, a Change of Possession <br>results.10  The Touch  <br>10.1	A Touch may be made by either a defending player or a player in Possession.10.2	A defending player may not claim a Touch if contact has not been made.If a <br>player claims a Touch has been made, but the Referee is unsure the Touch will <br>count.Ruling = A Penalty to the Attacking Team at the point of the Infringement and the offending <br>player sent to the Sin Bin.10.3	Players of both Defending and Attacking Teams are to use the minimum force <br>necessary to make a Touch.Players must ensure that the method employed in <br>making a Touch does not pose an unnecessary risk to player safety.Ruling = A Penalty to the non-offending Team at the point of the Infringement.</code>                                                                                               |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 32
+- `per_device_eval_batch_size`: 32
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 1
+- `lr_scheduler_type`: cosine
+- `warmup_ratio`: 0.3
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 32
+- `per_device_eval_batch_size`: 32
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: cosine
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.3
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss |
+|:------:|:----:|:-------------:|:---------------:|
+| 0.2222 | 2    | 2.7671        | nan             |
+| 0.4444 | 4    | 0.0           | nan             |
+| 0.6667 | 6    | 0.0           | nan             |
+| 0.8889 | 8    | 0.0           | nan             |
+
+
+### Framework Versions
+- Python: 3.12.4
+- Sentence Transformers: 3.3.1
+- Transformers: 4.48.0
+- PyTorch: 2.5.1
+- Accelerate: 1.3.0
+- Datasets: 2.17.1
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "nomic-ai/modernbert-embed-base",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": false,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0",
+  "vocab_size": 50368
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.48.0",
+    "pytorch": "2.5.1"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c17a60cc07bc4248e3ffd4d05a9ea38401c38601df4e40ae334d15d280c9350
+size 596070136

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}