jebish7 commited on
Commit
ba4e555
·
verified ·
1 Parent(s): 572e0c0

Add new SentenceTransformer model.

Browse files
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,492 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: jebish7/mpnet-base-all-obliqa_NMR
3
+ library_name: sentence-transformers
4
+ pipeline_tag: sentence-similarity
5
+ tags:
6
+ - sentence-transformers
7
+ - sentence-similarity
8
+ - feature-extraction
9
+ - generated_from_trainer
10
+ - dataset_size:29547
11
+ - loss:MultipleNegativesRankingLoss
12
+ widget:
13
+ - source_sentence: Are there any ADGM-specific guidelines or best practices for integrating
14
+ anti-money laundering (AML) compliance into our technology and financial systems
15
+ to manage operational risks effectively?
16
+ sentences:
17
+ - "REGULATORY REQUIREMENTS FOR AUTHORISED PERSONS ENGAGED IN REGULATED ACTIVITIES\
18
+ \ IN RELATION TO VIRTUAL ASSETS\nAnti-Money Laundering and Countering Financing\
19
+ \ of Terrorism\nIn order to develop a robust and sustainable regulatory framework\
20
+ \ for Virtual Assets, FSRA is of the view that a comprehensive application of\
21
+ \ its AML/CFT framework should be in place, including full compliance with, among\
22
+ \ other things, the:\n\na)\tUAE AML/CFT Federal Laws, including the UAE Cabinet\
23
+ \ Resolution No. (10) of 2019 Concerning the Executive Regulation of the Federal\
24
+ \ Law No. 20 of 2018 concerning Anti-Money Laundering and Combating Terrorism\
25
+ \ Financing;\n\nb)\tUAE Cabinet Resolution 20 of 2019 concerning the procedures\
26
+ \ of dealing with those listed under the UN sanctions list and UAE/local terrorist\
27
+ \ lists issued by the Cabinet, including the FSRA AML and Sanctions Rules and\
28
+ \ Guidance (“AML Rules”) or such other AML rules as may be applicable in ADGM\
29
+ \ from time to time; and\n\nc)\tadoption of international best practices (including\
30
+ \ the FATF Recommendations).\n"
31
+ - 'DIGITAL SECURITIES SETTLEMENT
32
+
33
+ Digital Settlement Facilities (DSFs)
34
+
35
+ For the purposes of this Guidance and distinct from RCHs, the FSRA will consider
36
+ DSFs suitable for the purposes of settlement (MIR Rule 3.8) and custody (MIR Rule
37
+ 2.10) of Digital Securities. A DSF, holding an FSP for Providing Custody, may
38
+ provide custody and settlement services in Digital Securities for RIEs and MTFs
39
+ (as applicable). Therefore, for the purposes of custody and settlement of Digital
40
+ Securities, the arrangements that a RIE or MTF would normally have in place with
41
+ a RCH can be replaced with arrangements provided by a DSF, provided that certain
42
+ requirements, as described in this section, are met.
43
+
44
+ '
45
+ - 'REGULATORY REQUIREMENTS FOR AUTHORISED PERSONS ENGAGED IN REGULATED ACTIVITIES
46
+ IN RELATION TO VIRTUAL ASSETS
47
+
48
+ Security measures and procedures
49
+
50
+ IT infrastructures should be strong enough to resist, without significant loss
51
+ to Clients, a number of scenarios, including but not limited to: accidental destruction
52
+ or breach of data, collusion or leakage of information by employees/former employees,
53
+ successful hack of a cryptographic and hardware security module or server, or
54
+ access by hackers of any single set of encryption/decryption keys that could result
55
+ in a complete system breach.
56
+
57
+ '
58
+ - source_sentence: How does the ADGM enforce the Market Abuse Provisions, such as
59
+ those outlined in section 92 of the FSMR, especially for Accepted Spot Commodities,
60
+ and what are the reporting obligations for companies in relation to market abuse
61
+ and transaction reporting?
62
+ sentences:
63
+ - The Regulator shall have the power to require an Institution in Resolution, or
64
+ any of its Group Entities, to provide any services or facilities (excluding any
65
+ financial support) that are necessary to enable the Recipient to operate the transferred
66
+ business effectively, including where the Institution under Resolution or relevant
67
+ Group Entity has entered into Insolvency Proceedings.
68
+ - If the Regulator considers that an auditor or actuary has committed a contravention
69
+ of these Regulations, it may disqualify the auditor or actuary from being the
70
+ auditor of, or (as the case may be), from acting as an actuary for, any Authorised
71
+ Person, Recognised Body or Reporting Entity or any particular class thereof.
72
+ - 'REGULATORY REQUIREMENTS - SPOT COMMODITY ACTIVITIES
73
+
74
+ Market Abuse and Transaction Reporting (FSMR)
75
+
76
+ Importantly, the Market Abuse Provisions (including section 92) in Part 8 of FSMR
77
+ specifically cover Market Abuse Behaviour in relation to Accepted Spot Commodities
78
+ admitted to trading on an RIE, MTF or OTF. In this regard, the FSRA imposes the
79
+ same high regulatory standards to Accepted Spot Commodities traded on RIEs, MTFs
80
+ or OTFs as it does to Financial Instruments traded on RIEs, MTFs or OTFs.
81
+
82
+ '
83
+ - source_sentence: Can you provide further clarification on the specific measures
84
+ deemed adequate for handling conflicts of interest related to the provision and
85
+ management of credit within an Authorised Person's organization?
86
+ sentences:
87
+ - 'Own estimate haircuts . If an Authorised Person fails to comply with Rule A4.3.18,
88
+ the Regulator may revoke its approval for the Authorised Person to use own estimate
89
+ haircuts. The Authorised Person may also be required to revise its estimates for
90
+ the purpose of calculating regulatory Capital Requirements if its estimates of
91
+ E*, does not adequately reflect its Exposure to Counterparty Credit Risk.
92
+
93
+
94
+ '
95
+ - Financial risk . All applicants are required to demonstrate they have a sound
96
+ initial capital base and funding and must be able to meet the relevant prudential
97
+ requirements of ADGM legislation, on an ongoing basis. This includes holding enough
98
+ capital resources to cover expenses even if expected revenue takes time to materialise.
99
+ Start-ups can encounter greater financial risks as they seek to establish and
100
+ grow a new business.
101
+ - An Authorised Person with one or more branches outside the ADGM must implement
102
+ and maintain Credit Risk policies adapted to each local market and its regulatory
103
+ conditions.
104
+ - source_sentence: What are the recommended best practices for ensuring that all disclosures
105
+ are prepared in accordance with the PRMS, and how can we validate that our classification
106
+ and reporting of Petroleum Resources meet the standards set forth?
107
+ sentences:
108
+ - 'DISCLOSURE REQUIREMENTS .
109
+
110
+ Material Exploration and drilling results
111
+
112
+ Rule 12.5.1 sets out the reporting requirements relevant to disclosures of material
113
+ Exploration and drilling results in relation to Petroleum Resources. Such disclosures
114
+ should be presented in a factual and balanced manner, and contain sufficient information
115
+ to allow investors and their advisers to make an informed judgement of its materiality. Care
116
+ needs to be taken to ensure that a disclosure does not suggest, without reasonable
117
+ grounds, that commercially recoverable or potentially recoverable quantities of
118
+ Petroleum have been discovered, in the absence of determining and disclosing estimates
119
+ of Petroleum Resources in accordance with Chapter 12 and the PRMS.
120
+
121
+ '
122
+ - If appointed, the Trustee must also take reasonable steps to ensure that its Employees
123
+ comply with IFR ‎6.2.6(a)‎(i)-‎(iv).
124
+ - Notwithstanding this Rule, an Authorised Person would generally be expected to
125
+ separate the roles of Compliance Officer and Senior Executive Officer. In addition,
126
+ the roles of Compliance Officer, Finance Officer and Money Laundering Reporting
127
+ Officer would not be expected to be combined with any other Controlled Functions
128
+ unless appropriate monitoring and control arrangements independent of the individual
129
+ concerned will be implemented by the Authorised Person. This may be possible in
130
+ the case of a Branch, where monitoring and controlling of the individual (carrying
131
+ out more than one role in the Branch) is conducted from the Authorised Person's
132
+ home state by an appropriate individual for each of the relevant Controlled Functions
133
+ as applicable. However, it is recognised that, on a case by case basis, there
134
+ may be exceptional circumstances in which this may not always be practical or
135
+ possible.
136
+ - source_sentence: Can the ADGM provide examples of legal risks associated with securitisation
137
+ that Authorised Persons should particularly be aware of and manage?
138
+ sentences:
139
+ - "When employing an eKYC System to assist with CDD, a Relevant Person should:\n\
140
+ a.\tensure that it has a thorough understanding of the eKYC System itself and\
141
+ \ the risks of eKYC, including those outlined by relevant guidance from FATF and\
142
+ \ other international standard setting bodies;\nb.\tcomply with all the Rules\
143
+ \ of the Regulator relevant to eKYC including, but not limited to, applicable\
144
+ \ requirements regarding the business risk assessment, as per Rule ‎6.1, and outsourcing,\
145
+ \ as per Rule ‎9.3;\nc.\tcombine eKYC with transaction monitoring, anti-fraud\
146
+ \ and cyber-security measures to support a wider framework preventing applicable\
147
+ \ Financial Crime; and\nd.\ttake appropriate steps to identify, assess and mitigate\
148
+ \ the risk of the eKYC system being misused for the purposes of Financial Crime."
149
+ - This Chapter includes the detailed Rules and associated guidance in respect of
150
+ a firm's obligation to manage effectively its Exposures to Operational Risk. Operational
151
+ Risk refers to the risk of incurring losses due to the failure of systems, processes,
152
+ and personnel to perform expected tasks. Operational Risk losses also include
153
+ losses arising out of legal risk. This Chapter aims to ensure that an Authorised
154
+ Person has a robust Operational Risk management framework commensurate with the
155
+ nature, scale and complexity of its operations and that it holds sufficient regulatory
156
+ capital against Operational Risk Exposures.
157
+ - 'An Insurer must calculate the asset management risk component in respect of a
158
+ Long Term Insurance Fund according to the method set out in Rule ‎A4.13, applied
159
+ as though all references in that Rule to an Insurer were instead references to
160
+ that fund.
161
+
162
+ '
163
+ ---
164
+
165
+ # SentenceTransformer based on jebish7/mpnet-base-all-obliqa_NMR
166
+
167
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [jebish7/mpnet-base-all-obliqa_NMR](https://huggingface.co/jebish7/mpnet-base-all-obliqa_NMR) on the csv dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
168
+
169
+ ## Model Details
170
+
171
+ ### Model Description
172
+ - **Model Type:** Sentence Transformer
173
+ - **Base model:** [jebish7/mpnet-base-all-obliqa_NMR](https://huggingface.co/jebish7/mpnet-base-all-obliqa_NMR) <!-- at revision 1e5dd5450bf7c54409b5ac5bba0a8336c233418d -->
174
+ - **Maximum Sequence Length:** 384 tokens
175
+ - **Output Dimensionality:** 768 tokens
176
+ - **Similarity Function:** Cosine Similarity
177
+ - **Training Dataset:**
178
+ - csv
179
+ <!-- - **Language:** Unknown -->
180
+ <!-- - **License:** Unknown -->
181
+
182
+ ### Model Sources
183
+
184
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
185
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
186
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
187
+
188
+ ### Full Model Architecture
189
+
190
+ ```
191
+ SentenceTransformer(
192
+ (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel
193
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
194
+ (2): Normalize()
195
+ )
196
+ ```
197
+
198
+ ## Usage
199
+
200
+ ### Direct Usage (Sentence Transformers)
201
+
202
+ First install the Sentence Transformers library:
203
+
204
+ ```bash
205
+ pip install -U sentence-transformers
206
+ ```
207
+
208
+ Then you can load this model and run inference.
209
+ ```python
210
+ from sentence_transformers import SentenceTransformer
211
+
212
+ # Download from the 🤗 Hub
213
+ model = SentenceTransformer("jebish7/mpnet-base-all-obliqa_NMR_3")
214
+ # Run inference
215
+ sentences = [
216
+ 'Can the ADGM provide examples of legal risks associated with securitisation that Authorised Persons should particularly be aware of and manage?',
217
+ "This Chapter includes the detailed Rules and associated guidance in respect of a firm's obligation to manage effectively its Exposures to Operational Risk. Operational Risk refers to the risk of incurring losses due to the failure of systems, processes, and personnel to perform expected tasks. Operational Risk losses also include losses arising out of legal risk. This Chapter aims to ensure that an Authorised Person has a robust Operational Risk management framework commensurate with the nature, scale and complexity of its operations and that it holds sufficient regulatory capital against Operational Risk Exposures.",
218
+ 'When employing an eKYC System to assist with CDD, a Relevant Person should:\na.\tensure that it has a thorough understanding of the eKYC System itself and the risks of eKYC, including those outlined by relevant guidance from FATF and other international standard setting bodies;\nb.\tcomply with all the Rules of the Regulator relevant to eKYC including, but not limited to, applicable requirements regarding the business risk assessment, as per Rule \u200e6.1, and outsourcing, as per Rule \u200e9.3;\nc.\tcombine eKYC with transaction monitoring, anti-fraud and cyber-security measures to support a wider framework preventing applicable Financial Crime; and\nd.\ttake appropriate steps to identify, assess and mitigate the risk of the eKYC system being misused for the purposes of Financial Crime.',
219
+ ]
220
+ embeddings = model.encode(sentences)
221
+ print(embeddings.shape)
222
+ # [3, 768]
223
+
224
+ # Get the similarity scores for the embeddings
225
+ similarities = model.similarity(embeddings, embeddings)
226
+ print(similarities.shape)
227
+ # [3, 3]
228
+ ```
229
+
230
+ <!--
231
+ ### Direct Usage (Transformers)
232
+
233
+ <details><summary>Click to see the direct usage in Transformers</summary>
234
+
235
+ </details>
236
+ -->
237
+
238
+ <!--
239
+ ### Downstream Usage (Sentence Transformers)
240
+
241
+ You can finetune this model on your own dataset.
242
+
243
+ <details><summary>Click to expand</summary>
244
+
245
+ </details>
246
+ -->
247
+
248
+ <!--
249
+ ### Out-of-Scope Use
250
+
251
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
252
+ -->
253
+
254
+ <!--
255
+ ## Bias, Risks and Limitations
256
+
257
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
258
+ -->
259
+
260
+ <!--
261
+ ### Recommendations
262
+
263
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
264
+ -->
265
+
266
+ ## Training Details
267
+
268
+ ### Training Dataset
269
+
270
+ #### csv
271
+
272
+ * Dataset: csv
273
+ * Size: 29,547 training samples
274
+ * Columns: <code>Question</code> and <code>positive</code>
275
+ * Approximate statistics based on the first 1000 samples:
276
+ | | Question | positive |
277
+ |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
278
+ | type | string | string |
279
+ | details | <ul><li>min: 15 tokens</li><li>mean: 34.89 tokens</li><li>max: 96 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 115.11 tokens</li><li>max: 384 tokens</li></ul> |
280
+ * Samples:
281
+ | Question | positive |
282
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
283
+ | <code>Under Rules 7.3.2 and 7.3.3, what are the two specific conditions related to the maturity of a financial instrument that would trigger a disclosure requirement?</code> | <code>Events that trigger a disclosure. For the purposes of Rules 7.3.2 and 7.3.3, a Person is taken to hold Financial Instruments in or relating to a Reporting Entity, if the Person holds a Financial Instrument that on its maturity will confer on him:<br>(1) an unconditional right to acquire the Financial Instrument; or<br>(2) the discretion as to his right to acquire the Financial Instrument.<br></code> |
284
+ | <code>**Best Execution and Transaction Handling**: What constitutes 'Best Execution' under Rule 6.5 in the context of virtual assets, and how should Authorised Persons document and demonstrate this?</code> | <code>The following COBS Rules should be read as applying to all Transactions undertaken by an Authorised Person conducting a Regulated Activity in relation to Virtual Assets, irrespective of any restrictions on application or any exception to these Rules elsewhere in COBS -<br>(a) Rule 3.4 (Suitability);<br>(b) Rule 6.5 (Best Execution);<br>(c) Rule 6.7 (Aggregation and Allocation);<br>(d) Rule 6.10 (Confirmation Notes);<br>(e) Rule 6.11 (Periodic Statements); and<br>(f) Chapter 12 (Key Information and Client Agreement).</code> |
285
+ | <code>How does the FSRA define and evaluate "principal risks and uncertainties" for a Petroleum Reporting Entity, particularly for the remaining six months of the financial year?</code> | <code>A Reporting Entity must:<br>(a) prepare such report:<br>(i) for the first six months of each financial year or period, and if there is a change to the accounting reference date, prepare such report in respect of the period up to the old accounting reference date; and<br>(ii) in accordance with the applicable IFRS standards or other standards acceptable to the Regulator;<br>(b) ensure the financial statements have either been audited or reviewed by auditors, and the audit or review by the auditor is included within the report; and<br>(c) ensure that the report includes:<br>(i) except in the case of a Mining Exploration Reporting Entity or a Petroleum Exploration Reporting Entity, an indication of important events that have occurred during the first six months of the financial year, and their impact on the financial statements;<br>(ii) except in the case of a Mining Exploration Reporting Entity or a Petroleum Exploration Reporting Entity, a description of the principal risks and uncertainties for the remaining six months of the financial year; and<br>(iii) a condensed set of financial statements, an interim management report and associated responsibility statements.</code> |
286
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
287
+ ```json
288
+ {
289
+ "scale": 20.0,
290
+ "similarity_fct": "cos_sim"
291
+ }
292
+ ```
293
+
294
+ ### Training Hyperparameters
295
+ #### Non-Default Hyperparameters
296
+
297
+ - `per_device_train_batch_size`: 24
298
+ - `learning_rate`: 2e-05
299
+ - `num_train_epochs`: 2
300
+ - `warmup_ratio`: 0.1
301
+ - `batch_sampler`: no_duplicates
302
+
303
+ #### All Hyperparameters
304
+ <details><summary>Click to expand</summary>
305
+
306
+ - `overwrite_output_dir`: False
307
+ - `do_predict`: False
308
+ - `eval_strategy`: no
309
+ - `prediction_loss_only`: True
310
+ - `per_device_train_batch_size`: 24
311
+ - `per_device_eval_batch_size`: 8
312
+ - `per_gpu_train_batch_size`: None
313
+ - `per_gpu_eval_batch_size`: None
314
+ - `gradient_accumulation_steps`: 1
315
+ - `eval_accumulation_steps`: None
316
+ - `torch_empty_cache_steps`: None
317
+ - `learning_rate`: 2e-05
318
+ - `weight_decay`: 0.0
319
+ - `adam_beta1`: 0.9
320
+ - `adam_beta2`: 0.999
321
+ - `adam_epsilon`: 1e-08
322
+ - `max_grad_norm`: 1.0
323
+ - `num_train_epochs`: 2
324
+ - `max_steps`: -1
325
+ - `lr_scheduler_type`: linear
326
+ - `lr_scheduler_kwargs`: {}
327
+ - `warmup_ratio`: 0.1
328
+ - `warmup_steps`: 0
329
+ - `log_level`: passive
330
+ - `log_level_replica`: warning
331
+ - `log_on_each_node`: True
332
+ - `logging_nan_inf_filter`: True
333
+ - `save_safetensors`: True
334
+ - `save_on_each_node`: False
335
+ - `save_only_model`: False
336
+ - `restore_callback_states_from_checkpoint`: False
337
+ - `no_cuda`: False
338
+ - `use_cpu`: False
339
+ - `use_mps_device`: False
340
+ - `seed`: 42
341
+ - `data_seed`: None
342
+ - `jit_mode_eval`: False
343
+ - `use_ipex`: False
344
+ - `bf16`: False
345
+ - `fp16`: False
346
+ - `fp16_opt_level`: O1
347
+ - `half_precision_backend`: auto
348
+ - `bf16_full_eval`: False
349
+ - `fp16_full_eval`: False
350
+ - `tf32`: None
351
+ - `local_rank`: 0
352
+ - `ddp_backend`: None
353
+ - `tpu_num_cores`: None
354
+ - `tpu_metrics_debug`: False
355
+ - `debug`: []
356
+ - `dataloader_drop_last`: False
357
+ - `dataloader_num_workers`: 0
358
+ - `dataloader_prefetch_factor`: None
359
+ - `past_index`: -1
360
+ - `disable_tqdm`: False
361
+ - `remove_unused_columns`: True
362
+ - `label_names`: None
363
+ - `load_best_model_at_end`: False
364
+ - `ignore_data_skip`: False
365
+ - `fsdp`: []
366
+ - `fsdp_min_num_params`: 0
367
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
368
+ - `fsdp_transformer_layer_cls_to_wrap`: None
369
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
370
+ - `deepspeed`: None
371
+ - `label_smoothing_factor`: 0.0
372
+ - `optim`: adamw_torch
373
+ - `optim_args`: None
374
+ - `adafactor`: False
375
+ - `group_by_length`: False
376
+ - `length_column_name`: length
377
+ - `ddp_find_unused_parameters`: None
378
+ - `ddp_bucket_cap_mb`: None
379
+ - `ddp_broadcast_buffers`: False
380
+ - `dataloader_pin_memory`: True
381
+ - `dataloader_persistent_workers`: False
382
+ - `skip_memory_metrics`: True
383
+ - `use_legacy_prediction_loop`: False
384
+ - `push_to_hub`: False
385
+ - `resume_from_checkpoint`: None
386
+ - `hub_model_id`: None
387
+ - `hub_strategy`: every_save
388
+ - `hub_private_repo`: False
389
+ - `hub_always_push`: False
390
+ - `gradient_checkpointing`: False
391
+ - `gradient_checkpointing_kwargs`: None
392
+ - `include_inputs_for_metrics`: False
393
+ - `eval_do_concat_batches`: True
394
+ - `fp16_backend`: auto
395
+ - `push_to_hub_model_id`: None
396
+ - `push_to_hub_organization`: None
397
+ - `mp_parameters`:
398
+ - `auto_find_batch_size`: False
399
+ - `full_determinism`: False
400
+ - `torchdynamo`: None
401
+ - `ray_scope`: last
402
+ - `ddp_timeout`: 1800
403
+ - `torch_compile`: False
404
+ - `torch_compile_backend`: None
405
+ - `torch_compile_mode`: None
406
+ - `dispatch_batches`: None
407
+ - `split_batches`: None
408
+ - `include_tokens_per_second`: False
409
+ - `include_num_input_tokens_seen`: False
410
+ - `neftune_noise_alpha`: None
411
+ - `optim_target_modules`: None
412
+ - `batch_eval_metrics`: False
413
+ - `eval_on_start`: False
414
+ - `use_liger_kernel`: False
415
+ - `eval_use_gather_object`: False
416
+ - `batch_sampler`: no_duplicates
417
+ - `multi_dataset_batch_sampler`: proportional
418
+
419
+ </details>
420
+
421
+ ### Training Logs
422
+ | Epoch | Step | Training Loss |
423
+ |:------:|:----:|:-------------:|
424
+ | 0.1623 | 100 | 0.4433 |
425
+ | 0.3247 | 200 | 0.3978 |
426
+ | 0.4870 | 300 | 0.4173 |
427
+ | 0.6494 | 400 | 0.4892 |
428
+ | 0.8117 | 500 | 0.5729 |
429
+ | 0.9740 | 600 | 0.5901 |
430
+ | 1.1331 | 700 | 0.4664 |
431
+ | 1.2955 | 800 | 0.3703 |
432
+ | 1.4578 | 900 | 0.3813 |
433
+ | 1.6201 | 1000 | 0.3964 |
434
+ | 1.7825 | 1100 | 0.4536 |
435
+ | 1.9448 | 1200 | 0.4513 |
436
+
437
+
438
+ ### Framework Versions
439
+ - Python: 3.10.14
440
+ - Sentence Transformers: 3.1.1
441
+ - Transformers: 4.45.2
442
+ - PyTorch: 2.4.0
443
+ - Accelerate: 0.34.2
444
+ - Datasets: 3.0.1
445
+ - Tokenizers: 0.20.0
446
+
447
+ ## Citation
448
+
449
+ ### BibTeX
450
+
451
+ #### Sentence Transformers
452
+ ```bibtex
453
+ @inproceedings{reimers-2019-sentence-bert,
454
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
455
+ author = "Reimers, Nils and Gurevych, Iryna",
456
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
457
+ month = "11",
458
+ year = "2019",
459
+ publisher = "Association for Computational Linguistics",
460
+ url = "https://arxiv.org/abs/1908.10084",
461
+ }
462
+ ```
463
+
464
+ #### MultipleNegativesRankingLoss
465
+ ```bibtex
466
+ @misc{henderson2017efficient,
467
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
468
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
469
+ year={2017},
470
+ eprint={1705.00652},
471
+ archivePrefix={arXiv},
472
+ primaryClass={cs.CL}
473
+ }
474
+ ```
475
+
476
+ <!--
477
+ ## Glossary
478
+
479
+ *Clearly define terms in order to be accessible across audiences.*
480
+ -->
481
+
482
+ <!--
483
+ ## Model Card Authors
484
+
485
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
486
+ -->
487
+
488
+ <!--
489
+ ## Model Card Contact
490
+
491
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
492
+ -->
config.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "jebish7/mpnet-base-all-obliqa_NMR",
3
+ "architectures": [
4
+ "MPNetModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "eos_token_id": 2,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "layer_norm_eps": 1e-05,
15
+ "max_position_embeddings": 514,
16
+ "model_type": "mpnet",
17
+ "num_attention_heads": 12,
18
+ "num_hidden_layers": 12,
19
+ "pad_token_id": 1,
20
+ "relative_attention_num_buckets": 32,
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.45.2",
23
+ "vocab_size": 30527
24
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.1.1",
4
+ "transformers": "4.45.2",
5
+ "pytorch": "2.4.0"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": null
10
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de254c1429df2815811bdfac92c0b1f0d1d90cf13af5d1190cdf9748f4e9f9c6
3
+ size 437967672
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 384,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "[UNK]",
46
+ "lstrip": false,
47
+ "normalized": false,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "104": {
36
+ "content": "[UNK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ },
43
+ "30526": {
44
+ "content": "<mask>",
45
+ "lstrip": true,
46
+ "normalized": false,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": true
50
+ }
51
+ },
52
+ "bos_token": "<s>",
53
+ "clean_up_tokenization_spaces": false,
54
+ "cls_token": "<s>",
55
+ "do_lower_case": true,
56
+ "eos_token": "</s>",
57
+ "mask_token": "<mask>",
58
+ "max_length": 128,
59
+ "model_max_length": 384,
60
+ "pad_to_multiple_of": null,
61
+ "pad_token": "<pad>",
62
+ "pad_token_type_id": 0,
63
+ "padding_side": "right",
64
+ "sep_token": "</s>",
65
+ "stride": 0,
66
+ "strip_accents": null,
67
+ "tokenize_chinese_chars": true,
68
+ "tokenizer_class": "MPNetTokenizer",
69
+ "truncation_side": "right",
70
+ "truncation_strategy": "longest_first",
71
+ "unk_token": "[UNK]"
72
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff