yahyaabd/allstats-search-mini-v1-1-mnrl-special-token-v4

SentenceTransformer based on yahyaabd/allstats-search-mini-v1-1-mnrl

This is a sentence-transformers model finetuned from yahyaabd/allstats-search-mini-v1-1-mnrl on the bps-pub-cosine-pairs dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: yahyaabd/allstats-search-mini-v1-1-mnrl
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • bps-pub-cosine-pairs

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("yahyaabd/allstats-search-mini-v1-1-mnrl-special-token-v4")
# Run inference
sentences = [
    'Berapa produksi sampah perkotaan per kapita per hari?',
    'Harga Konsumen Beberapa Barang dan Jasa Kelompok Sandang di 66 Kota di Indonesia 2013',
    'Indikator Ekonomi Juni 2003',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Semantic Similarity

• Datasets: sts-dev and sts-test
• Evaluated with EmbeddingSimilarityEvaluator

Metric	sts-dev	sts-test
pearson_cosine	0.9668	0.9692
spearman_cosine	0.8568	0.8589

Training Details

Training Dataset

bps-pub-cosine-pairs

• Dataset: bps-pub-cosine-pairs at d624648
• Size: 8,159 training samples
• Columns: query, title, and score
• Approximate statistics based on the first 1000 samples:

  	query	title	score
  type	string	string	float
  details	min: 4 tokens mean: 11.04 tokens max: 30 tokens	min: 5 tokens mean: 13.02 tokens max: 43 tokens	min: 0.1 mean: 0.55 max: 0.9

• Samples:

  query	title	score
  Nilai Tukar Nelayan	Statistik Hotel dan Akomodasi Lainnya di Indonesia 2013	0.1
  Berapa angka statistik pertambangan non migas Indonesia periode 2012?	Statistik Pertambangan Non Minyak dan Gas Bumi 2011-2015	0.9
  Bagaimana situasi angkatan kerja Indonesia di bulan Februari 2021?	Keadaan Angkatan Kerja di Indonesia Februari 2021	0.9

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Evaluation Dataset

bps-pub-cosine-pairs

• Dataset: bps-pub-cosine-pairs at d624648
• Size: 1,022 evaluation samples
• Columns: query, title, and score
• Approximate statistics based on the first 1000 samples:

  	query	title	score
  type	string	string	float
  details	min: 4 tokens mean: 11.19 tokens max: 31 tokens	min: 5 tokens mean: 13.24 tokens max: 44 tokens	min: 0.1 mean: 0.56 max: 0.9

• Samples:

  query	title	score
  Sosek Desember 2021	Laporan Bulanan Data Sosial Ekonomi Desember 2021	0.9
  Ekspor Indonesia menurut SITC 2019-2020	Statistik Perdagangan Luar Negeri Indonesia Ekspor Menurut Kode SITC, 2019-2020	0.9
  Pengeluaran konsumsi penduduk Indonesia Maret 2018	Pengeluaran untuk Konsumsi Penduduk Indonesia, Maret 2018	0.9

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• learning_rate: 1e-05
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True
• label_smoothing_factor: 0.01
• eval_on_start: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 1e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.01
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	Validation Loss	sts-dev_spearman_cosine	sts-test_spearman_cosine
0	0	-	0.0371	0.8433	-
0.0392	10	0.0365	0.0366	0.8434	-
0.0784	20	0.0514	0.0351	0.8440	-
0.1176	30	0.036	0.0330	0.8448	-
0.1569	40	0.0299	0.0310	0.8456	-
0.1961	50	0.0371	0.0293	0.8465	-
0.2353	60	0.037	0.0276	0.8479	-
0.2745	70	0.0307	0.0260	0.8495	-
0.3137	80	0.0283	0.0243	0.8508	-
0.3529	90	0.0251	0.0230	0.8517	-
0.3922	100	0.0213	0.0220	0.8520	-
0.4314	110	0.0244	0.0215	0.8522	-
0.4706	120	0.0234	0.0208	0.8526	-
0.5098	130	0.0219	0.0200	0.8530	-
0.5490	140	0.0187	0.0195	0.8536	-
0.5882	150	0.0188	0.0189	0.8538	-
0.6275	160	0.0189	0.0184	0.8540	-
0.6667	170	0.0193	0.0178	0.8543	-
0.7059	180	0.0171	0.0173	0.8545	-
0.7451	190	0.017	0.0171	0.8546	-
0.7843	200	0.0206	0.0168	0.8548	-
0.8235	210	0.016	0.0163	0.8549	-
0.8627	220	0.0173	0.0161	0.8552	-
0.9020	230	0.0161	0.0158	0.8553	-
0.9412	240	0.0173	0.0156	0.8553	-
0.9804	250	0.0131	0.0155	0.8552	-
1.0196	260	0.0175	0.0152	0.8554	-
1.0588	270	0.015	0.0149	0.8555	-
1.0980	280	0.0119	0.0145	0.8556	-
1.1373	290	0.0126	0.0143	0.8557	-
1.1765	300	0.0133	0.0141	0.8557	-
1.2157	310	0.0134	0.0138	0.8557	-
1.2549	320	0.0123	0.0136	0.8558	-
1.2941	330	0.0118	0.0135	0.8558	-
1.3333	340	0.0117	0.0134	0.8558	-
1.3725	350	0.0143	0.0133	0.8559	-
1.4118	360	0.0118	0.0131	0.8559	-
1.4510	370	0.0119	0.0129	0.8563	-
1.4902	380	0.0117	0.0126	0.8565	-
1.5294	390	0.0132	0.0125	0.8566	-
1.5686	400	0.0112	0.0124	0.8566	-
1.6078	410	0.0117	0.0125	0.8566	-
1.6471	420	0.013	0.0125	0.8566	-
1.6863	430	0.0109	0.0123	0.8566	-
1.7255	440	0.0135	0.0123	0.8566	-
1.7647	450	0.0116	0.0123	0.8566	-
1.8039	460	0.0115	0.0121	0.8566	-
1.8431	470	0.0116	0.0119	0.8566	-
1.8824	480	0.013	0.0118	0.8567	-
1.9216	490	0.0114	0.0117	0.8567	-
1.9608	500	0.0111	0.0117	0.8567	-
2.0	510	0.0114	0.0115	0.8567	-
2.0392	520	0.0098	0.0113	0.8567	-
2.0784	530	0.0075	0.0112	0.8567	-
2.1176	540	0.0089	0.0112	0.8567	-
2.1569	550	0.0083	0.0111	0.8567	-
2.1961	560	0.0077	0.0110	0.8567	-
2.2353	570	0.0128	0.0110	0.8567	-
2.2745	580	0.0092	0.0109	0.8567	-
2.3137	590	0.0103	0.0109	0.8567	-
2.3529	600	0.009	0.0108	0.8567	-
2.3922	610	0.0086	0.0108	0.8567	-
2.4314	620	0.0076	0.0108	0.8567	-
2.4706	630	0.0101	0.0107	0.8568	-
2.5098	640	0.0094	0.0107	0.8568	-
2.5490	650	0.0102	0.0107	0.8568	-
2.5882	660	0.008	0.0106	0.8568	-
2.6275	670	0.0091	0.0106	0.8568	-
2.6667	680	0.0101	0.0106	0.8568	-
2.7059	690	0.0119	0.0105	0.8568	-
2.7451	700	0.0081	0.0105	0.8568	-
2.7843	710	0.0098	0.0105	0.8568	-
2.8235	720	0.0076	0.0105	0.8568	-
2.8627	730	0.009	0.0105	0.8568	-
2.9020	740	0.0091	0.0105	0.8568	-
2.9412	750	0.0106	0.0105	0.8568	-
2.9804	760	0.0097	0.0105	0.8568	-
-1	-1	-	-	-	0.8589

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.4.0
• Transformers: 4.48.1
• PyTorch: 2.5.1+cu124
• Accelerate: 1.3.0
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}