README.md

---
pipeline_tag: sentence-similarity
language:
- es
tags:
- sentence-transformers
- feature-extraction
- sentence-similarity
- transformers
- LSG
- 4096 context
license: apache-2.0
---
### LSG Variant of hiiamsid/sentence_similarity_spanish_es

#### Overview

This model is an enhanced version of [hiiamsid/sentence_similarity_spanish_es](https://huggingface.co/hiiamsid/sentence_similarity_spanish_es), now transformed using Local Sparse Global (LSG) attention mechanism. The adaptation to LSG allows for efficient handling of longer sequences, making the model more versatile and robust in a wider range of natural language processing tasks.

This LSG adaptation enables the model to efficiently process sequences up to 4096 tokens in length.

#### About the LSG architecture

[LSG (Local Sparse Global)](https://github.com/ccdv-ai/convert_checkpoint_to_lsg) attention is a cutting-edge approach designed to mitigate the limitations of the traditional self-attention mechanism in Transformer models, particularly for processing long sequences. By incorporating local, sparse, and global attention, LSG attention significantly reduces computational complexity while maintaining, and often enhancing, model performance.

#### Model adaptation

This LSG variant has been adapted from our original model, ["hiiamsid/sentence_similarity_spanish_es"](https://huggingface.co/hiiamsid/sentence_similarity_spanish_es), with the primary goal of extending its capabilities to efficiently handle longer text inputs. This enhancement enables the model to maintain high accuracy and efficiency, even with extended sequence lengths that were previously challenging for the base model.

#### Use cases

The LSG-enhanced model is particularly adept at tasks involving longer documents, where capturing the essence of extended context is crucial.

#### Acknowledgments

This model has been adapted by Darío Muñoz Prudant, thanks to the Hugging Face community and contributors to the LSG attention mechanism for their resources and support.