Hugging Face's logo

zenml
/
finetuned-snowflake-arctic-embed-m

Sentence Similarity
sentence-transformers
Safetensors
English
bert
feature-extraction
Generated from Trainer
dataset_size:1490
loss:MatryoshkaLoss
loss:TripletLoss
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
finetuned-snowflake-arctic-embed-m / README.md
strickvl's picture
strickvl
Add new SentenceTransformer model.
87d67d4 verified
preview code
|
raw
history blame
71 kB
---
base_model: Snowflake/snowflake-arctic-embed-m
datasets: []
language:
- en
library_name: sentence-transformers
license: apache-2.0
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:1490
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: How can I list the 'kubernetes-cluster' resources that are accessible
by service connectors in my ZenML workspace?
sentences:
- 'lly registered orchestrator `<ORCHESTRATOR_NAME>`.$ zenml service-connector list-resources
--resource-type kubernetes-cluster -e
The following ''kubernetes-cluster'' resources can be accessed by service connectors
configured in your workspace:
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓
┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE
│ RESOURCE TYPE │ RESOURCE NAMES ┃
┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
┃ e33c9fac-5daa-48b2-87bb-0187d3782cde │ aws-iam-multi-eu │ 🔶 aws │
🌀 kubernetes-cluster │ kubeflowmultitenant ┃
┃ │ │ │ │
zenbox ┃
┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
┃ ed528d5a-d6cb-4fc4-bc52-c3d2d01643e5 │ aws-iam-multi-us │ 🔶 aws │
🌀 kubernetes-cluster │ zenhacks-cluster ┃
┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨
┃ 1c54b32a-4889-4417-abbd-42d3ace3d03a │ gcp-sa-multi │ 🔵 gcp │
🌀 kubernetes-cluster │ zenml-test-cluster ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛'
- 'Name your pipeline runs
In the output logs of a pipeline run you will see the name of the run:
Pipeline run training_pipeline-2023_05_24-12_41_04_576473 has finished in 3.742s.
This name is automatically generated based on the current date and time. To change
the name for a run, pass run_name as a parameter to the with_options() method:
training_pipeline = training_pipeline.with_options(
run_name="custom_pipeline_run_name"
training_pipeline()
Pipeline run names must be unique, so if you plan to run your pipelines multiple
times or run them on a schedule, make sure to either compute the run name dynamically
or include one of the following placeholders that ZenML will replace:
{{date}} will resolve to the current date, e.g. 2023_02_19
{{time}} will resolve to the current time, e.g. 11_07_09_326492
training_pipeline = training_pipeline.with_options(
run_name=f"custom_pipeline_run_name_{{date}}_{{time}}"
training_pipeline()
Be sure to include the f string prefix to allow for the placeholders to be replaced,
as shown in the example above. Without the f prefix, the placeholders will not
be replaced.
PreviousUsing a custom step invocation ID
NextUse failure/success hooks
Last updated 19 days ago'
- ' a_new_local_stack -o default -a my_artifact_storestack : This is the CLI group
that enables interactions with the stacks
register: Here we want to register a new stack. Explore other operations withzenml
stack --help.
a_new_local_stack : This is the unique name that the stack will have.
--orchestrator or -o are used to specify which orchestrator to use for the stack
--artifact-store or -a are used to specify which artifact store to use for the
stack
The output for the command should look something like this:
Using the default local database.
Running with active workspace: ''default'' (repository)
Stack ''a_new_local_stack'' successfully registered!
You can inspect the stack with the following command:
zenml stack describe a_new_local_stack
Which will give you an output like this:
Stack Configuration
┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━┓
┃ COMPONENT_TYPE │ COMPONENT_NAME ┃
┠────────────────┼───────────────────┨
┃ ORCHESTRATOR │ default ┃
┠────────────────┼───────────────────┨
┃ ARTIFACT_STORE │ my_artifact_store ┃
┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┛
''a_new_local_stack'' stack
Stack ''a_new_local_stack'' with id ''...'' is owned by user default and is ''private''.
Switch stacks with our VS Code extension
If you are using our VS Code extension, you can easily view and switch your stacks
by opening the sidebar (click on the ZenML icon). You can then click on the stack
you want to switch to as well as view the stack components it''s made up of.
Run a pipeline on the new local stack
Let''s use the pipeline in our starter project from the previous guide to see
it in action.
If you have not already, clone the starter template:
pip install "zenml[templates,server]" notebook
zenml integration install sklearn -y
mkdir zenml_starter
cd zenml_starter
zenml init --template starter --template-with-defaults
# Just in case, we install the requirements again
pip install -r requirements.txt
The starter template is the same as the ZenML quickstart. You can clone it like
so:'
- source_sentence: How can I explicitly name my model version in ZenML?
sentences:
- 'strator supports specifying resources in what way.If you''re using an orchestrator
which does not support this feature or its underlying infrastructure does not
cover your requirements, you can also take a look at step operators which allow
you to execute individual steps of ../...your pipeline in environments independent
of your orchestrator.
Ensure your container is CUDA-enabled
To run steps or pipelines on GPUs, it''s crucial to have the necessary CUDA tools
installed in the environment. This section will guide you on how to configure
your environment to utilize GPU capabilities effectively.
Note that these configuration changes are required for the GPU hardware to be
properly utilized. If you don''t update the settings, your steps might run, but
they will not see any boost in performance from the custom hardware.
All steps running on GPU-backed hardware will be executed within a containerized
environment, whether you''re using the local Docker orchestrator or a cloud instance
of Kubeflow. Therefore, you need to make two amendments to your Docker settings
for the relevant steps:
1. Specify a CUDA-enabled parent image in your DockerSettings
For complete details, refer to the containerization page that explains how to
do this. As an example, if you want to use the latest CUDA-enabled official PyTorch
image for your entire pipeline run, you can include the following code:
from zenml import pipeline
from zenml.config import DockerSettings
docker_settings = DockerSettings(parent_image="pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime")
@pipeline(settings={"docker": docker_settings})
def my_pipeline(...):
...
For TensorFlow, you might use the tensorflow/tensorflow:latest-gpu image, as detailed
in the official TensorFlow documentation or their DockerHub overview.
2. Add ZenML as an explicit pip requirement'
- 'Reuse Docker builds to speed up Docker build times
Avoid building Docker images each time a pipeline runs
When using containerized components in your stack, ZenML needs to build Docker
images to remotely execute your code. Building Docker images without connecting
a git repository includes your step code in the built Docker image. This, however,
means that new Docker images will be built and pushed whenever you make changes
to any of your source files.
One way of skipping Docker builds each time is to pass in the ID of a build as
you run the pipeline:
my_pipeline = my_pipeline.with_options(build=<BUILD_ID>)
or when running a pipeline from the CLI:
zenml pipeline run <PIPELINE_NAME> --build=<BUILD_ID>
Please note, that this means specifying a custom build when running a pipeline
will not run the code on your client machine but will use the code included in
the Docker images of the build. As a consequence, even if you make local code
changes, reusing a build will always execute the code bundled in the Docker image,
rather than the local code. Therefore, if you would like to reuse a Docker build
AND make sure your local code changes are also downloaded into the image, you
need to connect a git repository.
PreviousWhich files are built into the image
NextUse code repositories to automate Docker build reuse
Last updated 19 days ago'
- 'Controlling Model versions
Each model can have many versions. Model versions are a way for you to track different
iterations of your training process, complete with some extra dashboard and API
functionality to support the full ML lifecycle.
E.g. Based on your business rules during training, you can associate model version
with stages and promote them to production. You have an interface that allows
you to link these versions with non-technical artifacts and data, e.g. business
data, datasets, or even stages in your process and workflow.
Model versions are created implicitly as you are running your machine learning
training, so you don''t have to immediately think about this. If you want more
control over versions, our API has you covered, with an option to explicitly name
your versions.
Explicitly name your model version
If you want to explicitly name your model version, you can do so by passing in
the version argument to the Model object. If you don''t do this, ZenML will automatically
generate a version number for you.
from zenml import Model, step, pipeline
model= Model(
name="my_model",
version="1.0.5"
# The step configuration will take precedence over the pipeline
@step(model=model)
def svc_trainer(...) -> ...:
...
# This configures it for all steps within the pipeline
@pipeline(model=model)
def training_pipeline( ... ):
# training happens here
Here we are specifically setting the model configuration for a particular step
or for the pipeline as a whole.
Please note in the above example if the model version exists, it is automatically
associated with the pipeline and becomes active in the pipeline context. Therefore,
a user should be careful and intentional as to whether you want to create a new
pipeline, or fetch an existing one. See below for an example of fetching a model
from an existing version/stage.
Fetching model versions by stage'
- source_sentence: What are the different roles available for users in an organization
within ZenML Pro?
sentences:
- 'User Management
In ZenML Pro, there is a slightly different entity hierarchy as compared to the
open-source ZenML framework. This document walks you through the key differences
and new concepts that are pro-only.
Organizations, Tenants, and Roles
ZenML Pro arranges various aspects of your work experience around the concept
of an Organization. This is the top-most level structure within the ZenML Cloud
environment. Generally, an organization contains a group of users and one or more
tenants. Tenants are individual, isolated deployments of the ZenML server.
Every user in an organization has a distinct role. Each role configures what they
can view, modify, and their level of involvement in collaborative tasks. A role
thus helps determine the level of access that a user has within an organization.
The admin has all permissions on an organization. They are allowed to add members,
adjust the billing information and assign roles. The editor can still fully manage
tenants and members but is not allowed to access the subscription information
or delete the organization. The viewer Role allows you to allow users to access
the tenants within the organization with only view permissions.
Inviting Team Members
Inviting users to your organization to work on the organization''s tenants is
easy. Simply click Add Member in the Organization settings, and give them an initial
Role. The User will be sent an invitation email. If a user is part of an organization,
they can utilize their login on all tenants they have authority to access.
PreviousZenML SaaS
NextStarter guide
Last updated 12 days ago'
- ' more information.
Get the last run of a pipelineTo access the most recent run of a pipeline, you
can either use the last_run property or access it through the runs list:
last_run = pipeline_model.last_run # OR: pipeline_model.runs[0]
If your most recent runs have failed, and you want to find the last run that has
succeeded, you can use the last_successful_run property instead.
Get the latest run from a pipeline
Calling a pipeline executes it and then returns the response of the freshly executed
run.
run = training_pipeline()
The run that you get back is the model stored in the ZenML database at the point
of the method call. This means the pipeline run is still initializing and no steps
have been run. To get the latest state can get a refreshed version from the client:
from zenml.client import Client
Client().get_pipeline_run(run.id) to get a refreshed version
Get a run via the client
If you already know the exact run that you want to fetch (e.g., from looking at
the dashboard), you can use the Client.get_pipeline_run() method to fetch the
run directly without having to query the pipeline first:
from zenml.client import Client
pipeline_run = Client().get_pipeline_run("first_pipeline-2023_06_20-16_20_13_274466")
Similar to pipelines, you can query runs by either ID, name, or name prefix, and
you can also discover runs through the Client or CLI via the Client.list_pipeline_runs()
or zenml pipeline runs list commands.
Run information
Each run has a collection of useful information which can help you reproduce your
runs. In the following, you can find a list of some of the most useful pipeline
run information, but there is much more available. See the PipelineRunResponse
definition for a comprehensive list.
Status
The status of a pipeline run. There are five possible states: initialized, failed,
completed, running, and cached.
status = run.status
Configuration'
- 'ner(gamma=gamma, X_train=X_train, y_train=y_train)if __name__ == "__main__":
first_pipeline()
python run.py
...
Registered pipeline first_pipeline (version 2).
...
This will now create a single run for version 2 of the pipeline called first_pipeline.
PreviousHyperparameter tuning
NextAccess secrets in a step
Last updated 15 days ago'
- source_sentence: How can I check which GCP Service Connectors can access a GCS bucket
in my ZenML deployment?
sentences:
- 'dashboard.
Warning! Usage in remote orchestratorsThe current ZenML version has a limitation
in its base Docker image that requires a workaround for all pipelines using Deepchecks
with a remote orchestrator (e.g. Kubeflow , Vertex). The limitation being that
the base Docker image needs to be extended to include binaries that are required
by opencv2, which is a package that Deepchecks requires.
While these binaries might be available on most operating systems out of the box
(and therefore not a problem with the default local orchestrator), we need to
tell ZenML to add them to the containerization step when running in remote settings.
Here is how:
First, create a file called deepchecks-zenml.Dockerfile and place it on the same
level as your runner script (commonly called run.py). The contents of the Dockerfile
are as follows:
ARG ZENML_VERSION=0.20.0
FROM zenmldocker/zenml:${ZENML_VERSION} AS base
RUN apt-get update
RUN apt-get install ffmpeg libsm6 libxext6 -y
Then, place the following snippet above your pipeline definition. Note that the
path of the dockerfile are relative to where the pipeline definition file is.
Read the containerization guide for more details:
import zenml
from zenml import pipeline
from zenml.config import DockerSettings
from pathlib import Path
import sys
docker_settings = DockerSettings(
dockerfile="deepchecks-zenml.Dockerfile",
build_options={
"buildargs": {
"ZENML_VERSION": f"{zenml.__version__}"
},
},
@pipeline(settings={"docker": docker_settings})
def my_pipeline(...):
# same code as always
...
From here on, you can continue to use the deepchecks integration as is explained
below.
The Deepchecks standard steps
ZenML wraps the Deepchecks functionality for tabular data in the form of four
standard steps:
DeepchecksDataIntegrityCheckStep: use it in your pipelines to run data integrity
tests on a single dataset
DeepchecksDataDriftCheckStep: use it in your pipelines to run data drift tests
on two datasets as input: target and reference.'
- ' gs://zenml-core_cloudbuild ┃┃ │ gs://zenml-datasets ┃
┃ │ gs://zenml-internal-artifact-store ┃
┃ │ gs://zenml-kubeflow-artifact-store ┃
┃ │ gs://zenml-project-time-series-bucket ┃
┠───────────────────────┼─────────────────────────────────────────────────┨
┃ 🌀 kubernetes-cluster │ zenml-test-cluster ┃
┠───────────────────────┼─────────────────────────────────────────────────┨
┃ 🐳 docker-registry │ gcr.io/zenml-core ┃
┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Long-lived credentials (API keys, account keys)
This is the magic formula of authentication methods. When paired with another
ability, such as automatically generating short-lived API tokens, or impersonating
accounts or assuming roles, this is the ideal authentication mechanism to use,
particularly when using ZenML in production and when sharing results with other
members of your ZenML team.
As a general best practice, but implemented particularly well for cloud platforms,
account passwords are never directly used as a credential when authenticating
to the cloud platform APIs. There is always a process in place that exchanges
the account/password credential for another type of long-lived credential:
AWS uses the aws configure CLI command
GCP offers the gcloud auth application-default login CLI commands
Azure provides the az login CLI command
None of your original login information is stored on your local machine or used
to access workloads. Instead, an API key, account key or some other form of intermediate
credential is generated and stored on the local host and used to authenticate
to remote cloud service APIs.'
- ' should pick the one that best fits your use case.If you already have one or
more GCP Service Connectors configured in your ZenML deployment, you can check
which of them can be used to access the GCS bucket you want to use for your GCS
Artifact Store by running e.g.:
zenml service-connector list-resources --resource-type gcs-bucket
Example Command Output
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE
│ RESOURCE TYPE │ RESOURCE NAMES ┃
┠──────────────────────────────────────┼─────────────────────┼────────────────┼───────────────┼─────────────────────────────────────────────────┨
┃ 7f0c69ba-9424-40ae-8ea6-04f35c2eba9d │ gcp-user-account │ 🔵 gcp │
📦 gcs-bucket │ gs://zenml-bucket-sl ┃
┃ │ │ │ │
gs://zenml-core.appspot.com ┃
┃ │ │ │ │
gs://zenml-core_cloudbuild ┃
┃ │ │ │ │
gs://zenml-datasets ┃
┃ │ │ │ │
gs://zenml-internal-artifact-store ┃
┃ │ │ │ │
gs://zenml-kubeflow-artifact-store ┃
┠──────────────────────────────────────┼─────────────────────┼────────────────┼───────────────┼─────────────────────────────────────────────────┨
┃ 2a0bec1b-9787-4bd7-8d4a-9a47b6f61643 │ gcs-zenml-bucket-sl │ 🔵 gcp │
📦 gcs-bucket │ gs://zenml-bucket-sl ┃'
- source_sentence: Is it possible to update the local AWS CLI configuration with credentials
extracted from the AWS Service Connector?
sentences:
- '36a885: Pull complete
c9c0554c8e6a: Pull completebacdcd847a66: Pull complete
482033770844: Pull complete
Digest: sha256:bf2cc3895e70dfa1ee1cd90bbfa599fa4cd8df837e27184bac1ce1cc239ecd3f
Status: Downloaded newer image for 715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest
715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest
It is also possible to update the local AWS CLI configuration with credentials
extracted from the AWS Service Connector:
zenml service-connector login aws-session-token --resource-type aws-generic
Example Command Output
Configured local AWS SDK profile ''zenml-c0f8e857''.
The ''aws-session-token'' AWS Service Connector connector was used to successfully
configure the local Generic AWS resource client/SDK.
A new profile is created in the local AWS CLI configuration holding the credentials.
It can be used to access AWS resources and services, e.g.:
aws --profile zenml-c0f8e857 s3 ls
Stack Components use
The S3 Artifact Store Stack Component can be connected to a remote AWS S3 bucket
through an AWS Service Connector.
The AWS Service Connector can also be used with any Orchestrator or Model Deployer
stack component flavor that relies on Kubernetes clusters to manage workloads.
This allows EKS Kubernetes container workloads to be managed without the need
to configure and maintain explicit AWS or Kubernetes kubectl configuration contexts
and credentials in the target environment and in the Stack Component.
Similarly, Container Registry Stack Components can be connected to an ECR Container
Registry through an AWS Service Connector. This allows container images to be
built and published to ECR container registries without the need to configure
explicit AWS credentials in the target environment or the Stack Component.
End-to-end examples'
- ' ┃┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ RESOURCE TYPES │ 🌀 kubernetes-cluster ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ RESOURCE NAME │ arn:aws:eks:us-east-1:715803424590:cluster/zenhacks-cluster ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ SECRET ID │ ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ SESSION DURATION │ N/A ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ EXPIRES IN │ 11h59m57s ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ OWNER │ default ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ WORKSPACE │ default ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ SHARED │ ➖ ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ CREATED_AT │ 2023-06-16 10:17:46.931091 ┃
┠──────────────────┼─────────────────────────────────────────────────────────────────────┨
┃ UPDATED_AT │ 2023-06-16 10:17:46.931094 ┃
┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
Configuration'
- 'e --authentication_secret. For example, you''d run:zenml secret create argilla_secrets
--api_key="<your_argilla_api_key>"
(Visit the Argilla documentation and interface to obtain your API key.)
Then register your annotator with ZenML:
zenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets
When using a deployed instance of Argilla, the instance URL must be specified
without any trailing / at the end. If you are using a Hugging Face Spaces instance
and its visibility is set to private, you must also set the extra_headers parameter
which would include a Hugging Face token. For example:
zenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets
--instance_url="https://[your-owner-name]-[your_space_name].hf.space" --extra_headers="{"Authorization":
f"Bearer {<your_hugging_face_token>}"}"
Finally, add all these components to a stack and set it as your active stack.
For example:
zenml stack copy default annotation
# this must be done separately so that the other required stack components are
first registered
zenml stack update annotation -an <YOUR_ARGILLA_ANNOTATOR>
zenml stack set annotation
# optionally also
zenml stack describe
Now if you run a simple CLI command like zenml annotator dataset list this should
work without any errors. You''re ready to use your annotator in your ML workflow!
How do you use it?
ZenML supports access to your data and annotations via the zenml annotator ...
CLI command. We have also implemented an interface to some of the common Argilla
functionality via the ZenML SDK.
You can access information about the datasets you''re using with the zenml annotator
dataset list. To work on annotation for a particular dataset, you can run zenml
annotator dataset annotate <dataset_name>. What follows is an overview of some
key components to the Argilla integration and how it can be used.
Argilla Annotator Stack Component'
model-index:
- name: zenml/finetuned-snowflake-arctic-embed-m
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 384
type: dim_384
metrics:
- type: cosine_accuracy@1
value: 0.29518072289156627
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.6204819277108434
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6927710843373494
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7891566265060241
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.29518072289156627
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.20682730923694775
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.13855421686746985
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.0789156626506024
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.29518072289156627
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.6204819277108434
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6927710843373494
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7891566265060241
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5524302146116403
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.4758486326257412
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4836255621339311
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.28313253012048195
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5963855421686747
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.6807228915662651
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7771084337349398
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.28313253012048195
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.19879518072289157
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.136144578313253
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07771084337349396
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.28313253012048195
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5963855421686747
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.6807228915662651
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7771084337349398
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.5376005319054157
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.46014534327787354
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4690725321460052
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.2710843373493976
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.5301204819277109
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.5963855421686747
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.7409638554216867
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.2710843373493976
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.17670682730923692
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.11927710843373492
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.07409638554216866
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.2710843373493976
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.5301204819277109
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.5963855421686747
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.7409638554216867
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.49831034220322973
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.4218158347676423
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.43128822737879013
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.26506024096385544
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.4819277108433735
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.5662650602409639
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.6566265060240963
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.26506024096385544
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.1606425702811245
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.11325301204819276
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.06566265060240963
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.26506024096385544
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.4819277108433735
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.5662650602409639
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.6566265060240963
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.45413104746517285
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.38985465672212666
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.4019553541721889
name: Cosine Map@100
---
# zenml/finetuned-snowflake-arctic-embed-m
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m) <!-- at revision 71bc94c8f9ea1e54fba11167004205a65e5da2cc -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m")
# Run inference
sentences = [
'Is it possible to update the local AWS CLI configuration with credentials extracted from the AWS Service Connector?',
"36a885: Pull complete\n\nc9c0554c8e6a: Pull completebacdcd847a66: Pull complete\n\n482033770844: Pull complete\n\nDigest: sha256:bf2cc3895e70dfa1ee1cd90bbfa599fa4cd8df837e27184bac1ce1cc239ecd3f\n\nStatus: Downloaded newer image for 715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest\n\n715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest\n\nIt is also possible to update the local AWS CLI configuration with credentials extracted from the AWS Service Connector:\n\nzenml service-connector login aws-session-token --resource-type aws-generic\n\nExample Command Output\n\nConfigured local AWS SDK profile 'zenml-c0f8e857'.\n\nThe 'aws-session-token' AWS Service Connector connector was used to successfully configure the local Generic AWS resource client/SDK.\n\nA new profile is created in the local AWS CLI configuration holding the credentials. It can be used to access AWS resources and services, e.g.:\n\naws --profile zenml-c0f8e857 s3 ls\n\nStack Components use\n\nThe S3 Artifact Store Stack Component can be connected to a remote AWS S3 bucket through an AWS Service Connector.\n\nThe AWS Service Connector can also be used with any Orchestrator or Model Deployer stack component flavor that relies on Kubernetes clusters to manage workloads. This allows EKS Kubernetes container workloads to be managed without the need to configure and maintain explicit AWS or Kubernetes kubectl configuration contexts and credentials in the target environment and in the Stack Component.\n\nSimilarly, Container Registry Stack Components can be connected to an ECR Container Registry through an AWS Service Connector. This allows container images to be built and published to ECR container registries without the need to configure explicit AWS credentials in the target environment or the Stack Component.\n\nEnd-to-end examples",
'e --authentication_secret. For example, you\'d run:zenml secret create argilla_secrets --api_key="<your_argilla_api_key>"\n\n(Visit the Argilla documentation and interface to obtain your API key.)\n\nThen register your annotator with ZenML:\n\nzenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets\n\nWhen using a deployed instance of Argilla, the instance URL must be specified without any trailing / at the end. If you are using a Hugging Face Spaces instance and its visibility is set to private, you must also set the extra_headers parameter which would include a Hugging Face token. For example:\n\nzenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets --instance_url="https://[your-owner-name]-[your_space_name].hf.space" --extra_headers="{"Authorization": f"Bearer {<your_hugging_face_token>}"}"\n\nFinally, add all these components to a stack and set it as your active stack. For example:\n\nzenml stack copy default annotation\n\n# this must be done separately so that the other required stack components are first registered\n\nzenml stack update annotation -an <YOUR_ARGILLA_ANNOTATOR>\n\nzenml stack set annotation\n\n# optionally also\n\nzenml stack describe\n\nNow if you run a simple CLI command like zenml annotator dataset list this should work without any errors. You\'re ready to use your annotator in your ML workflow!\n\nHow do you use it?\n\nZenML supports access to your data and annotations via the zenml annotator ... CLI command. We have also implemented an interface to some of the common Argilla functionality via the ZenML SDK.\n\nYou can access information about the datasets you\'re using with the zenml annotator dataset list. To work on annotation for a particular dataset, you can run zenml annotator dataset annotate <dataset_name>. What follows is an overview of some key components to the Argilla integration and how it can be used.\n\nArgilla Annotator Stack Component',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `dim_384`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.2952 |
| cosine_accuracy@3 | 0.6205 |
| cosine_accuracy@5 | 0.6928 |
| cosine_accuracy@10 | 0.7892 |
| cosine_precision@1 | 0.2952 |
| cosine_precision@3 | 0.2068 |
| cosine_precision@5 | 0.1386 |
| cosine_precision@10 | 0.0789 |
| cosine_recall@1 | 0.2952 |
| cosine_recall@3 | 0.6205 |
| cosine_recall@5 | 0.6928 |
| cosine_recall@10 | 0.7892 |
| cosine_ndcg@10 | 0.5524 |
| cosine_mrr@10 | 0.4758 |
| **cosine_map@100** | **0.4836** |
#### Information Retrieval
* Dataset: `dim_256`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.2831 |
| cosine_accuracy@3 | 0.5964 |
| cosine_accuracy@5 | 0.6807 |
| cosine_accuracy@10 | 0.7771 |
| cosine_precision@1 | 0.2831 |
| cosine_precision@3 | 0.1988 |
| cosine_precision@5 | 0.1361 |
| cosine_precision@10 | 0.0777 |
| cosine_recall@1 | 0.2831 |
| cosine_recall@3 | 0.5964 |
| cosine_recall@5 | 0.6807 |
| cosine_recall@10 | 0.7771 |
| cosine_ndcg@10 | 0.5376 |
| cosine_mrr@10 | 0.4601 |
| **cosine_map@100** | **0.4691** |
#### Information Retrieval
* Dataset: `dim_128`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.2711 |
| cosine_accuracy@3 | 0.5301 |
| cosine_accuracy@5 | 0.5964 |
| cosine_accuracy@10 | 0.741 |
| cosine_precision@1 | 0.2711 |
| cosine_precision@3 | 0.1767 |
| cosine_precision@5 | 0.1193 |
| cosine_precision@10 | 0.0741 |
| cosine_recall@1 | 0.2711 |
| cosine_recall@3 | 0.5301 |
| cosine_recall@5 | 0.5964 |
| cosine_recall@10 | 0.741 |
| cosine_ndcg@10 | 0.4983 |
| cosine_mrr@10 | 0.4218 |
| **cosine_map@100** | **0.4313** |
#### Information Retrieval
* Dataset: `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:----------|
| cosine_accuracy@1 | 0.2651 |
| cosine_accuracy@3 | 0.4819 |
| cosine_accuracy@5 | 0.5663 |
| cosine_accuracy@10 | 0.6566 |
| cosine_precision@1 | 0.2651 |
| cosine_precision@3 | 0.1606 |
| cosine_precision@5 | 0.1133 |
| cosine_precision@10 | 0.0657 |
| cosine_recall@1 | 0.2651 |
| cosine_recall@3 | 0.4819 |
| cosine_recall@5 | 0.5663 |
| cosine_recall@10 | 0.6566 |
| cosine_ndcg@10 | 0.4541 |
| cosine_mrr@10 | 0.3899 |
| **cosine_map@100** | **0.402** |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 1,490 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
| | positive | anchor |
|:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 10 tokens</li><li>mean: 21.19 tokens</li><li>max: 64 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 374.24 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
| positive | anchor |
|:---------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>How do I use it to start the local Airflow server for ZenML?</code> | <code> use it<br><br>To use the Airflow orchestrator, we need:The ZenML airflow integration installed. If you haven't done so, runCopyzenml integration install airflow<br><br>Docker installed and running.<br><br>The orchestrator registered and part of our active stack:<br><br>zenml orchestrator register <ORCHESTRATOR_NAME> \<br><br>--flavor=airflow \<br><br>--local=True # set this to `False` if using a remote Airflow deployment<br><br># Register and activate a stack with the new orchestrator<br><br>zenml stack register <STACK_NAME> -o <ORCHESTRATOR_NAME> ... --set<br><br>In the local case, we need to reinstall in a certain way for the local Airflow server:<br><br>pip install "apache-airflow-providers-docker<3.8.0" "apache-airflow==2.4.0" "pendulum<3.0.0" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.0/constraints-3.9.txt"<br><br>Please make sure to replace 3.9 with your Python (major) version in the constraints file URL given above.<br><br>Once that is installed, we can start the local Airflow server by running the following command in your terminal. See further below on an alternative way to set up the Airflow server manually since the zenml stack up command is deprecated.<br><br>zenml stack up<br><br>This command will start up an Airflow server on your local machine that's running in the same Python environment that you used to provision it. When it is finished, it will print a username and password which you can use to log in to the Airflow UI here.<br><br>As long as you didn't configure any custom value for the dag_output_dir attribute of your orchestrator, running a pipeline locally is as simple as calling:<br><br>python file_that_runs_a_zenml_pipeline.py<br><br>This call will produce a .zip file containing a representation of your ZenML pipeline to the Airflow DAGs directory. From there, the local Airflow server will load it and run your pipeline (It might take a few seconds until the pipeline shows up in the Airflow UI).</code> |
| <code>What are the best practices for various authentication methods implemented by Service Connectors?</code> | <code>Security best practices<br><br>Best practices concerning the various authentication methods implemented by Service Connectors.<br><br>Service Connector Types, especially those targeted at cloud providers, offer a plethora of authentication methods matching those supported by remote cloud platforms. While there is no single authentication standard that unifies this process, there are some patterns that are easily identifiable and can be used as guidelines when deciding which authentication method to use to configure a Service Connector.<br><br>This section explores some of those patterns and gives some advice regarding which authentication methods are best suited for your needs.<br><br>This section may require some general knowledge about authentication and authorization to be properly understood. We tried to keep it simple and limit ourselves to talking about high-level concepts, but some areas may get a bit too technical.<br><br>Username and password<br><br>The key takeaway is this: you should avoid using your primary account password as authentication credentials as much as possible. If there are alternative authentication methods that you can use or other types of credentials (e.g. session tokens, API keys, API tokens), you should always try to use those instead.<br><br>Ultimately, if you have no choice, be cognizant of the third parties you share your passwords with. If possible, they should never leave the premises of your local host or development environment.<br><br>This is the typical authentication method that uses a username or account name plus the associated password. While this is the de facto method used to log in with web consoles and local CLIs, this is the least secure of all authentication methods and never something you want to share with other members of your team or organization or use to authenticate automated workloads.</code> |
| <code>How can I interact with a model deployer after deploying a model in ZenML?</code> | <code>eline_step_name="huggingface_model_deployer_step",model_name="LLAMA-7B",<br><br>model_uri="s3://zenprojects/huggingface_model_deployer_step/output/884/huggingface",<br><br>revision="main",<br><br>task="text-classification",<br><br>region="us-east-1",<br><br>vendor="aws",<br><br>token="huggingface_token",<br><br>namespace="zenml-workloads",<br><br>endpoint_type="public",<br><br>print(f"Model server {service.config['model_name']} is deployed at {service.status['prediction_url']}")<br><br>How to Interact with a model deployer after deployment?<br><br>When a Model Deployer is part of the active ZenML Stack, it is also possible to interact with it from the CLI to list, start, stop, or delete the model servers that is managed:<br><br>$ zenml model-deployer models list<br><br>┏━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓<br><br>┃ STATUS │ UUID │ PIPELINE_NAME │ PIPELINE_STEP_NAME ┃<br><br>┠────────┼──────────────────────────────────────┼────────────────────────────────┼────────────────────────────┨<br><br>┃ ✅ │ 8cbe671b-9fce-4394-a051-68e001f92765 │ seldon_deployment_pipeline │ seldon_model_deployer_step ┃<br><br>┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛<br><br>$ zenml model-deployer models describe 8cbe671b-9fce-4394-a051-68e001f92765<br><br>Properties of Served Model 8cbe671b-9fce-4394-a051-68e001f92765<br><br>┏━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓<br><br>┃ MODEL SERVICE PROPERTY │ VALUE ┃<br><br>┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨<br><br>┃ MODEL_NAME │ mnist ┃<br><br>┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
384,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: True
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
| Epoch | Step | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_384_cosine_map@100 | dim_64_cosine_map@100 |
|:----------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 0.6667 | 1 | 0.3484 | 0.4016 | 0.4286 | 0.2904 |
| 2.0 | 3 | 0.4266 | 0.4650 | 0.4774 | 0.3927 |
| **2.6667** | **4** | **0.4313** | **0.4691** | **0.4836** | **0.402** |
* The bold row denotes the saved checkpoint.
### Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.3.1+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->