--- base_model: Snowflake/snowflake-arctic-embed-m datasets: [] language: - en library_name: sentence-transformers license: apache-2.0 metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:1490 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: How can I list the 'kubernetes-cluster' resources that are accessible by service connectors in my ZenML workspace? sentences: - 'lly registered orchestrator ``.$ zenml service-connector list-resources --resource-type kubernetes-cluster -e The following ''kubernetes-cluster'' resources can be accessed by service connectors configured in your workspace: ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┓ ┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE │ RESOURCE TYPE │ RESOURCE NAMES ┃ ┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨ ┃ e33c9fac-5daa-48b2-87bb-0187d3782cde │ aws-iam-multi-eu │ 🔶 aws │ 🌀 kubernetes-cluster │ kubeflowmultitenant ┃ ┃ │ │ │ │ zenbox ┃ ┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨ ┃ ed528d5a-d6cb-4fc4-bc52-c3d2d01643e5 │ aws-iam-multi-us │ 🔶 aws │ 🌀 kubernetes-cluster │ zenhacks-cluster ┃ ┠──────────────────────────────────────┼───────────────────────┼────────────────┼───────────────────────┼─────────────────────┨ ┃ 1c54b32a-4889-4417-abbd-42d3ace3d03a │ gcp-sa-multi │ 🔵 gcp │ 🌀 kubernetes-cluster │ zenml-test-cluster ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┛' - 'Name your pipeline runs In the output logs of a pipeline run you will see the name of the run: Pipeline run training_pipeline-2023_05_24-12_41_04_576473 has finished in 3.742s. This name is automatically generated based on the current date and time. To change the name for a run, pass run_name as a parameter to the with_options() method: training_pipeline = training_pipeline.with_options( run_name="custom_pipeline_run_name" training_pipeline() Pipeline run names must be unique, so if you plan to run your pipelines multiple times or run them on a schedule, make sure to either compute the run name dynamically or include one of the following placeholders that ZenML will replace: {{date}} will resolve to the current date, e.g. 2023_02_19 {{time}} will resolve to the current time, e.g. 11_07_09_326492 training_pipeline = training_pipeline.with_options( run_name=f"custom_pipeline_run_name_{{date}}_{{time}}" training_pipeline() Be sure to include the f string prefix to allow for the placeholders to be replaced, as shown in the example above. Without the f prefix, the placeholders will not be replaced. PreviousUsing a custom step invocation ID NextUse failure/success hooks Last updated 19 days ago' - ' a_new_local_stack -o default -a my_artifact_storestack : This is the CLI group that enables interactions with the stacks register: Here we want to register a new stack. Explore other operations withzenml stack --help. a_new_local_stack : This is the unique name that the stack will have. --orchestrator or -o are used to specify which orchestrator to use for the stack --artifact-store or -a are used to specify which artifact store to use for the stack The output for the command should look something like this: Using the default local database. Running with active workspace: ''default'' (repository) Stack ''a_new_local_stack'' successfully registered! You can inspect the stack with the following command: zenml stack describe a_new_local_stack Which will give you an output like this: Stack Configuration ┏━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━┓ ┃ COMPONENT_TYPE │ COMPONENT_NAME ┃ ┠────────────────┼───────────────────┨ ┃ ORCHESTRATOR │ default ┃ ┠────────────────┼───────────────────┨ ┃ ARTIFACT_STORE │ my_artifact_store ┃ ┗━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━┛ ''a_new_local_stack'' stack Stack ''a_new_local_stack'' with id ''...'' is owned by user default and is ''private''. Switch stacks with our VS Code extension If you are using our VS Code extension, you can easily view and switch your stacks by opening the sidebar (click on the ZenML icon). You can then click on the stack you want to switch to as well as view the stack components it''s made up of. Run a pipeline on the new local stack Let''s use the pipeline in our starter project from the previous guide to see it in action. If you have not already, clone the starter template: pip install "zenml[templates,server]" notebook zenml integration install sklearn -y mkdir zenml_starter cd zenml_starter zenml init --template starter --template-with-defaults # Just in case, we install the requirements again pip install -r requirements.txt The starter template is the same as the ZenML quickstart. You can clone it like so:' - source_sentence: How can I explicitly name my model version in ZenML? sentences: - 'strator supports specifying resources in what way.If you''re using an orchestrator which does not support this feature or its underlying infrastructure does not cover your requirements, you can also take a look at step operators which allow you to execute individual steps of ../...your pipeline in environments independent of your orchestrator. Ensure your container is CUDA-enabled To run steps or pipelines on GPUs, it''s crucial to have the necessary CUDA tools installed in the environment. This section will guide you on how to configure your environment to utilize GPU capabilities effectively. Note that these configuration changes are required for the GPU hardware to be properly utilized. If you don''t update the settings, your steps might run, but they will not see any boost in performance from the custom hardware. All steps running on GPU-backed hardware will be executed within a containerized environment, whether you''re using the local Docker orchestrator or a cloud instance of Kubeflow. Therefore, you need to make two amendments to your Docker settings for the relevant steps: 1. Specify a CUDA-enabled parent image in your DockerSettings For complete details, refer to the containerization page that explains how to do this. As an example, if you want to use the latest CUDA-enabled official PyTorch image for your entire pipeline run, you can include the following code: from zenml import pipeline from zenml.config import DockerSettings docker_settings = DockerSettings(parent_image="pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime") @pipeline(settings={"docker": docker_settings}) def my_pipeline(...): ... For TensorFlow, you might use the tensorflow/tensorflow:latest-gpu image, as detailed in the official TensorFlow documentation or their DockerHub overview. 2. Add ZenML as an explicit pip requirement' - 'Reuse Docker builds to speed up Docker build times Avoid building Docker images each time a pipeline runs When using containerized components in your stack, ZenML needs to build Docker images to remotely execute your code. Building Docker images without connecting a git repository includes your step code in the built Docker image. This, however, means that new Docker images will be built and pushed whenever you make changes to any of your source files. One way of skipping Docker builds each time is to pass in the ID of a build as you run the pipeline: my_pipeline = my_pipeline.with_options(build=) or when running a pipeline from the CLI: zenml pipeline run --build= Please note, that this means specifying a custom build when running a pipeline will not run the code on your client machine but will use the code included in the Docker images of the build. As a consequence, even if you make local code changes, reusing a build will always execute the code bundled in the Docker image, rather than the local code. Therefore, if you would like to reuse a Docker build AND make sure your local code changes are also downloaded into the image, you need to connect a git repository. PreviousWhich files are built into the image NextUse code repositories to automate Docker build reuse Last updated 19 days ago' - 'Controlling Model versions Each model can have many versions. Model versions are a way for you to track different iterations of your training process, complete with some extra dashboard and API functionality to support the full ML lifecycle. E.g. Based on your business rules during training, you can associate model version with stages and promote them to production. You have an interface that allows you to link these versions with non-technical artifacts and data, e.g. business data, datasets, or even stages in your process and workflow. Model versions are created implicitly as you are running your machine learning training, so you don''t have to immediately think about this. If you want more control over versions, our API has you covered, with an option to explicitly name your versions. Explicitly name your model version If you want to explicitly name your model version, you can do so by passing in the version argument to the Model object. If you don''t do this, ZenML will automatically generate a version number for you. from zenml import Model, step, pipeline model= Model( name="my_model", version="1.0.5" # The step configuration will take precedence over the pipeline @step(model=model) def svc_trainer(...) -> ...: ... # This configures it for all steps within the pipeline @pipeline(model=model) def training_pipeline( ... ): # training happens here Here we are specifically setting the model configuration for a particular step or for the pipeline as a whole. Please note in the above example if the model version exists, it is automatically associated with the pipeline and becomes active in the pipeline context. Therefore, a user should be careful and intentional as to whether you want to create a new pipeline, or fetch an existing one. See below for an example of fetching a model from an existing version/stage. Fetching model versions by stage' - source_sentence: What are the different roles available for users in an organization within ZenML Pro? sentences: - 'User Management In ZenML Pro, there is a slightly different entity hierarchy as compared to the open-source ZenML framework. This document walks you through the key differences and new concepts that are pro-only. Organizations, Tenants, and Roles ZenML Pro arranges various aspects of your work experience around the concept of an Organization. This is the top-most level structure within the ZenML Cloud environment. Generally, an organization contains a group of users and one or more tenants. Tenants are individual, isolated deployments of the ZenML server. Every user in an organization has a distinct role. Each role configures what they can view, modify, and their level of involvement in collaborative tasks. A role thus helps determine the level of access that a user has within an organization. The admin has all permissions on an organization. They are allowed to add members, adjust the billing information and assign roles. The editor can still fully manage tenants and members but is not allowed to access the subscription information or delete the organization. The viewer Role allows you to allow users to access the tenants within the organization with only view permissions. Inviting Team Members Inviting users to your organization to work on the organization''s tenants is easy. Simply click Add Member in the Organization settings, and give them an initial Role. The User will be sent an invitation email. If a user is part of an organization, they can utilize their login on all tenants they have authority to access. PreviousZenML SaaS NextStarter guide Last updated 12 days ago' - ' more information. Get the last run of a pipelineTo access the most recent run of a pipeline, you can either use the last_run property or access it through the runs list: last_run = pipeline_model.last_run # OR: pipeline_model.runs[0] If your most recent runs have failed, and you want to find the last run that has succeeded, you can use the last_successful_run property instead. Get the latest run from a pipeline Calling a pipeline executes it and then returns the response of the freshly executed run. run = training_pipeline() The run that you get back is the model stored in the ZenML database at the point of the method call. This means the pipeline run is still initializing and no steps have been run. To get the latest state can get a refreshed version from the client: from zenml.client import Client Client().get_pipeline_run(run.id) to get a refreshed version Get a run via the client If you already know the exact run that you want to fetch (e.g., from looking at the dashboard), you can use the Client.get_pipeline_run() method to fetch the run directly without having to query the pipeline first: from zenml.client import Client pipeline_run = Client().get_pipeline_run("first_pipeline-2023_06_20-16_20_13_274466") Similar to pipelines, you can query runs by either ID, name, or name prefix, and you can also discover runs through the Client or CLI via the Client.list_pipeline_runs() or zenml pipeline runs list commands. Run information Each run has a collection of useful information which can help you reproduce your runs. In the following, you can find a list of some of the most useful pipeline run information, but there is much more available. See the PipelineRunResponse definition for a comprehensive list. Status The status of a pipeline run. There are five possible states: initialized, failed, completed, running, and cached. status = run.status Configuration' - 'ner(gamma=gamma, X_train=X_train, y_train=y_train)if __name__ == "__main__": first_pipeline() python run.py ... Registered pipeline first_pipeline (version 2). ... This will now create a single run for version 2 of the pipeline called first_pipeline. PreviousHyperparameter tuning NextAccess secrets in a step Last updated 15 days ago' - source_sentence: How can I check which GCP Service Connectors can access a GCS bucket in my ZenML deployment? sentences: - 'dashboard. Warning! Usage in remote orchestratorsThe current ZenML version has a limitation in its base Docker image that requires a workaround for all pipelines using Deepchecks with a remote orchestrator (e.g. Kubeflow , Vertex). The limitation being that the base Docker image needs to be extended to include binaries that are required by opencv2, which is a package that Deepchecks requires. While these binaries might be available on most operating systems out of the box (and therefore not a problem with the default local orchestrator), we need to tell ZenML to add them to the containerization step when running in remote settings. Here is how: First, create a file called deepchecks-zenml.Dockerfile and place it on the same level as your runner script (commonly called run.py). The contents of the Dockerfile are as follows: ARG ZENML_VERSION=0.20.0 FROM zenmldocker/zenml:${ZENML_VERSION} AS base RUN apt-get update RUN apt-get install ffmpeg libsm6 libxext6 -y Then, place the following snippet above your pipeline definition. Note that the path of the dockerfile are relative to where the pipeline definition file is. Read the containerization guide for more details: import zenml from zenml import pipeline from zenml.config import DockerSettings from pathlib import Path import sys docker_settings = DockerSettings( dockerfile="deepchecks-zenml.Dockerfile", build_options={ "buildargs": { "ZENML_VERSION": f"{zenml.__version__}" }, }, @pipeline(settings={"docker": docker_settings}) def my_pipeline(...): # same code as always ... From here on, you can continue to use the deepchecks integration as is explained below. The Deepchecks standard steps ZenML wraps the Deepchecks functionality for tabular data in the form of four standard steps: DeepchecksDataIntegrityCheckStep: use it in your pipelines to run data integrity tests on a single dataset DeepchecksDataDriftCheckStep: use it in your pipelines to run data drift tests on two datasets as input: target and reference.' - ' gs://zenml-core_cloudbuild ┃┃ │ gs://zenml-datasets ┃ ┃ │ gs://zenml-internal-artifact-store ┃ ┃ │ gs://zenml-kubeflow-artifact-store ┃ ┃ │ gs://zenml-project-time-series-bucket ┃ ┠───────────────────────┼─────────────────────────────────────────────────┨ ┃ 🌀 kubernetes-cluster │ zenml-test-cluster ┃ ┠───────────────────────┼─────────────────────────────────────────────────┨ ┃ 🐳 docker-registry │ gcr.io/zenml-core ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ Long-lived credentials (API keys, account keys) This is the magic formula of authentication methods. When paired with another ability, such as automatically generating short-lived API tokens, or impersonating accounts or assuming roles, this is the ideal authentication mechanism to use, particularly when using ZenML in production and when sharing results with other members of your ZenML team. As a general best practice, but implemented particularly well for cloud platforms, account passwords are never directly used as a credential when authenticating to the cloud platform APIs. There is always a process in place that exchanges the account/password credential for another type of long-lived credential: AWS uses the aws configure CLI command GCP offers the gcloud auth application-default login CLI commands Azure provides the az login CLI command None of your original login information is stored on your local machine or used to access workloads. Instead, an API key, account key or some other form of intermediate credential is generated and stored on the local host and used to authenticate to remote cloud service APIs.' - ' should pick the one that best fits your use case.If you already have one or more GCP Service Connectors configured in your ZenML deployment, you can check which of them can be used to access the GCS bucket you want to use for your GCS Artifact Store by running e.g.: zenml service-connector list-resources --resource-type gcs-bucket Example Command Output ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ CONNECTOR ID │ CONNECTOR NAME │ CONNECTOR TYPE │ RESOURCE TYPE │ RESOURCE NAMES ┃ ┠──────────────────────────────────────┼─────────────────────┼────────────────┼───────────────┼─────────────────────────────────────────────────┨ ┃ 7f0c69ba-9424-40ae-8ea6-04f35c2eba9d │ gcp-user-account │ 🔵 gcp │ 📦 gcs-bucket │ gs://zenml-bucket-sl ┃ ┃ │ │ │ │ gs://zenml-core.appspot.com ┃ ┃ │ │ │ │ gs://zenml-core_cloudbuild ┃ ┃ │ │ │ │ gs://zenml-datasets ┃ ┃ │ │ │ │ gs://zenml-internal-artifact-store ┃ ┃ │ │ │ │ gs://zenml-kubeflow-artifact-store ┃ ┠──────────────────────────────────────┼─────────────────────┼────────────────┼───────────────┼─────────────────────────────────────────────────┨ ┃ 2a0bec1b-9787-4bd7-8d4a-9a47b6f61643 │ gcs-zenml-bucket-sl │ 🔵 gcp │ 📦 gcs-bucket │ gs://zenml-bucket-sl ┃' - source_sentence: Is it possible to update the local AWS CLI configuration with credentials extracted from the AWS Service Connector? sentences: - '36a885: Pull complete c9c0554c8e6a: Pull completebacdcd847a66: Pull complete 482033770844: Pull complete Digest: sha256:bf2cc3895e70dfa1ee1cd90bbfa599fa4cd8df837e27184bac1ce1cc239ecd3f Status: Downloaded newer image for 715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest 715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest It is also possible to update the local AWS CLI configuration with credentials extracted from the AWS Service Connector: zenml service-connector login aws-session-token --resource-type aws-generic Example Command Output Configured local AWS SDK profile ''zenml-c0f8e857''. The ''aws-session-token'' AWS Service Connector connector was used to successfully configure the local Generic AWS resource client/SDK. A new profile is created in the local AWS CLI configuration holding the credentials. It can be used to access AWS resources and services, e.g.: aws --profile zenml-c0f8e857 s3 ls Stack Components use The S3 Artifact Store Stack Component can be connected to a remote AWS S3 bucket through an AWS Service Connector. The AWS Service Connector can also be used with any Orchestrator or Model Deployer stack component flavor that relies on Kubernetes clusters to manage workloads. This allows EKS Kubernetes container workloads to be managed without the need to configure and maintain explicit AWS or Kubernetes kubectl configuration contexts and credentials in the target environment and in the Stack Component. Similarly, Container Registry Stack Components can be connected to an ECR Container Registry through an AWS Service Connector. This allows container images to be built and published to ECR container registries without the need to configure explicit AWS credentials in the target environment or the Stack Component. End-to-end examples' - ' ┃┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ RESOURCE TYPES │ 🌀 kubernetes-cluster ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ RESOURCE NAME │ arn:aws:eks:us-east-1:715803424590:cluster/zenhacks-cluster ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ SECRET ID │ ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ SESSION DURATION │ N/A ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ EXPIRES IN │ 11h59m57s ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ OWNER │ default ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ WORKSPACE │ default ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ SHARED │ ➖ ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ CREATED_AT │ 2023-06-16 10:17:46.931091 ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────┨ ┃ UPDATED_AT │ 2023-06-16 10:17:46.931094 ┃ ┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ Configuration' - 'e --authentication_secret. For example, you''d run:zenml secret create argilla_secrets --api_key="" (Visit the Argilla documentation and interface to obtain your API key.) Then register your annotator with ZenML: zenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets When using a deployed instance of Argilla, the instance URL must be specified without any trailing / at the end. If you are using a Hugging Face Spaces instance and its visibility is set to private, you must also set the extra_headers parameter which would include a Hugging Face token. For example: zenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets --instance_url="https://[your-owner-name]-[your_space_name].hf.space" --extra_headers="{"Authorization": f"Bearer {}"}" Finally, add all these components to a stack and set it as your active stack. For example: zenml stack copy default annotation # this must be done separately so that the other required stack components are first registered zenml stack update annotation -an zenml stack set annotation # optionally also zenml stack describe Now if you run a simple CLI command like zenml annotator dataset list this should work without any errors. You''re ready to use your annotator in your ML workflow! How do you use it? ZenML supports access to your data and annotations via the zenml annotator ... CLI command. We have also implemented an interface to some of the common Argilla functionality via the ZenML SDK. You can access information about the datasets you''re using with the zenml annotator dataset list. To work on annotation for a particular dataset, you can run zenml annotator dataset annotate . What follows is an overview of some key components to the Argilla integration and how it can be used. Argilla Annotator Stack Component' model-index: - name: zenml/finetuned-snowflake-arctic-embed-m results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy@1 value: 0.29518072289156627 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.6204819277108434 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6927710843373494 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7891566265060241 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.29518072289156627 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.20682730923694775 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.13855421686746985 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.0789156626506024 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.29518072289156627 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.6204819277108434 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6927710843373494 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7891566265060241 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5524302146116403 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4758486326257412 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4836255621339311 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.28313253012048195 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5963855421686747 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6807228915662651 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7771084337349398 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.28313253012048195 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.19879518072289157 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.136144578313253 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07771084337349396 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.28313253012048195 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5963855421686747 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6807228915662651 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7771084337349398 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5376005319054157 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.46014534327787354 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4690725321460052 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.2710843373493976 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5301204819277109 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.5963855421686747 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7409638554216867 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.2710843373493976 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.17670682730923692 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.11927710843373492 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07409638554216866 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.2710843373493976 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5301204819277109 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.5963855421686747 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7409638554216867 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.49831034220322973 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4218158347676423 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.43128822737879013 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.26506024096385544 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.4819277108433735 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.5662650602409639 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.6566265060240963 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.26506024096385544 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.1606425702811245 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.11325301204819276 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.06566265060240963 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.26506024096385544 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.4819277108433735 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.5662650602409639 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.6566265060240963 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.45413104746517285 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.38985465672212666 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4019553541721889 name: Cosine Map@100 --- # zenml/finetuned-snowflake-arctic-embed-m This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m") # Run inference sentences = [ 'Is it possible to update the local AWS CLI configuration with credentials extracted from the AWS Service Connector?', "36a885: Pull complete\n\nc9c0554c8e6a: Pull completebacdcd847a66: Pull complete\n\n482033770844: Pull complete\n\nDigest: sha256:bf2cc3895e70dfa1ee1cd90bbfa599fa4cd8df837e27184bac1ce1cc239ecd3f\n\nStatus: Downloaded newer image for 715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest\n\n715803424590.dkr.ecr.us-east-1.amazonaws.com/zenml-server:latest\n\nIt is also possible to update the local AWS CLI configuration with credentials extracted from the AWS Service Connector:\n\nzenml service-connector login aws-session-token --resource-type aws-generic\n\nExample Command Output\n\nConfigured local AWS SDK profile 'zenml-c0f8e857'.\n\nThe 'aws-session-token' AWS Service Connector connector was used to successfully configure the local Generic AWS resource client/SDK.\n\nA new profile is created in the local AWS CLI configuration holding the credentials. It can be used to access AWS resources and services, e.g.:\n\naws --profile zenml-c0f8e857 s3 ls\n\nStack Components use\n\nThe S3 Artifact Store Stack Component can be connected to a remote AWS S3 bucket through an AWS Service Connector.\n\nThe AWS Service Connector can also be used with any Orchestrator or Model Deployer stack component flavor that relies on Kubernetes clusters to manage workloads. This allows EKS Kubernetes container workloads to be managed without the need to configure and maintain explicit AWS or Kubernetes kubectl configuration contexts and credentials in the target environment and in the Stack Component.\n\nSimilarly, Container Registry Stack Components can be connected to an ECR Container Registry through an AWS Service Connector. This allows container images to be built and published to ECR container registries without the need to configure explicit AWS credentials in the target environment or the Stack Component.\n\nEnd-to-end examples", 'e --authentication_secret. For example, you\'d run:zenml secret create argilla_secrets --api_key=""\n\n(Visit the Argilla documentation and interface to obtain your API key.)\n\nThen register your annotator with ZenML:\n\nzenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets\n\nWhen using a deployed instance of Argilla, the instance URL must be specified without any trailing / at the end. If you are using a Hugging Face Spaces instance and its visibility is set to private, you must also set the extra_headers parameter which would include a Hugging Face token. For example:\n\nzenml annotator register argilla --flavor argilla --authentication_secret=argilla_secrets --instance_url="https://[your-owner-name]-[your_space_name].hf.space" --extra_headers="{"Authorization": f"Bearer {}"}"\n\nFinally, add all these components to a stack and set it as your active stack. For example:\n\nzenml stack copy default annotation\n\n# this must be done separately so that the other required stack components are first registered\n\nzenml stack update annotation -an \n\nzenml stack set annotation\n\n# optionally also\n\nzenml stack describe\n\nNow if you run a simple CLI command like zenml annotator dataset list this should work without any errors. You\'re ready to use your annotator in your ML workflow!\n\nHow do you use it?\n\nZenML supports access to your data and annotations via the zenml annotator ... CLI command. We have also implemented an interface to some of the common Argilla functionality via the ZenML SDK.\n\nYou can access information about the datasets you\'re using with the zenml annotator dataset list. To work on annotation for a particular dataset, you can run zenml annotator dataset annotate . What follows is an overview of some key components to the Argilla integration and how it can be used.\n\nArgilla Annotator Stack Component', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_384` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.2952 | | cosine_accuracy@3 | 0.6205 | | cosine_accuracy@5 | 0.6928 | | cosine_accuracy@10 | 0.7892 | | cosine_precision@1 | 0.2952 | | cosine_precision@3 | 0.2068 | | cosine_precision@5 | 0.1386 | | cosine_precision@10 | 0.0789 | | cosine_recall@1 | 0.2952 | | cosine_recall@3 | 0.6205 | | cosine_recall@5 | 0.6928 | | cosine_recall@10 | 0.7892 | | cosine_ndcg@10 | 0.5524 | | cosine_mrr@10 | 0.4758 | | **cosine_map@100** | **0.4836** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.2831 | | cosine_accuracy@3 | 0.5964 | | cosine_accuracy@5 | 0.6807 | | cosine_accuracy@10 | 0.7771 | | cosine_precision@1 | 0.2831 | | cosine_precision@3 | 0.1988 | | cosine_precision@5 | 0.1361 | | cosine_precision@10 | 0.0777 | | cosine_recall@1 | 0.2831 | | cosine_recall@3 | 0.5964 | | cosine_recall@5 | 0.6807 | | cosine_recall@10 | 0.7771 | | cosine_ndcg@10 | 0.5376 | | cosine_mrr@10 | 0.4601 | | **cosine_map@100** | **0.4691** | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.2711 | | cosine_accuracy@3 | 0.5301 | | cosine_accuracy@5 | 0.5964 | | cosine_accuracy@10 | 0.741 | | cosine_precision@1 | 0.2711 | | cosine_precision@3 | 0.1767 | | cosine_precision@5 | 0.1193 | | cosine_precision@10 | 0.0741 | | cosine_recall@1 | 0.2711 | | cosine_recall@3 | 0.5301 | | cosine_recall@5 | 0.5964 | | cosine_recall@10 | 0.741 | | cosine_ndcg@10 | 0.4983 | | cosine_mrr@10 | 0.4218 | | **cosine_map@100** | **0.4313** | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:----------| | cosine_accuracy@1 | 0.2651 | | cosine_accuracy@3 | 0.4819 | | cosine_accuracy@5 | 0.5663 | | cosine_accuracy@10 | 0.6566 | | cosine_precision@1 | 0.2651 | | cosine_precision@3 | 0.1606 | | cosine_precision@5 | 0.1133 | | cosine_precision@10 | 0.0657 | | cosine_recall@1 | 0.2651 | | cosine_recall@3 | 0.4819 | | cosine_recall@5 | 0.5663 | | cosine_recall@10 | 0.6566 | | cosine_ndcg@10 | 0.4541 | | cosine_mrr@10 | 0.3899 | | **cosine_map@100** | **0.402** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 1,490 training samples * Columns: positive and anchor * Approximate statistics based on the first 1000 samples: | | positive | anchor | |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details |
  • min: 10 tokens
  • mean: 21.19 tokens
  • max: 64 tokens
|
  • min: 21 tokens
  • mean: 374.24 tokens
  • max: 512 tokens
| * Samples: | positive | anchor | |:---------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | How do I use it to start the local Airflow server for ZenML? | use it

To use the Airflow orchestrator, we need:The ZenML airflow integration installed. If you haven't done so, runCopyzenml integration install airflow

Docker installed and running.

The orchestrator registered and part of our active stack:

zenml orchestrator register \

--flavor=airflow \

--local=True # set this to `False` if using a remote Airflow deployment

# Register and activate a stack with the new orchestrator

zenml stack register -o ... --set

In the local case, we need to reinstall in a certain way for the local Airflow server:

pip install "apache-airflow-providers-docker<3.8.0" "apache-airflow==2.4.0" "pendulum<3.0.0" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.4.0/constraints-3.9.txt"

Please make sure to replace 3.9 with your Python (major) version in the constraints file URL given above.

Once that is installed, we can start the local Airflow server by running the following command in your terminal. See further below on an alternative way to set up the Airflow server manually since the zenml stack up command is deprecated.

zenml stack up

This command will start up an Airflow server on your local machine that's running in the same Python environment that you used to provision it. When it is finished, it will print a username and password which you can use to log in to the Airflow UI here.

As long as you didn't configure any custom value for the dag_output_dir attribute of your orchestrator, running a pipeline locally is as simple as calling:

python file_that_runs_a_zenml_pipeline.py

This call will produce a .zip file containing a representation of your ZenML pipeline to the Airflow DAGs directory. From there, the local Airflow server will load it and run your pipeline (It might take a few seconds until the pipeline shows up in the Airflow UI). | | What are the best practices for various authentication methods implemented by Service Connectors? | Security best practices

Best practices concerning the various authentication methods implemented by Service Connectors.

Service Connector Types, especially those targeted at cloud providers, offer a plethora of authentication methods matching those supported by remote cloud platforms. While there is no single authentication standard that unifies this process, there are some patterns that are easily identifiable and can be used as guidelines when deciding which authentication method to use to configure a Service Connector.

This section explores some of those patterns and gives some advice regarding which authentication methods are best suited for your needs.

This section may require some general knowledge about authentication and authorization to be properly understood. We tried to keep it simple and limit ourselves to talking about high-level concepts, but some areas may get a bit too technical.

Username and password

The key takeaway is this: you should avoid using your primary account password as authentication credentials as much as possible. If there are alternative authentication methods that you can use or other types of credentials (e.g. session tokens, API keys, API tokens), you should always try to use those instead.

Ultimately, if you have no choice, be cognizant of the third parties you share your passwords with. If possible, they should never leave the premises of your local host or development environment.

This is the typical authentication method that uses a username or account name plus the associated password. While this is the de facto method used to log in with web consoles and local CLIs, this is the least secure of all authentication methods and never something you want to share with other members of your team or organization or use to authenticate automated workloads. | | How can I interact with a model deployer after deploying a model in ZenML? | eline_step_name="huggingface_model_deployer_step",model_name="LLAMA-7B",

model_uri="s3://zenprojects/huggingface_model_deployer_step/output/884/huggingface",

revision="main",

task="text-classification",

region="us-east-1",

vendor="aws",

token="huggingface_token",

namespace="zenml-workloads",

endpoint_type="public",

print(f"Model server {service.config['model_name']} is deployed at {service.status['prediction_url']}")

How to Interact with a model deployer after deployment?

When a Model Deployer is part of the active ZenML Stack, it is also possible to interact with it from the CLI to list, start, stop, or delete the model servers that is managed:

$ zenml model-deployer models list

┏━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ STATUS │ UUID │ PIPELINE_NAME │ PIPELINE_STEP_NAME ┃

┠────────┼──────────────────────────────────────┼────────────────────────────────┼────────────────────────────┨

┃ ✅ │ 8cbe671b-9fce-4394-a051-68e001f92765 │ seldon_deployment_pipeline │ seldon_model_deployer_step ┃

┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

$ zenml model-deployer models describe 8cbe671b-9fce-4394-a051-68e001f92765

Properties of Served Model 8cbe671b-9fce-4394-a051-68e001f92765

┏━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓

┃ MODEL SERVICE PROPERTY │ VALUE ┃

┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨

┃ MODEL_NAME │ mnist ┃

┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨ | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: True - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_384_cosine_map@100 | dim_64_cosine_map@100 | |:----------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | 0.6667 | 1 | 0.3484 | 0.4016 | 0.4286 | 0.2904 | | 2.0 | 3 | 0.4266 | 0.4650 | 0.4774 | 0.3927 | | **2.6667** | **4** | **0.4313** | **0.4691** | **0.4836** | **0.402** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.14 - Sentence Transformers: 3.0.1 - Transformers: 4.41.2 - PyTorch: 2.3.1+cu121 - Accelerate: 0.31.0 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```