--- base_model: Snowflake/snowflake-arctic-embed-m datasets: [] language: - en library_name: sentence-transformers license: apache-2.0 metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:1490 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: How does ZenML facilitate connecting your deployment to various cloud providers and infrastructure services? sentences: - 'πŸ”ŒConnect services (AWS, GCP, Azure, K8s etc) Connect your ZenML deployment to a cloud provider and other infrastructure services and resources. A production-grade MLOps platform involves interactions between a diverse combination of third-party libraries and external services sourced from various different vendors. One of the most daunting hurdles in building and operating an MLOps platform composed of multiple components is configuring and maintaining uninterrupted and secured access to the infrastructure resources and services that it consumes. In layman''s terms, your pipeline code needs to "connect" to a handful of different services to run successfully and do what it''s designed to do. For example, it might need to connect to a private AWS S3 bucket to read and store artifacts, a Kubernetes cluster to execute steps with Kubeflow or Tekton, and a private GCR container registry to build and store container images. ZenML makes this possible by allowing you to configure authentication information and credentials embedded directly into your Stack Components, but this doesn''t scale well when you have more than a few Stack Components and has many other disadvantages related to usability and security. Gaining access to infrastructure resources and services requires knowledge about the different authentication and authorization mechanisms and involves configuring and maintaining valid credentials. It gets even more complicated when these different services need to access each other. For instance, the Kubernetes container running your pipeline step needs access to the S3 bucket to store artifacts or needs to access a cloud service like AWS SageMaker, VertexAI, or AzureML to run a CPU/GPU intensive task like training a model.' - ' ┃┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ ID β”‚ e316bcb3-6659-467b-81e5-5ec25bfd36b0 ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ NAME β”‚ aws-sts-token ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ TYPE β”‚ πŸ”Ά aws ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ AUTH METHOD β”‚ sts-token ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ RESOURCE TYPES β”‚ πŸ”Ά aws-generic, πŸ“¦ s3-bucket, πŸŒ€ kubernetes-cluster, 🐳 docker-registry ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ RESOURCE NAME β”‚ ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ SECRET ID β”‚ 971318c9-8db9-4297-967d-80cda070a121 ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ SESSION DURATION β”‚ N/A ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ EXPIRES IN β”‚ 11h58m17s ┃ ┠──────────────────┼─────────────────────────────────────────────────────────────────────────┨ ┃ OWNER β”‚ default ┃' - 'io ┃ ┗━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛If you already have one or more Docker Service Connectors configured in your ZenML deployment, you can check which of them can be used to access the container registry you want to use for your Default Container Registry by running e.g.: zenml service-connector list-resources --connector-type docker --resource-id Example Command Output $ zenml service-connector list-resources --connector-type docker --resource-id docker.io The resource with name ''docker.io'' can be accessed by ''docker'' service connectors configured in your workspace: ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓ ┃ CONNECTOR ID β”‚ CONNECTOR NAME β”‚ CONNECTOR TYPE β”‚ RESOURCE TYPE β”‚ RESOURCE NAMES ┃ ┠──────────────────────────────────────┼────────────────┼────────────────┼────────────────────┼────────────────┨ ┃ cf55339f-dbc8-4ee6-862e-c25aff411292 β”‚ dockerhub β”‚ 🐳 docker β”‚ 🐳 docker-registry β”‚ docker.io ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛ After having set up or decided on a Docker Service Connector to use to connect to the target container registry, you can register the Docker Container Registry as follows: # Register the container registry and reference the target registry URI zenml container-registry register -f default \ --uri= # Connect the container registry to the target registry via a Docker Service Connector zenml container-registry connect -i A non-interactive version that connects the Default Container Registry to a target registry through a Docker Service Connector: zenml container-registry connect --connector Example Command Output $ zenml container-registry connect dockerhub --connector dockerhub' - source_sentence: How can I configure the orchestrator settings for each cloud provider in ZenML? sentences: - 'kip scoping its Resource Type during registration.a multi-instance Service Connector instance can be configured once and used to gain access to multiple resources of the same type, each identifiable by a Resource Name. Not all types of connectors and not all types of resources support multiple instances. Some Service Connectors Types like the generic Kubernetes and Docker connector types only allow single-instance configurations: a Service Connector instance can only be used to access a single Kubernetes cluster and a single Docker registry. To configure a multi-instance Service Connector, you can simply skip scoping its Resource Name during registration. The following is an example of configuring a multi-type AWS Service Connector instance capable of accessing multiple AWS resources of different types: zenml service-connector register aws-multi-type --type aws --auto-configure Example Command Output β ‹ Registering service connector ''aws-multi-type''... Successfully registered service connector `aws-multi-type` with access to the following resources: ┏━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ RESOURCE TYPE β”‚ RESOURCE NAMES ┃ ┠───────────────────────┼──────────────────────────────────────────────┨ ┃ πŸ”Ά aws-generic β”‚ us-east-1 ┃ ┠───────────────────────┼──────────────────────────────────────────────┨ ┃ πŸ“¦ s3-bucket β”‚ s3://aws-ia-mwaa-715803424590 ┃ ┃ β”‚ s3://zenfiles ┃ ┃ β”‚ s3://zenml-demos ┃ ┃ β”‚ s3://zenml-generative-chat ┃ ┃ β”‚ s3://zenml-public-datasets ┃ ┃ β”‚ s3://zenml-public-swagger-spec ┃ ┠───────────────────────┼──────────────────────────────────────────────┨ ┃ πŸŒ€ kubernetes-cluster β”‚ zenhacks-cluster ┃' - 'ister -a ... --setWhen you register the Azure Artifact Store, you can create a ZenML Secret to store a variety of Azure credentials and then reference it in the Artifact Store configuration: to use an Azure storage account key , set account_name to your account name and one of account_key or sas_token to the Azure key or SAS token value as attributes in the ZenML secret to use an Azure storage account key connection string , configure the connection_string attribute in the ZenML secret to your Azure Storage Key connection string to use Azure Service Principal credentials , create an Azure Service Principal and then set account_name to your account name and client_id, client_secret and tenant_id to the client ID, secret and tenant ID of your service principal in the ZenML secret This method has some advantages over the implicit authentication method: you don''t need to install and configure the Azure CLI on your host you don''t need to care about enabling your other stack components (orchestrators, step operators and model deployers) to have access to the artifact store through Azure Managed Identities you can combine the Azure artifact store with other stack components that are not running in Azure Configuring Azure credentials in a ZenML secret and then referencing them in the Artifact Store configuration could look like this: # Store the Azure storage account key in a ZenML secret zenml secret create az_secret \ --account_name='''' \ --account_key='''' # or if you want to use a connection string zenml secret create az_secret \ --connection_string='''' # or if you want to use Azure ServicePrincipal credentials zenml secret create az_secret \ --account_name='''' \ --tenant_id='''' \ --client_id='''' \ --client_secret=''''' - '. If not set, the cluster will not be autostopped.down: Tear down the cluster after all jobs finish (successfully or abnormally). If idle_minutes_to_autostop is also set, the cluster will be torn down after the specified idle time. Note that if errors occur during provisioning/data syncing/setting up, the cluster will not be torn down for debugging purposes. stream_logs: If True, show the logs in the terminal as they are generated while the cluster is running. docker_run_args: Additional arguments to pass to the docker run command. For example, [''--gpus=all''] to use all GPUs available on the VM. The following code snippets show how to configure the orchestrator settings for each cloud provider: Code Example: from zenml.integrations.skypilot_aws.flavors.skypilot_orchestrator_aws_vm_flavor import SkypilotAWSOrchestratorSettings skypilot_settings = SkypilotAWSOrchestratorSettings( cpus="2", memory="16", accelerators="V100:2", accelerator_args={"tpu_vm": True, "runtime_version": "tpu-vm-base"}, use_spot=True, spot_recovery="recovery_strategy", region="us-west-1", zone="us-west1-a", image_id="ami-1234567890abcdef0", disk_size=100, disk_tier="high", cluster_name="my_cluster", retry_until_up=True, idle_minutes_to_autostop=60, down=True, stream_logs=True docker_run_args=["--gpus=all"] @pipeline( settings={ "orchestrator.vm_aws": skypilot_settings Code Example: from zenml.integrations.skypilot_gcp.flavors.skypilot_orchestrator_gcp_vm_flavor import SkypilotGCPOrchestratorSettings skypilot_settings = SkypilotGCPOrchestratorSettings( cpus="2", memory="16", accelerators="V100:2", accelerator_args={"tpu_vm": True, "runtime_version": "tpu-vm-base"}, use_spot=True, spot_recovery="recovery_strategy", region="us-west1", zone="us-west1-a", image_id="ubuntu-pro-2004-focal-v20231101", disk_size=100, disk_tier="high", cluster_name="my_cluster", retry_until_up=True, idle_minutes_to_autostop=60, down=True, stream_logs=True @pipeline( settings={ "orchestrator.vm_gcp": skypilot_settings' - source_sentence: What command do you use to create the resources after setting up the roleRef for a Kubernetes cluster? sentences: - 'pace: spark-namespace roleRef: kind: ClusterRolename: edit apiGroup: rbac.authorization.k8s.io --- And then execute the following command to create the resources: aws eks --region=$REGION update-kubeconfig --name=$EKS_CLUSTER_NAME kubectl create -f rbac.yaml Lastly, note down the namespace and the name of the service account since you will need them when registering the stack component in the next step. How to use it To use the KubernetesSparkStepOperator, you need: the ZenML spark integration. If you haven''t installed it already, runCopyzenml integration install spark Docker installed and running. A remote artifact store as part of your stack. A remote container registry as part of your stack. A Kubernetes cluster deployed. We can then register the step operator and use it in our active stack: zenml step-operator register spark_step_operator \ --flavor=spark-kubernetes \ --master=k8s://$EKS_API_SERVER_ENDPOINT \ --namespace= \ --service_account= # Register the stack zenml stack register spark_stack \ o default \ s spark_step_operator \ a spark_artifact_store \ c spark_container_registry \ i local_builder \ --set Once you added the step operator to your active stack, you can use it to execute individual steps of your pipeline by specifying it in the @step decorator as follows: from zenml import step @step(step_operator=) def step_on_spark(...) -> ...: """Some step that should run with Spark on Kubernetes.""" ... After successfully running any step with a KubernetesSparkStepOperator, you should be able to see that a Spark driver pod was created in your cluster for each pipeline step when running kubectl get pods -n $KUBERNETES_NAMESPACE. Instead of hardcoding a step operator name, you can also use the Client to dynamically use the step operator of your active stack: from zenml.client import Client step_operator = Client().active_stack.step_operator @step(step_operator=step_operator.name)' - 'et_historical_features(entity_dict, features) ...Note that ZenML''s use of Pydantic to serialize and deserialize inputs stored in the ZenML metadata means that we are limited to basic data types. Pydantic cannot handle Pandas DataFrames, for example, or datetime values, so in the above code you can see that we have to convert them at various points. For more information and a full list of configurable attributes of the Feast feature store, check out the SDK Docs . PreviousFeature Stores NextDevelop a Custom Feature Store Last updated 8 days ago' - 'to get a quick global overview of our performance.# passing the results from all our previous evaluation steps @step(enable_cache=False) def visualize_evaluation_results( small_retrieval_eval_failure_rate: float, small_retrieval_eval_failure_rate_reranking: float, full_retrieval_eval_failure_rate: float, full_retrieval_eval_failure_rate_reranking: float, failure_rate_bad_answers: float, failure_rate_bad_immediate_responses: float, failure_rate_good_responses: float, average_toxicity_score: float, average_faithfulness_score: float, average_helpfulness_score: float, average_relevance_score: float, ) -> Optional[Image.Image]: """Visualizes the evaluation results.""" step_context = get_step_context() pipeline_run_name = step_context.pipeline_run.name normalized_scores = [ score / 20 for score in [ small_retrieval_eval_failure_rate, small_retrieval_eval_failure_rate_reranking, full_retrieval_eval_failure_rate, full_retrieval_eval_failure_rate_reranking, failure_rate_bad_answers, scores = normalized_scores + [ failure_rate_bad_immediate_responses, failure_rate_good_responses, average_toxicity_score, average_faithfulness_score, average_helpfulness_score, average_relevance_score, labels = [ "Small Retrieval Eval Failure Rate", "Small Retrieval Eval Failure Rate Reranking", "Full Retrieval Eval Failure Rate", "Full Retrieval Eval Failure Rate Reranking", "Failure Rate Bad Answers", "Failure Rate Bad Immediate Responses", "Failure Rate Good Responses", "Average Toxicity Score", "Average Faithfulness Score", "Average Helpfulness Score", "Average Relevance Score", # Create a new figure and axis fig, ax = plt.subplots(figsize=(10, 6)) # Plot the horizontal bar chart y_pos = np.arange(len(labels)) ax.barh(y_pos, scores, align="center") ax.set_yticks(y_pos) ax.set_yticklabels(labels) ax.invert_yaxis() # Labels read top-to-bottom ax.set_xlabel("Score") ax.set_xlim(0, 5) ax.set_title(f"Evaluation Metrics for {pipeline_run_name}") # Adjust the layout plt.tight_layout()' - source_sentence: What is the command to register and connect a Vertex AI Orchestrator Stack Component to the target GCP project using ZenML? sentences: - 'ggingFaceModelDeployer.get_active_model_deployer()# fetch existing services with same pipeline name, step name and model name existing_services = model_deployer.find_model_server( pipeline_name=pipeline_name, pipeline_step_name=pipeline_step_name, model_name=model_name, running=running, if not existing_services: raise RuntimeError( f"No Hugging Face inference endpoint deployed by step " f"''{pipeline_step_name}'' in pipeline ''{pipeline_name}'' with name " f"''{model_name}'' is currently running." return existing_services[0] # Use the service for inference @step def predictor( service: HuggingFaceDeploymentService, data: str ) -> Annotated[str, "predictions"]: """Run a inference request against a prediction service""" prediction = service.predict(data) return prediction @pipeline def huggingface_deployment_inference_pipeline( pipeline_name: str, pipeline_step_name: str = "huggingface_model_deployer_step", ): inference_data = ... model_deployment_service = prediction_service_loader( pipeline_name=pipeline_name, pipeline_step_name=pipeline_step_name, predictions = predictor(model_deployment_service, inference_data) For more information and a full list of configurable attributes of the Hugging Face Model Deployer, check out the SDK Docs. PreviousBentoML NextDevelop a Custom Model Deployer Last updated 15 days ago' - 'Set up CI/CD Managing the lifecycle of a ZenML pipeline with Continuous Integration and Delivery Until now, we have been executing ZenML pipelines locally. While this is a good mode of operating pipelines, in production it is often desirable to mediate runs through a central workflow engine baked into your CI. This allows data scientists to experiment with data processing and model training locally and then have code changes automatically tested and validated through the standard pull request/merge request peer review process. Changes that pass the CI and code-review are then deployed automatically to production. Here is how this could look like: Breaking it down To illustrate this, let''s walk through how this process might be set up on a GitHub Repository. A data scientist wants to make improvements to the ML pipeline. They clone the repository, create a new branch, and experiment with new models or data processing steps on their local machine. Once the data scientist thinks they have improved the pipeline, they create a pull request for their branch on GitHub. This automatically triggers a GitHub Action that will run the same pipeline in the staging environment (e.g. a pipeline running on a cloud stack in GCP), potentially with different test data. As long as the pipeline does not run successfully in the staging environment, the PR cannot be merged. The pipeline also generates a set of metrics and test results that are automatically published to the PR, where they can be peer-reviewed to decide if the changes should be merged. Once the PR has been reviewed and passes all checks, the branch is merged into main. This automatically triggers another GitHub Action that now runs a pipeline in the production environment, which trains the same model on production data, runs some checks to compare its performance with the model currently served in production and then, if all checks pass, automatically deploys the new model.' - '━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛ ```register and connect a Vertex AI Orchestrator Stack Component to the target GCP projectNOTE: If we do not specify a workload service account, the Vertex AI Pipelines Orchestrator uses the Compute Engine default service account in the target project to run pipelines. You must grant this account the Vertex AI Service Agent role, otherwise the pipelines will fail. More information on other configurations possible for the Vertex AI Orchestrator can be found here.Copyzenml orchestrator register vertex-ai-zenml-core --flavor=vertex --location=europe-west1 --synchronous=true Example Command Output ```text Running with active workspace: ''default'' (repository) Running with active stack: ''default'' (repository) Successfully registered orchestrator `vertex-ai-zenml-core`. ``` ```sh zenml orchestrator connect vertex-ai-zenml-core --connector vertex-ai-zenml-core ``` Example Command Output ```text Running with active workspace: ''default'' (repository) Running with active stack: ''default'' (repository) Successfully connected orchestrator `vertex-ai-zenml-core` to the following resources: ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┓ ┃ CONNECTOR ID β”‚ CONNECTOR NAME β”‚ CONNECTOR TYPE β”‚ RESOURCE TYPE β”‚ RESOURCE NAMES ┃ ┠──────────────────────────────────────┼──────────────────────┼────────────────┼────────────────┼────────────────┨ ┃ f97671b9-8c73-412b-bf5e-4b7c48596f5f β”‚ vertex-ai-zenml-core β”‚ πŸ”΅ gcp β”‚ πŸ”΅ gcp-generic β”‚ zenml-core ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┛ ``` Register and connect a GCP Container Registry Stack Component to a GCR container registry:Copyzenml container-registry register gcr-zenml-core --flavor gcp --uri=gcr.io/zenml-core Example Command Output ```text Running with active workspace: ''default'' (repository)' - source_sentence: How can I develop a custom step operator in ZenML? sentences: - 'Develop a Custom Step Operator Learning how to develop a custom step operator. Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML''s component flavor concepts. Base Abstraction The BaseStepOperator is the abstract base class that needs to be subclassed in order to run specific steps of your pipeline in a separate environment. As step operators can come in many shapes and forms, the base class exposes a deliberately basic and generic interface: from abc import ABC, abstractmethod from typing import List, Type from zenml.enums import StackComponentType from zenml.stack import StackComponent, StackComponentConfig, Flavor from zenml.config.step_run_info import StepRunInfo class BaseStepOperatorConfig(StackComponentConfig): """Base config for step operators.""" class BaseStepOperator(StackComponent, ABC): """Base class for all ZenML step operators.""" @abstractmethod def launch( self, info: StepRunInfo, entrypoint_command: List[str], ) -> None: """Abstract method to execute a step. Subclasses must implement this method and launch a **synchronous** job that executes the `entrypoint_command`. Args: info: Information about the step run. entrypoint_command: Command that executes the step. """ class BaseStepOperatorFlavor(Flavor): """Base class for all ZenML step operator flavors.""" @property @abstractmethod def name(self) -> str: """Returns the name of the flavor.""" @property def type(self) -> StackComponentType: """Returns the flavor type.""" return StackComponentType.STEP_OPERATOR @property def config_class(self) -> Type[BaseStepOperatorConfig]: """Returns the config class for this flavor.""" return BaseStepOperatorConfig @property @abstractmethod def implementation_class(self) -> Type[BaseStepOperator]:' - '-grade deployments. Installing the mlstacks extraTo install mlstacks, either run pip install mlstacks or pip install "zenml[mlstacks]" to install it along with ZenML. MLStacks uses Terraform on the backend to manage infrastructure. You will need to have Terraform installed. Please visit the Terraform docs for installation instructions. MLStacks also uses Helm to deploy Kubernetes resources. You will need to have Helm installed. Please visit the Helm docs for installation instructions. Deploying a stack component The ZenML CLI allows you to deploy individual stack components using the deploy subcommand which is implemented for all supported stack components. You can find the list of supported stack components here. Deploying a stack For deploying a full stack, use the zenml stack deploy command. See the stack deployment page for more details of which cloud providers and stack components are supported. How does mlstacks work? MLStacks is built around the concept of a stack specification. A stack specification is a YAML file that describes the stack and includes references to component specification files. A component specification is a YAML file that describes a component. (Currently all deployments of components (in various combinations) must be defined within the context of a stack.) ZenML handles the creation of stack specifications for you when you run one of the deploy subcommands using the CLI. A valid specification is generated and used by mlstacks to deploy your stack using Terraform. The Terraform definitions and state are stored in your global configuration directory along with any state files generated while deploying your stack. Your configuration directory could be in a number of different places depending on your operating system, but read more about it in the Click docs to see which location applies to your situation. Deploy stack components individuallyIndividually deploying different stack components.' - 'rray": [[1,2,3,4]] } }'' Using a Service ConnectorTo set up the Seldon Core Model Deployer to authenticate to a remote Kubernetes cluster, it is recommended to leverage the many features provided by the Service Connectors such as auto-configuration, local client login, best security practices regarding long-lived credentials and fine-grained access control and reusing the same credentials across multiple stack components. Depending on where your target Kubernetes cluster is running, you can use one of the following Service Connectors: the AWS Service Connector, if you are using an AWS EKS cluster. the GCP Service Connector, if you are using a GKE cluster. the Azure Service Connector, if you are using an AKS cluster. the generic Kubernetes Service Connector for any other Kubernetes cluster. If you don''t already have a Service Connector configured in your ZenML deployment, you can register one using the interactive CLI command. You have the option to configure a Service Connector that can be used to access more than one Kubernetes cluster or even more than one type of cloud resource: zenml service-connector register -i A non-interactive CLI example that leverages the AWS CLI configuration on your local machine to auto-configure an AWS Service Connector targeting a single EKS cluster is: zenml service-connector register --type aws --resource-type kubernetes-cluster --resource-name --auto-configure Example Command Output $ zenml service-connector register eks-zenhacks --type aws --resource-type kubernetes-cluster --resource-id zenhacks-cluster --auto-configure β Ό Registering service connector ''eks-zenhacks''... Successfully registered service connector `eks-zenhacks` with access to the following resources: ┏━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━┓ ┃ RESOURCE TYPE β”‚ RESOURCE NAMES ┃ ┠───────────────────────┼──────────────────┨ ┃ πŸŒ€ kubernetes-cluster β”‚ zenhacks-cluster ┃ ┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━┛' model-index: - name: zenml/finetuned-snowflake-arctic-embed-m results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy@1 value: 0.29518072289156627 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.6385542168674698 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.7228915662650602 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7891566265060241 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.29518072289156627 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.21285140562248994 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.14457831325301201 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.0789156626506024 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.29518072289156627 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.6385542168674698 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.7228915662650602 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7891566265060241 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5552191347520903 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.47847819850831885 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.48706201897841145 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.3253012048192771 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.6144578313253012 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6987951807228916 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7891566265060241 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.3253012048192771 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2048192771084337 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.1397590361445783 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.0789156626506024 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.3253012048192771 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.6144578313253012 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6987951807228916 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7891566265060241 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5597682297824715 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4859987569324918 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4930658557873217 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.2710843373493976 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5662650602409639 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6385542168674698 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7891566265060241 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.2710843373493976 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.18875502008032125 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.12771084337349395 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.0789156626506024 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.2710843373493976 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.5662650602409639 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.6385542168674698 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.7891566265060241 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5242689178594545 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4403614457831327 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4468744710389297 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.25301204819277107 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.4759036144578313 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.5783132530120482 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.6626506024096386 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.25301204819277107 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.15863453815261042 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.11566265060240961 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.06626506024096386 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.25301204819277107 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.4759036144578313 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.5783132530120482 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.6626506024096386 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.45397796379806826 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.38746175176898084 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.39859357699776915 name: Cosine Map@100 --- # zenml/finetuned-snowflake-arctic-embed-m This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [Snowflake/snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the πŸ€— Hub model = SentenceTransformer("zenml/finetuned-snowflake-arctic-embed-m") # Run inference sentences = [ 'How can I develop a custom step operator in ZenML?', 'Develop a Custom Step Operator\n\nLearning how to develop a custom step operator.\n\nBefore diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML\'s component flavor concepts.\n\nBase Abstraction\n\nThe BaseStepOperator is the abstract base class that needs to be subclassed in order to run specific steps of your pipeline in a separate environment. As step operators can come in many shapes and forms, the base class exposes a deliberately basic and generic interface:\n\nfrom abc import ABC, abstractmethod\n\nfrom typing import List, Type\n\nfrom zenml.enums import StackComponentType\n\nfrom zenml.stack import StackComponent, StackComponentConfig, Flavor\n\nfrom zenml.config.step_run_info import StepRunInfo\n\nclass BaseStepOperatorConfig(StackComponentConfig):\n\n"""Base config for step operators."""\n\nclass BaseStepOperator(StackComponent, ABC):\n\n"""Base class for all ZenML step operators."""\n\n@abstractmethod\n\ndef launch(\n\nself,\n\ninfo: StepRunInfo,\n\nentrypoint_command: List[str],\n\n) -> None:\n\n"""Abstract method to execute a step.\n\nSubclasses must implement this method and launch a **synchronous**\n\njob that executes the `entrypoint_command`.\n\nArgs:\n\ninfo: Information about the step run.\n\nentrypoint_command: Command that executes the step.\n\n"""\n\nclass BaseStepOperatorFlavor(Flavor):\n\n"""Base class for all ZenML step operator flavors."""\n\n@property\n\n@abstractmethod\n\ndef name(self) -> str:\n\n"""Returns the name of the flavor."""\n\n@property\n\ndef type(self) -> StackComponentType:\n\n"""Returns the flavor type."""\n\nreturn StackComponentType.STEP_OPERATOR\n\n@property\n\ndef config_class(self) -> Type[BaseStepOperatorConfig]:\n\n"""Returns the config class for this flavor."""\n\nreturn BaseStepOperatorConfig\n\n@property\n\n@abstractmethod\n\ndef implementation_class(self) -> Type[BaseStepOperator]:', 'rray": [[1,2,3,4]] } }\'\n\nUsing a Service ConnectorTo set up the Seldon Core Model Deployer to authenticate to a remote Kubernetes cluster, it is recommended to leverage the many features provided by the Service Connectors such as auto-configuration, local client login, best security practices regarding long-lived credentials and fine-grained access control and reusing the same credentials across multiple stack components.\n\nDepending on where your target Kubernetes cluster is running, you can use one of the following Service Connectors:\n\nthe AWS Service Connector, if you are using an AWS EKS cluster.\n\nthe GCP Service Connector, if you are using a GKE cluster.\n\nthe Azure Service Connector, if you are using an AKS cluster.\n\nthe generic Kubernetes Service Connector for any other Kubernetes cluster.\n\nIf you don\'t already have a Service Connector configured in your ZenML deployment, you can register one using the interactive CLI command. You have the option to configure a Service Connector that can be used to access more than one Kubernetes cluster or even more than one type of cloud resource:\n\nzenml service-connector register -i\n\nA non-interactive CLI example that leverages the AWS CLI configuration on your local machine to auto-configure an AWS Service Connector targeting a single EKS cluster is:\n\nzenml service-connector register --type aws --resource-type kubernetes-cluster --resource-name --auto-configure\n\nExample Command Output\n\n$ zenml service-connector register eks-zenhacks --type aws --resource-type kubernetes-cluster --resource-id zenhacks-cluster --auto-configure\n\nβ Ό Registering service connector \'eks-zenhacks\'...\n\nSuccessfully registered service connector `eks-zenhacks` with access to the following resources:\n\n┏━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━┓\n\n┃ RESOURCE TYPE β”‚ RESOURCE NAMES ┃\n\n┠───────────────────────┼──────────────────┨\n\n┃ πŸŒ€ kubernetes-cluster β”‚ zenhacks-cluster ┃\n\n┗━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━┛', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_384` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.2952 | | cosine_accuracy@3 | 0.6386 | | cosine_accuracy@5 | 0.7229 | | cosine_accuracy@10 | 0.7892 | | cosine_precision@1 | 0.2952 | | cosine_precision@3 | 0.2129 | | cosine_precision@5 | 0.1446 | | cosine_precision@10 | 0.0789 | | cosine_recall@1 | 0.2952 | | cosine_recall@3 | 0.6386 | | cosine_recall@5 | 0.7229 | | cosine_recall@10 | 0.7892 | | cosine_ndcg@10 | 0.5552 | | cosine_mrr@10 | 0.4785 | | **cosine_map@100** | **0.4871** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.3253 | | cosine_accuracy@3 | 0.6145 | | cosine_accuracy@5 | 0.6988 | | cosine_accuracy@10 | 0.7892 | | cosine_precision@1 | 0.3253 | | cosine_precision@3 | 0.2048 | | cosine_precision@5 | 0.1398 | | cosine_precision@10 | 0.0789 | | cosine_recall@1 | 0.3253 | | cosine_recall@3 | 0.6145 | | cosine_recall@5 | 0.6988 | | cosine_recall@10 | 0.7892 | | cosine_ndcg@10 | 0.5598 | | cosine_mrr@10 | 0.486 | | **cosine_map@100** | **0.4931** | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.2711 | | cosine_accuracy@3 | 0.5663 | | cosine_accuracy@5 | 0.6386 | | cosine_accuracy@10 | 0.7892 | | cosine_precision@1 | 0.2711 | | cosine_precision@3 | 0.1888 | | cosine_precision@5 | 0.1277 | | cosine_precision@10 | 0.0789 | | cosine_recall@1 | 0.2711 | | cosine_recall@3 | 0.5663 | | cosine_recall@5 | 0.6386 | | cosine_recall@10 | 0.7892 | | cosine_ndcg@10 | 0.5243 | | cosine_mrr@10 | 0.4404 | | **cosine_map@100** | **0.4469** | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.253 | | cosine_accuracy@3 | 0.4759 | | cosine_accuracy@5 | 0.5783 | | cosine_accuracy@10 | 0.6627 | | cosine_precision@1 | 0.253 | | cosine_precision@3 | 0.1586 | | cosine_precision@5 | 0.1157 | | cosine_precision@10 | 0.0663 | | cosine_recall@1 | 0.253 | | cosine_recall@3 | 0.4759 | | cosine_recall@5 | 0.5783 | | cosine_recall@10 | 0.6627 | | cosine_ndcg@10 | 0.454 | | cosine_mrr@10 | 0.3875 | | **cosine_map@100** | **0.3986** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 1,490 training samples * Columns: positive and anchor * Approximate statistics based on the first 1000 samples: | | positive | anchor | |:--------|:----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------| | type | string | string | | details |
  • min: 9 tokens
  • mean: 21.08 tokens
  • max: 49 tokens
|
  • min: 21 tokens
  • mean: 374.42 tokens
  • max: 512 tokens
| * Samples: | positive | anchor | |:------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | How can I configure SSH keys for authentication in the HyperAI orchestrator using the ZenML framework? | authentication.

ED25519 key based authentication.SSH private keys configured in the connector will be distributed to all clients that use them to run pipelines with the HyperAI orchestrator. SSH keys are long-lived credentials that give unrestricted access to HyperAI instances.

When configuring the Service Connector, it is required to provide at least one hostname via hostnames and the username with which to login. Optionally, it is possible to provide an ssh_passphrase if applicable. This way, it is possible to use the HyperAI service connector in multiple ways:

Create one service connector per HyperAI instance with different SSH keys.

Configure a reused SSH key just once for multiple HyperAI instances, then select the individual instance when creating the HyperAI orchestrator component.

Auto-configuration

This Service Connector does not support auto-discovery and extraction of authentication credentials from HyperAI instances. If this feature is useful to you or your organization, please let us know by messaging us in Slack or creating an issue on GitHub.

Stack Components use

The HyperAI Service Connector can be used by the HyperAI Orchestrator to deploy pipeline runs to HyperAI instances.

PreviousAzure Service Connector

NextManage stacks

Last updated 19 days ago | | What additional settings are required to enable CUDA for GPU-backed hardware when using the LocalDockerOrchestratorSettings? | or.local_docker": LocalDockerOrchestratorSettings(run_args={"cpu_count": 3}

@pipeline(settings=settings)

def simple_pipeline():

return_one()

Enabling CUDA for GPU-backed hardware

Note that if you wish to use this orchestrator to run steps on a GPU, you will need to follow the instructions on this page to ensure that it works. It requires adding some extra settings customization and is essential to enable CUDA for the GPU to give its full acceleration.

PreviousLocal Orchestrator

NextKubeflow Orchestrator

Last updated 15 days ago | | What is the SECRET ID for the gcs-bucket resource type? | ──────────┼──────────────────────────────────────┨┃ RESOURCE TYPES β”‚ πŸ“¦ gcs-bucket ┃

┠──────────────────┼──────────────────────────────────────┨

┃ RESOURCE NAME β”‚ ┃

┠──────────────────┼──────────────────────────────────────┨

┃ SECRET ID β”‚ 0d0a42bb-40a4-4f43-af9e-6342eeca3f28 ┃

┠──────────────────┼──────────────────────────────────────┨

┃ SESSION DURATION β”‚ N/A ┃

┠──────────────────┼──────────────────────────────────────┨

┃ EXPIRES IN β”‚ N/A ┃

┠──────────────────┼──────────────────────────────────────┨

┃ OWNER β”‚ default ┃

┠──────────────────┼──────────────────────────────────────┨

┃ WORKSPACE β”‚ default ┃

┠──────────────────┼──────────────────────────────────────┨

┃ SHARED β”‚ βž– ┃

┠──────────────────┼──────────────────────────────────────┨

┃ CREATED_AT β”‚ 2023-05-19 08:15:48.056937 ┃

┠──────────────────┼──────────────────────────────────────┨

┃ UPDATED_AT β”‚ 2023-05-19 08:15:48.056940 ┃

┗━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

Configuration

┏━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━┓

┃ PROPERTY β”‚ VALUE ┃

┠──────────────────────┼────────────┨

┃ project_id β”‚ zenml-core ┃

┠──────────────────────┼────────────┨

┃ service_account_json β”‚ [HIDDEN] ┃

┗━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━┛

GCP Service Account impersonation

Generates temporary STS credentials by impersonating another GCP service account. | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 384, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: True - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_384_cosine_map@100 | dim_64_cosine_map@100 | |:----------:|:-----:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | 0.6667 | 1 | 0.3889 | 0.4114 | 0.4339 | 0.2694 | | 1.9583 | 3 | 0.4463 | 0.4920 | 0.4852 | 0.3876 | | **2.5833** | **4** | **0.4469** | **0.4931** | **0.4871** | **0.3986** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.10.14 - Sentence Transformers: 3.0.1 - Transformers: 4.41.2 - PyTorch: 2.3.1+cu121 - Accelerate: 0.31.0 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```