FareedKhan's picture
Add new SentenceTransformer model.
51fdcaf verified
metadata
base_model: intfloat/multilingual-e5-large-instruct
datasets: []
language: []
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:1814
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-


      The gene in question appears to be involved in multiple cellular
      processes, many of which are central to neuronal function and health,
      especially in the context of neurodegenerative diseases. Here's a brief
      overview of its functions and context:


      ### Key Functions:

      1. **Transcription Regulation**: Involved in RNA polymerase II
      transcription and regulation of gene expression.

      2. **Protein Processing**: Positive regulation of proteasomal
      ubiquitin-dependent protein catabolic process, indicating it might play a
      role in the degradation and recycling of proteins.

      3. **Cellular Stress Response**: Regulation of positive transcription by
      p53 (a known DNA damage response gene), positive regulation of I-kappaB
      kinase/NF-kappaB signaling (involved in inflammatory response), and
      negative regulation of cell death under oxidative stress.

      4. **Cellular Repair and Maintenance**: Autophagy of mitochondria
      (self-eating of organelles to clear damaged components), regulated the
      negative regulation of intrinsic apoptotic signaling pathways,
      facilitating cell survival rather than death.

      5. **Neurotransmitter and Ion Handling**: Involvement in dopamine
      secretion, response to manganese ion, and within synaptic transmission
      processes.

      6. **Metabolic Activities**: Influences glucose metabolism by regulation
      of glucokinase activity.


      ### Context Specific:

      - **Manganese Exposure**: This gene's role in transcriptional regulation
      is particularly implicated in the context of manganese exposure. Manganese
      can be neurotoxic, particularly affecting the nervous system. Its
      regulation might help in the cellular response to manganese toxicity,
      including signaling pathways that
    sentences:
      - >-
        Identify genes or proteins that interact with CLDN11 and are also
        implicated in the same medical condition.
      - >-
        Search for ailments that have no drugs indicated for treatment and have
        a connection to Dermatographic urticaria.
      - >-
        Is there an interaction between the parkin RBR E3 ubiquitin protein
        ligase and the DNA-damage-inducible transcript 4 (DDIT4), and if so,
        what biological effects or phenotypes have been associated with this
        interaction?
  - source_sentence: >-

      Risdiplam, an orally bioavailable mRNA splicing modifier, serves as a
      treatment for spinal muscular atrophy (SMA), increasing systemic SMN
      protein
    sentences:
      - >-
        Which medication shares a gene or protein carrier with Clobetasol
        propionate and also elevates SMN protein levels to aid in the management
        of Spinal Muscular Atrophy?
      - >-
        Which gene or protein is known to engage with SIRT1 and also modulates
        cholesterol levels, and has been linked to effects or phenotypes that
        resemble those associated with SIRT1?
      - >-
        What is the name of the heart disease associated with the abnormal
        tricuspid subvalvular apparatus that leads to restricted leaflet motion
        and regurgitation in the tricuspid valve?
  - source_sentence: >-

      The list you provided includes a wide variety of substances that are
      categorized based on their primary function or use. Here's a breakdown
      based on their classifications:


      ### Medications and Drugs

      - **Anabolics (e.g., Methandrostenolone, Nandrolone)**: Steroids used for
      increasing muscle mass and strength.

      - **Antidepressants (e.g., Sertraline, Flunarizine)**: Drugs used to treat
      depression and anxiety disorders.

      - **Anticoagulants (e.g., Warfarin, Apixaban)**: Medications used to
      prevent blood clots.

      - **Antivirals (e.g., Adefovir, Etoricoxib)**: Antiviral drugs used to
      treat viral infections such as hepatitis.

      - **Anticancer Drugs (e.g., Atemzan, Sorafenib)**: Used in chemotherapy
      and targeted therapy for cancer treatment.

      - **Antipsychotics (e.g., Haloperidol, Sertaconazole)**: Medications used
      in the treatment of psychosis and other mental disorders.

      - **Anesthetic Agents (e.g., Ketamine, Etomidate)**: Used for inducing and
      maintaining general anesthesia
    sentences:
      - >-
        What disease acts as a common subtype to both prostate carcinoma and
        prostatic urethra urothelial carcinoma, indicating an overlap in their
        clinical categorizations?
      - >-
        Could you tell me about the medications available that specifically
        target mantle cell lymphoma, particularly those acting on the BTK
        pathway, to effectively address the underlying disease mechanism?
      - >-
        Which medication shares a gene or protein transporter with Zileuton and
        serves as a chain-terminating agent by integrating into the viral DNA
        replication process?
  - source_sentence: >-

      Xanthogranulomatous pyelonephritis is a disease characterized by chronic,
      destructive infections of the kidney, marked by the infiltration of
      lipid-laden macrophages. This condition is often linked to obstruction due
      to infected renal stones, commonly caused by bacteria such as Proteus or
      Escherichia coli. The UMLS description highlights that this disease
      results in diffuse renal destruction, leading to a grossly enlarged and
      nonfunctional kidney. It is closely related to pyelonephritis, and its
      symptoms include nephrolithiasis and kidney stones. The involvement of
      macrophage infiltration and tissue destruction, along with its links to
      nephrolithiasis and urinary obstruction, makes xanthogranulomatous
      pyelonephritis a disease with significant similarities to pyelonephritis.
    sentences:
      - >-
        Identify genes or proteins that interact with IL1R1 and are concurrently
        linked to identical phenotypes or biological effects.
      - >-
        What renal disease might involve macrophage infiltration and tissue
        destruction, with links to nephrolithiasis and urinary obstruction,
        similar to pyelonephritis?
      - >-
        Please find genes or proteins that interact with AGO2 and bind to the
        AU-rich elements in mRNA 3'-UTRs.
  - source_sentence: >-


      The gene in question appears to have a multifaceted role and involvement
      in various biological processes, diseases, and anatomical structures, with
      implications for both physiology and pathology. Here is a summary of its
      characteristics:


      ### Function and Interactions

      - **Name**: mTORC1, a component of the mammalian target of rapamycin
      complex 1.

      - **Role**: Involved in regulation of membrane potential
    sentences:
      - >-
        Which solid-state medications specifically engage with the METAP2
        gene/protein through direct interaction?
      - >-
        Which pharmacological agents designed to bind genes or proteins
        associated with RNA polymerase I that have the potential to induce
        regression of lung tumors?
      - >-
        Identify genes or proteins that interact with KCNMB1 and share an
        associated phenotype or effect.
model-index:
  - name: SentenceTransformer based on intfloat/multilingual-e5-large-instruct
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.400990099009901
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4603960396039604
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.4900990099009901
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.5445544554455446
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.400990099009901
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.15346534653465344
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.09801980198019801
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.05445544554455445
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.400990099009901
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.4603960396039604
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.4900990099009901
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.5445544554455446
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.46500004508597764
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4406471004243281
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.44882168670509337
            name: Cosine Map@100

SentenceTransformer based on intfloat/multilingual-e5-large-instruct

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-large-instruct. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: intfloat/multilingual-e5-large-instruct
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 1024 tokens
  • Similarity Function: Cosine Similarity

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("FareedKhan/just_for_testing_model")
# Run inference
sentences = [
    '\n\nThe gene in question appears to have a multifaceted role and involvement in various biological processes, diseases, and anatomical structures, with implications for both physiology and pathology. Here is a summary of its characteristics:\n\n### Function and Interactions\n- **Name**: mTORC1, a component of the mammalian target of rapamycin complex 1.\n- **Role**: Involved in regulation of membrane potential',
    'Identify genes or proteins that interact with KCNMB1 and share an associated phenotype or effect.',
    'Which solid-state medications specifically engage with the METAP2 gene/protein through direct interaction?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.401
cosine_accuracy@3 0.4604
cosine_accuracy@5 0.4901
cosine_accuracy@10 0.5446
cosine_precision@1 0.401
cosine_precision@3 0.1535
cosine_precision@5 0.098
cosine_precision@10 0.0545
cosine_recall@1 0.401
cosine_recall@3 0.4604
cosine_recall@5 0.4901
cosine_recall@10 0.5446
cosine_ndcg@10 0.465
cosine_mrr@10 0.4406
cosine_map@100 0.4488

Training Details

Training Dataset

Unnamed Dataset

  • Size: 1,814 training samples
  • Columns: positive and anchor
  • Approximate statistics based on the first 1000 samples:
    positive anchor
    type string string
    details
    • min: 3 tokens
    • mean: 270.4 tokens
    • max: 512 tokens
    • min: 15 tokens
    • mean: 39.58 tokens
    • max: 127 tokens
  • Samples:
    positive anchor

    Racecadotril, a drug from the Antidiarrheals and Intestinal Antiinflammatory/antiinfective Agents therapeutic group, targets the treatment of acute diarrhea and acute gastroenteritis. This drug falls under Alimentary Tract and Metabolism (ATC Classification) and is related to other antidiarrheals (ATC Specificity Class). Its chemical and functional group is classified under other antidiarrheals, reflecting its role in gastrointestinal health. With a molecular weight of 385.48 grams, and a topological polar surface area of 72.47, Racecadotril demonstrates specific properties that contribute to its effectiveness. The drug's inclusion in the investigational group suggests ongoing research to further validate its efficacy and safety in managing acute diarrhea. Racecadotril's diverse chemical nature, including amino acids, peptides, and sulfur-compound-based inhibitors, highlights its broad mechanism of action, which is crucial for addressing the multifaceted causes of acute diarrhea.
    Could you suggest some effective medications for acute diarrhea?

    The gene KCNJ6, also known as potassium inwardly rectifying channel subfamily J member 6, is responsible for encoding a member of the G protein-coupled inwardly-rectifying potassium channel family. This type of potassium channel allows for a greater flow of potassium into the cell than out, playing a critical role in various physiological processes. The KCNJ6 gene modulates functions in cardiac cells and neuronal cells through G-protein coupled receptor stimulation. Mutations in KCNJ6 can lead to Keppen-Lubinsky Syndrome, a rare condition characterized by severe developmental delay, facial dysmorphism, and intellectual disability.

    In terms of its expression, the KCNJ6 gene is notably absent in the mucosal tissues of the mouth and the small intestine as indicated by its 'expression_absent' attribute. The absence of KCNJ6 in these specific anatomical areas might imply a tailored physiological function or evolutionary adaptation distinct from that in other organs where it is expressed. The gene's absence in the small intestine and mouth tissues might be crucial for ensuring optimal functionality within those specific environments, emphasizing the importance of precise gene expression in various parts of the body. Overall, the gene plays a vital role in cellular and physiological activities and its absence from certain tissues like the mucosal surfaces of the mouth and small intestine is indicative of tailored biological functions within these areas.
    Which gene or protein is consistently not expressed in the mucosal tissues of the mouth and the small intestine?

    The term you're referring to seems to describe a wide range of biological entities or contexts, possibly within the framework of a scientific study or database that categorizes anatomical locations, tissues, or organs in the human body. Here's a breakdown of the classification:

    1. Lower Body and Tissue Levels:
    - Intestines (Large and Small): Refers to the large intestine (colon) and small intestine, parts of the digestive system.
    - Kidneys: Includes both the adult kidney and metanephros, an embryonic kidney stage.
    - Adrenal Gland: Part of the endocrine system related to metabolism, stress response, and more.
    - Liver: A primary organ involved in detoxification and metabolism.
    - Pancreas: Produces digestive enzymes and hormones such as insulin.
    - Renal Glomerulus: Tiny filtration units in the kidney.
    - Muscles (Various): Refers to a classification of skeletal, smooth, and cardiac muscles, each serving different bodily functions.
    - Bones and Connective Tissue: Includes bones, regions like the spinal cord, and structures like the heart, aorta, etc.

    2. Anatomical Areas:
    - Dorsal Root Ganglion: A cluster of neurons in the spinal cord.
    - Cerebral Cortex: Outer layers of the brain responsible for higher functions.
    - Embryonic Stages: Describes entities like the embryonic kidney (metanephros) and placenta, which are crucial during fetal development.
    - Reproductive Organ Systems: Such as the uterus, vagina, and mammary gland, highlighting the female reproductive system.

    3. Tissue Types:
    - Connective Tissue: Includes bones, the meninges, and fat.
    - Adipose Tissue: Found in various organs and is involved in energy storage and insulation.

    4. Impact of Exposure: The mention of "blood" might indicate the interest in studying the effects of environmental
    Which genes or proteins exhibit interactions with HNRNPU, share an association with its related disease(s), and participate in the peroxisomal beta-oxidation process of fatty acid metabolism?
  • Loss: MatryoshkaLoss with these parameters:
    {
        "loss": "MultipleNegativesRankingLoss",
        "matryoshka_dims": [
            768
        ],
        "matryoshka_weights": [
            1
        ],
        "n_dims_per_step": -1
    }
    

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: epoch
  • per_device_train_batch_size: 2
  • learning_rate: 1e-05
  • num_train_epochs: 2
  • warmup_ratio: 0.1
  • bf16: True
  • tf32: False
  • load_best_model_at_end: True

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: epoch
  • prediction_loss_only: True
  • per_device_train_batch_size: 2
  • per_device_eval_batch_size: 8
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 1e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 2
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: False
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • eval_use_gather_object: False
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss dim_768_cosine_map@100
0 0 - 0.2774
0.0220 10 0.7928 -
0.0441 20 0.7435 -
0.0661 30 0.6181 -
0.0881 40 0.5851 -
0.1101 50 0.4896 -
0.1322 60 0.5216 -
0.1542 70 0.3562 -
0.1762 80 0.4002 -
0.1982 90 0.286 -
0.2203 100 0.3835 -
0.2423 110 0.3237 -
0.2643 120 0.5041 -
0.2863 130 0.4061 -
0.3084 140 0.3758 -
0.3304 150 0.4442 -
0.3524 160 0.3714 -
0.3744 170 0.4349 -
0.3965 180 0.3492 -
0.4185 190 0.1045 -
0.4405 200 0.2965 -
0.4626 210 0.1913 -
0.4846 220 0.4259 -
0.5066 230 0.4671 -
0.5286 240 0.4812 -
0.5507 250 0.2442 -
0.5727 260 0.157 -
0.5947 270 0.4386 -
0.6167 280 0.0979 -
0.6388 290 0.7879 -
0.6608 300 0.073 -
0.6828 310 0.252 -
0.7048 320 0.3913 -
0.7269 330 0.1331 -
0.7489 340 0.1311 -
0.7709 350 0.3487 -
0.7930 360 0.2204 -
0.8150 370 0.1718 -
0.8370 380 0.4277 -
0.8590 390 0.4798 -
0.8811 400 0.1381 -
0.9031 410 0.4986 -
0.9251 420 0.2379 -
0.9471 430 0.2717 -
0.9692 440 0.5997 -
0.9912 450 0.2738 -
1.0 454 - 0.4476
1.0132 460 0.0649 -
1.0352 470 0.1113 -
1.0573 480 0.0916 -
1.0793 490 0.0866 -
1.1013 500 0.1341 -
1.1233 510 0.1591 -
1.1454 520 0.0737 -
1.1674 530 0.2395 -
1.1894 540 0.051 -
1.2115 550 0.1838 -
1.2335 560 0.0741 -
1.2555 570 0.2529 -
1.2775 580 0.1624 -
1.2996 590 0.1957 -
1.3216 600 0.1015 -
1.3436 610 0.056 -
1.3656 620 0.0592 -
1.3877 630 0.2027 -
1.4097 640 0.0874 -
1.4317 650 0.144 -
1.4537 660 0.2371 -
1.4758 670 0.083 -
1.4978 680 0.1608 -
1.5198 690 0.1924 -
1.5419 700 0.1765 -
1.5639 710 0.0068 -
1.5859 720 0.1316 -
1.6079 730 0.1538 -
1.6300 740 0.1136 -
1.6520 750 0.1216 -
1.6740 760 0.2417 -
1.6960 770 0.1868 -
1.7181 780 0.2164 -
1.7401 790 0.1186 -
1.7621 800 0.0155 -
1.7841 810 0.033 -
1.8062 820 0.024 -
1.8282 830 0.2094 -
1.8502 840 0.0761 -
1.8722 850 0.0876 -
1.8943 860 0.308 -
1.9163 870 0.0557 -
1.9383 880 0.2808 -
1.9604 890 0.0886 -
1.9824 900 0.2489 -
2.0 908 - 0.4488
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.10
  • Sentence Transformers: 3.0.1
  • Transformers: 4.43.4
  • PyTorch: 2.4.0+cu121
  • Accelerate: 0.33.0
  • Datasets: 2.21.0
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}