---
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
base_model: avsolatorio/GIST-small-Embedding-v0
metrics:
- accuracy
widget:
- text: 'User: Hello, I want to transfer some funds to another bank account.
Bank Bot: Hi, you can do that through our online banking system. Can you please
confirm the amount and the account details?
User: Sure, I want to transfer $500 to account number 123456789.
Bank Bot: Okay, I have processed your request to transfer $500 to account number
123456789. Is there anything else I can assist you with?
User: No, actually I wanted to transfer $1000, not $500.'
- text: 'User: Hello, I''m having trouble with my ATM card.
Bank Bot: I''m sorry to hear that. To assist you further, could you please provide
me with your account number.
User: Yeah, 1234454673838 this is my account number.
Bank Bot: It seems like your account is blocked. We will investigate and get back
to you. Do you have any other issues?
User: That''s all I needed help with. Thank you for your assistance.'
- text: 'User: Hello, how do I transfer funds to my friend''s account?
Bank Bot: Hi, you can transfer funds through our online banking system or mobile
banking app. Which one would you prefer?
User: I would prefer the mobile banking app.
Bank Bot: Great! To transfer funds through the app, you need to login first. Once
logged in, select the option "Transfer Funds" from the menu.
User: Okay, I have logged in and selected "Transfer Funds". What details do I
need to provide?
Bank Bot: You need to provide the account number, name of the recipient, and the
bank name and branch where your friend''s account is held. Do you have these details
handy?
User: Yes, I do. But can you confirm the daily transfer limit for me?'
- text: 'User: Hello, I''m having trouble with my ATM card.
Bank Bot: I''m sorry to hear that. To assist you further, could you please provide
me with your account number.
User: Yes, my account number is 9872123424.
Bank Bot: When did you transferred your money?
User: The transaction happened on 5th September.
Bank Bot: Okay, I''ll investigate.
User: Sorry, I provided the wrong date for the transaction. It was actually on
6th September.'
- text: 'User: Hello, how do I transfer funds to my friend''s account?
Bank Bot: Hi, you can transfer funds through our online banking system or mobile
banking app. Which one would you prefer?
User: I would prefer the online banking.
Bank Bot: Alright. Can you please provide me the account number of your friend
to initiate the transfer?
User: I''m sorry, I cannot provide you with that information.
Bank Bot: I understand. However, we need the account number to initiate the transfer.
Alternatively, you can ask your friend to provide their account number to you
and then initiate the transfer.
User: I still refuse to provide the account number. Is there any other way to
transfer funds?'
pipeline_tag: text-classification
inference: true
model-index:
- name: SetFit with avsolatorio/GIST-small-Embedding-v0
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.5714285714285714
name: Accuracy
---
# SetFit with avsolatorio/GIST-small-Embedding-v0
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 512 tokens
- **Number of Classes:** 14 classes
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:---------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Restart Conversation | - "User: HEY! TRANSFER FUNDS. MY TRANSFER NOT PROCESS. PLS HELP.\nBot: Hello there. I'm happy to assist you with your fund transfer. Can you please provide me with the transaction details?\nUser: SRY I NO MEMB I THINK 50 DOLLAR TO MOM.\nBot: I see. Do you have your mom's account details and can you please confirm the amount and reference number of the transaction?\nUser: I THINK YES, WAIT PLS. AHA IT 123456.\nBot: Thank you. I'm checking the transaction now. Please hold on for a moment.\nUser: KK.\nBot: I've checked the transaction and it appears to be successful. Can you please check your mom's account to confirm?\nUser: OH OKAY. I SEE NOW. THNK U SO MUCH. I'm sorry for the mistakes in my previous texts. Can we start over again, please?"
- "User: Hi, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. To assist you further, could you please provide me with the details of the transaction you wish to dispute, including the transaction amount, date, and description?\nUser: Can we start over? I missed the details you asked for earlier.\nBank Bot: Of course. Could you please provide me with the specifics of the transaction you are disputing, including the transaction amount, date, and description?\nUser: $100 on March 15th, at a restaurant. However, I don't recall making this transaction.\nBank Bot: Thank you for providing the details. Based on your information, I can see that the transaction was made at a restaurant. Do you often dine at this particular restaurant?\nUser: No, I have never been there before.\nBank Bot: I'm sorry to hear that. Have you checked with anyone who may have access to your card information, such as a family member or friend?\nUser: Yes, I have checked with my family and friends, but none of them made this transaction.\nBank Bot: I see. In this case, I will need to escalate the issue to our dispute resolution team. Can you please confirm that you did not authorize this transaction?\nUser: Actually I've lost in process can we restart it?"
- "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: Actually, let's start over. My account was hacked, and someone made a withdrawal of $1000 that I did not authorize.\nBank Bot: I see. When did this happen, and which transaction specifically are you disputing?\nUser: I forgot the context, can we restart it?"
|
| Correction Request | - 'User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: Actually, I had selected the wrong option earlier. I am interested in opening a checking account.\nBank Assist Bot: No worries at all! Can you please provide your name and contact number so we can begin the process again?\nUser: Yes, my name is Jake Miller and my contact number is 555-555-5555.\nBank Assist Bot: Thanks, Jake! Before we proceed further, may I know if you already have an existing account with our bank?\nUser: Yes, I do.\nBank Assist Bot: Great! Can you please provide your account number and social security number to proceed?\nUser: Yes, my account number is 123456789 and my social security number is 123-45-6789.\nBank Assist Bot: Thanks for sharing the information, Jake. Can you please verify your date of birth as well?\nUser: Oops, I had given the wrong account number. It is actually 987654321.'
- "User: Hi, I would like to dispute a transaction on my account.\nBank bot: Sure, can you please provide me with the details of the transaction you wish to dispute?\nUser: Yes, it was a charge of $100 on my account that I didn't authorize.\nBank bot: I see. During our investigation, we found that the transaction was made through your debit card. Can you confirm if you have authorized any transactions through your card?\nUser: No, I haven't made any such transaction. I think my card details were stolen."
- "User: hi, i have to talk to you about a transfer. i don't know what to do.\nBot: Of course, I'd be happy to assist you with that. Can you be more specific about the transfer you'd like to dispute?\nUser: i don t knw rly. some sort of mistake with my account or smth.\nBot: I understand, could you please provide me with your account number and the date of the transaction so I can take a closer look at the issue and help you resolve it?\nUser: umm.. i think it was two weeks ago and the account number is 12345.\nBot: Thank you for that information. Could you please tell me your full name for verification purposes?\nUser: sure, my name is Emma.\nBot: Thank you, Emma. I'll check that transaction and get back to you with an update shortly.\nUser: oh wait, actually my account number is wrong. It's 54321"
|
| Affirmative Confirmation | - "User: I want to dispute a transaction.\nBank bot: Sure, I'd be happy to help. Can you tell me the date and amount of the transaction?\nUser: Yes, it was on June 15th for $50.\nBank bot : Are you sure?\nUser : Yes."
- "User: hi im not suree if i transfered mony but my balence is to low can u help\nBot: Hello, I'm sorry for the confusion. Can you provide me with more information regarding the transfer in question?\nUser: yea its from cheking to savInc and was $300 last week tuesday\nBot: Alright, thank you for clarifying that for me. I see the transfer you mentioned. It looks like your balance is reflecting that transaction. Is there anything else you're unsure about?\nUser: no thats it thx your helpful"
- 'User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: A savings account.\nBank Assist Bot: Fantastic! Would you prefer one with a higher interest rate or more flexibility with withdrawals?\nUser: I would prefer a higher interest rate.\nBank Assist Bot: Alright, thank you. Can you please provide me with your name and contact number to begin the process of opening the account?\nUser: Yes, my name is John Smith and my contact number is 555-555-5555.\nBank Assist Bot: Perfect! Before we proceed further, may I know if you have an existing account with our bank?\nUser: Yes'
|
| Questions Requiring External Information | - "User: Hi, I think something went wrong with my accunt. There is transaction that I did not make and it think I have been charged.\nBot: I'm sorry to hear that. Could you please give me more information about the transaction so I can investigate?\nUser: Ya, it was on November 20th for about $150. I id not recognize the recipient and id not approve the payment.\nBot: I see. Based on the information you provided, it sounds like this may be fraud. I willassist you with disputing the transaction. Please confirm if you contacted the recipient regarding this issue.\nUser: I did not contact them yet. How do I do it?\nBot: Just get in touch with the merchant or seller involved in the transaction and try to get a refund or more information on what the transaction was for. If you're not able to resolve it that way, we can start the dispute process. Is there anything else I can help you with?\nUser: Not really, thanks. Just one question, how do I make a transfer to other bank accounts?"
- "Certainly! Here's another example:\nUser: Hi, I need to dispute a transaction.\nBank Bot: Okay, I’m sorry to hear that. Could you please provide some more details about the transaction?\nUser: Yes, I see a charge of $200 that I didn’t make.\nBank Bot: I understand. Could you please confirm the date and the name of the merchant?\nUser: It was on September 5th and the name is ABC Electronics.\nBank Bot: Thank you for the information. Is there anything else you'd like to add or modify?\nUser: Can you provide me with the status of my recent deposit?"
- "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: This is ridiculous! Someone has stolen $1000 from my account, and I want it back immediately!\nBank Bot: I'm sorry to hear about the unauthorized transaction on your account. Could you please confirm the details of the transaction and your account information to help me assist you better?\nUser: Can you tell me if the transaction was made from a foreign country?"
|
| Repeat Conversation | - "User: Hello, I'd like to dispute a transaction on my account.Bank \nBot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorize this transaction.\nBank Bot: Thank you for bringing this to our attention. Could you please provide me with your account number and contact information in case we need to follow up with you?\nUser: Sorry, could you Restart Conversation that? I didn't catch the last part."
- "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: What did you say? I couldn't hear you properly."
- "User: Hello, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorise this transaction.\nBank Bot: The transaction you reported was for $500 on April 1st and was an online purchase. Please confirm if this is correct.\nUser: Sorry, could you please Restart Conversation the details?"
|
| Pause Conversation | - "User: Hi, I want to dispute a transaction on my account.\nBank Bot: Hi there! I'd be happy to assist you with that. Can you please provide me with the details of the transaction you wish to dispute?\nUser: I noticed a charge on my account for $100 that I don't recognize.\nBank Bot: I'm sorry to hear that. Can you please tell me the date the transaction took place and the name of the merchant that charged your account?\nUser: It happened on June 1st and the merchant is called XYZ Company.\nBank Bot: Thanks for the information. We'll need to investigate it further. Can you confirm that you did not authorize this transaction?\nUser: Can I pause the conversation for a moment, please?"
- "User: My transaction is wrong, somthing's happend and I don't recognize this transaction.\nBot: I'm sorry, could you please provide more information regarding the transaction in question such as the date and amount?\nUser: I not know the exact date but the ammaunt was around $30. Let me give second."
- 'User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: Can I pause for a moment? I need to grab some information first.\nBank Assist Bot: Of course! Take your time and let me know when you are ready.\nUser: Sorry for the wait. I am interested in opening a checking account.\nBank Assist Bot: No problem at all! Can you please provide your name and contact information to begin the process?\nUser: Yes, my name is Sarah Smith and my contact number is 555-555-5555.\nBank Assist Bot: Great, Sarah! Before we proceed further, may I know if you already have an existing account with our bank?\nUser: No, I do not. Can we take a pause here.'
|
| Negative Confirmation | - "User: Hi, I noticed a transaction on my account that I don't recognize. Can you help me with this?\nBank Bot: Sure, I can assist you with that. Can you please provide me with the details of the transaction in question?\nUser: It's a charge for $100 to a website I've never heard of.\nBank Bot: I see. Based on our records, the transaction was processed on the 15th of this month. Are you disputing the transaction?\nUser: No I did not make this transaction."
- "User: Hi there, I need to speak to someone about a problem I'm having with a recent transaction.\nBank Bot: Alright, I'm here to help. What seems to be the issue?\nUser: I received a charge on my account that I don't recognize. Can you tell me what it's for?\nBank Bot: Yes, it looks like the charge is from a restaurant on Main Street for $100. Is that familiar to you?\nUser: No, it's definitely not. I haven't been to Main Street in months."
- "User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: Actually, I wanted to check the status of my existing account.\nBank Assist Bot: Oh, I see. Apologies for the confusion. Can you please tell me your account number?\nUser: Actually, I don't have it with me right now."
|
| Continue Conversation | - "User: Hi, I want to dispute a transaction on my account.\nBank Bot: Hi there! I'd be happy to assist you with that. Can you please provide me with the details of the transaction you wish to dispute?\nUser: I noticed a charge on my account for $100 that I don't recognize.\nBank Bot: I'm sorry to hear that. Can you please tell me the date the transaction took place and the name of the merchant that charged your account?\nUser: It happened on June 1st and the merchant is called XYZ Company."
- "User: Hello, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorise this transaction.\nBank Bot: Thank you for providing me with the information. We will investigate this transaction and get back to you with an update as soon as possible. Do you have any other transaction disputes that you want to report?\nUser: No, this is the only one. Thank you for your help.\nBank Bot: You're welcome. Is there anything else I can assist you with today?\nUser: No, that's all for now. Thank you again."
- "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: This is ridiculous! Someone has stolen $1000 from my account, and I want it back immediately!\nBank Bot: I completely understand your frustration. Can you please tell me which transaction you are disputing and when it took place?\nUser: It happened on September 15th, and it was a withdrawal at an ATM that I didn't make.\nBank Bot: I see. Just to clarify, was this at an ATM that you normally use?\nUser: No, it was at a different location that I have never been to before.\nBank Bot: Thank you for providing that information. We will investigate this immediately and get back to you with the results. Is there anything else that I can assist you with?\nUser: No, just please resolve this issue as soon as possible."
|
| End Conversation | - "User: Hi, I am not sure about the transaction I made yesterday.\nBot: Hello, I'm sorry to hear that. Can you please elaborate on the transaction in question?\nUser: Yeah, I transferred funds to a friend, but I think it's not reflecting in his account.\nBot: I understand your concern. Could you please tell me the transaction details such as the amount and the recipient's account details? I can check our records and provide a solution.\nUser: Oh sure, let me check the details and get back to you.\nBot: Alright, take your time and let me know if you need any further assistance. Have a good day!\nUser: Thank you, bye."
- 'User: I want to dispute a transaction.\nBank Bot: Sure, please provide me with the details of the transaction.\nUser: There was a charge of $75 from a restaurant that I never went to.\nBank Bot: I understand. Can you please provide me with the transaction date and amount?\nUser: The transaction was on September 1st for $75.\nBank Bot: Okay, I have initiated the dispute process. You will receive an update within the next 7 business days.\nUser: Thank you, I appreciate your help.'
- "User: I need help disputing a transaction.\nBank Bot: Sure, I can help you with that. Can you please provide me with the details of the transaction?\nUser: It was a charge of $50 from a vendor I don't recognize.\nBank Bot: Okay, I will need to verify some information to start the dispute process. Can you please confirm the date and amount of the transaction?\nUser: The transaction was on August 25th for $50.\nBank Bot: Thank you. I will initiate the dispute process and provide you with updates promptly.\nUser: Alright, thanks for your help."
|
| New Inquiry or Request | - "User: Hi, I was looking at my account and noticed a transaction that I don't recognize. Can you explain it to me please?\nBot: Of course, I'd be glad to help. Can you provide me with the transaction details (date, amount, description)?\nUser: Sorry, I don't have the details at the moment. But can you check if there's any discrepancy in my account?\nBot: Sure, I can review your account transactions for any inaccuracies. However, it would be helpful if you could provide me with more information about the transaction in question.\nUser: I understand. I'll check my records and get back to you. By the way, can you tell me if your bank offers international transfer services?"
- "User: hai, i need helpe with my accaunt. ive been charge for something i didn't dorecentely.\nBot: Hello, I apologize but I did not understand your request. Could you please provide more details about the transaction in question?\nUser: Yes soree, i meen i woz chargd for a purchaes that i didnt mad laat week. Can you helip?\nBot: I see. I believe you are notifying me of an unauthorized charge on your account. To help resolve this issue, please provide the date, amount, and description of the transaction so that I can investigate further.\nUser: Ok, lemmi chek. By the waay, i judt want to knoe if i can take a hom loan?"
- "User: Hi, I have a question about disputing a transaction on my account.\nBank Bot: Hello! I'm happy to help. Please provide me with some details about the transaction in question.\nUser: I received a product that was damaged, but the merchant has refused to issue a refund. What can I do?\nBank Bot: I'm sorry to hear that. Have you tried contacting the merchant to resolve the issue?\nUser: Yes, but they are not being cooperative. Can you help me dispute the transaction?"
|
| Questions Answerable by Conversational Context / History | - "User: Hello, I'd like to dispute a transaction on my account.Bank \nBot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorize this transaction.\nBank Bot: Thank you for bringing this to our attention. Could you please provide me with your account number and contact information in case we need to follow up with you?\nUser: Sure, my account number is XXXXXXXX and my phone number is XXX-XXX-XXXX.\nBank Bot: Thank you for the information. We will investigate the transaction and get back to you within 3-5 business days.\nUser: How long will it take?"
- "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: This is ridiculous! Someone has stolen $1000 from my account, and I want it back immediately!\nBank Bot: I'm sorry to hear about the unauthorized transaction on your account. Could you please confirm the details of the transaction and your account information to help me assist you better?\nUser: Can you tell me if there have been any other unauthorized transactions on my account?"
- "User: Hi, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. To assist you further, could you please provide me with the details of the transaction you wish to dispute, including the transaction amount, date, and description?\nUser: Yes, my account number is 12937492174.\nBank Bot: Thank you for providing the details. Based on your information, I can see that the transaction was made at a restaurant. Do you often dine at this particular restaurant?\nUser: No, I have never been there before.\nBank Bot: I'm sorry to hear that. Have you checked with anyone who may have access to your card information, such as a family member or friend?\nUser: Yes, I have checked with my family and friends, but none of them made this transaction.\nBank Bot: I see. In this case, I will need to escalate the issue to our dispute resolution team. Can you please confirm that you did not authorize this transaction?\nUser: Yes, I did not authorize this transaction.\nBank Bot: Great, I will initiate the dispute process for you and send you an update via email within the next 5-7 business days. Is there anything else I can assist you with today?\nUser: Could you let me know what the dispute process entails?"
|
| Refuse to Answer | - "User: Hello, I'd like to dispute a transaction on my account.Bank \nBot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorize this transaction.\nBank Bot: Thank you for bringing this to our attention. Could you please provide me with your account number and contact information in case we need to follow up with you?\nUser: I'd rather not give out my personal information."
- "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: That's none of your business. Just help me get my money back."
- "User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: I don't feel comfortable giving out my personal information."
|
| Transfer to Human Agent | - "User: Hi, I want to dispute a transaction on my account.\nBank Bot: Hi there! I'd be happy to assist you with that. Can you please provide me with the details of the transaction you wish to dispute?\nUser: I noticed a charge on my account for $100 that I don't recognize.\nBank Bot: I'm sorry to hear that. Can you please tell me the date the transaction took place and the name of the merchant that charged your account?\nUser: It happened on June 1st and the merchant is called XYZ Company.\nBank Bot: Thanks for the information. We'll need to investigate it further. Can you confirm that you did not authorize this transaction?\nUser: Transfer to a human agent please."
- "User: What the hell is going on with this transaction? I'm having serious issues, and your bot is so useless.\nBot: I'm sorry for any inconvenience caused. Can you please provide me with the transaction details so I can assist you better?\nUser: I don't have time to waste. Why can't you understand my problem? Your bot is such a waste of time.\nBot: I apologize for any frustration you're experiencing. Can you please explain the problem you're facing so I can forward this to our customer support executive?\nUser: Just transfer me to a human agent right now. I'm done with your bot."
- "User: Hi, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. To assist you further, could you please provide me with the details of the transaction you wish to dispute, including the transaction amount, date, and description?\nUser: I've already provided that information. Can I please speak with a human agent about this?"
|
| Questions on Process | - 'User: What the hell is this? I did not authorize a transaction for $100 with "Dan\'s Pizza"! This is ridiculous!\nBot: I\'m sorry for the inconvenience you\'ve experienced, and I would be happy to help. Can you please confirm your name and the last four digits of your credit card number so I can pull up your account and investigate the transaction?\nUser: My name is John Doe and the last four digits of my card are 6789. This is such bullshit, I have never even been to Dan\'s Pizza!\nBot: Thank you for providing me with that information. I understand how frustrating and concerning this is for you, and I apologize for any inconvenience caused. Our team will investigate the transaction and follow up with you within 2-3 business days with more information or next steps.\nUser: I need to know what steps are being taken in this process?'
- "User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: A savings account with higher interest rates.\nBank Assist Bot: Great! Can you please provide me with your name and contact number to begin the process of opening the account?\nUser: Yes, my name is Jane Smith and my contact number is 555-555-5555.\nBank Assist Bot: Perfect! Before we proceed further, can you please confirm that you do not have an existing account with our bank?\nUser: That's correct, I don't.\nBank Assist Bot: Understood. To open a new account, we will need some additional information from you. Can you please provide your date of birth and address?\nUser: Sure, my date of birth is 01/01/1990 and my address is 123 Main Street.\nBank Assist Bot: Thank you for that, Jane. Please note that it might take up to 24-48 hours to process your application and create your account. Is there anything else you need assistance with?\nUser: Yes, actually. Can you explain what happens after I submit my application?"
- 'User: Can you please explain the process for disputing a transaction?\nBank Bot: Absolutely. First, you need to report the transaction as unauthorized or fraudulent within 60 days of the date it appeared on your statement. Then, we will investigate the issue and work with you to resolve it.\nUser: What kind of information do I need to provide during the investigation?\nBank Bot: You will need to provide documentation proving that you did not make the transaction, such as a police report or signed affidavit. In some cases, we may also need additional information about the transaction in question.\nUser: How long does the investigation typically take?'
|
## Evaluation
### Metrics
| Label | Accuracy |
|:--------|:---------|
| **all** | 0.5714 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("User: Hello, I'm having trouble with my ATM card.
Bank Bot: I'm sorry to hear that. To assist you further, could you please provide me with your account number.
User: Yeah, 1234454673838 this is my account number.
Bank Bot: It seems like your account is blocked. We will investigate and get back to you. Do you have any other issues?
User: That's all I needed help with. Thank you for your assistance.")
```
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:---------|:----|
| Word count | 34 | 109.9786 | 227 |
| Label | Training Sample Count |
|:---------------------------------------------------------|:----------------------|
| Continue Conversation | 10 |
| Pause Conversation | 10 |
| Restart Conversation | 10 |
| Repeat Conversation | 10 |
| End Conversation | 10 |
| Refuse to Answer | 10 |
| Affirmative Confirmation | 10 |
| Negative Confirmation | 10 |
| Correction Request | 10 |
| Questions Answerable by Conversational Context / History | 10 |
| Questions Requiring External Information | 10 |
| Questions on Process | 10 |
| New Inquiry or Request | 10 |
| Transfer to Human Agent | 10 |
### Training Hyperparameters
- batch_size: (12, 12)
- num_epochs: (3, 3)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0007 | 1 | 0.3613 | - |
| 0.0330 | 50 | 0.3367 | - |
| 0.0659 | 100 | 0.2621 | - |
| 0.0989 | 150 | 0.1997 | - |
| 0.1318 | 200 | 0.1906 | - |
| 0.1648 | 250 | 0.1034 | - |
| 0.1978 | 300 | 0.0784 | - |
| 0.2307 | 350 | 0.1119 | - |
| 0.2637 | 400 | 0.0694 | - |
| 0.2966 | 450 | 0.0693 | - |
| 0.3296 | 500 | 0.0542 | - |
| 0.3626 | 550 | 0.0669 | - |
| 0.3955 | 600 | 0.0594 | - |
| 0.4285 | 650 | 0.0175 | - |
| 0.4614 | 700 | 0.0125 | - |
| 0.4944 | 750 | 0.0057 | - |
| 0.5274 | 800 | 0.0086 | - |
| 0.5603 | 850 | 0.076 | - |
| 0.5933 | 900 | 0.0077 | - |
| 0.6262 | 950 | 0.0135 | - |
| 0.6592 | 1000 | 0.012 | - |
| 0.6922 | 1050 | 0.0094 | - |
| 0.7251 | 1100 | 0.0735 | - |
| 0.7581 | 1150 | 0.0047 | - |
| 0.7910 | 1200 | 0.0699 | - |
| 0.8240 | 1250 | 0.0063 | - |
| 0.8570 | 1300 | 0.0044 | - |
| 0.8899 | 1350 | 0.0028 | - |
| 0.9229 | 1400 | 0.0706 | - |
| 0.9558 | 1450 | 0.0047 | - |
| 0.9888 | 1500 | 0.0711 | - |
| 1.0218 | 1550 | 0.0036 | - |
| 1.0547 | 1600 | 0.0024 | - |
| 1.0877 | 1650 | 0.1245 | - |
| 1.1206 | 1700 | 0.0044 | - |
| 1.1536 | 1750 | 0.0566 | - |
| 1.1866 | 1800 | 0.0045 | - |
| 1.2195 | 1850 | 0.0046 | - |
| 1.2525 | 1900 | 0.0033 | - |
| 1.2854 | 1950 | 0.0031 | - |
| 1.3184 | 2000 | 0.0095 | - |
| 1.3514 | 2050 | 0.0034 | - |
| 1.3843 | 2100 | 0.0031 | - |
| 1.4173 | 2150 | 0.049 | - |
| 1.4502 | 2200 | 0.0023 | - |
| 1.4832 | 2250 | 0.0034 | - |
| 1.5162 | 2300 | 0.0039 | - |
| 1.5491 | 2350 | 0.0056 | - |
| 1.5821 | 2400 | 0.0027 | - |
| 1.6150 | 2450 | 0.0025 | - |
| 1.6480 | 2500 | 0.0014 | - |
| 1.6809 | 2550 | 0.0029 | - |
| 1.7139 | 2600 | 0.0024 | - |
| 1.7469 | 2650 | 0.0017 | - |
| 1.7798 | 2700 | 0.0018 | - |
| 1.8128 | 2750 | 0.0018 | - |
| 1.8457 | 2800 | 0.0018 | - |
| 1.8787 | 2850 | 0.0025 | - |
| 1.9117 | 2900 | 0.0024 | - |
| 1.9446 | 2950 | 0.0022 | - |
| 1.9776 | 3000 | 0.002 | - |
| 2.0105 | 3050 | 0.0017 | - |
| 2.0435 | 3100 | 0.0021 | - |
| 2.0765 | 3150 | 0.0019 | - |
| 2.1094 | 3200 | 0.0016 | - |
| 2.1424 | 3250 | 0.0017 | - |
| 2.1753 | 3300 | 0.0016 | - |
| 2.2083 | 3350 | 0.0015 | - |
| 2.2413 | 3400 | 0.0017 | - |
| 2.2742 | 3450 | 0.0015 | - |
| 2.3072 | 3500 | 0.0014 | - |
| 2.3401 | 3550 | 0.0012 | - |
| 2.3731 | 3600 | 0.0011 | - |
| 2.4061 | 3650 | 0.0015 | - |
| 2.4390 | 3700 | 0.0016 | - |
| 2.4720 | 3750 | 0.0018 | - |
| 2.5049 | 3800 | 0.0012 | - |
| 2.5379 | 3850 | 0.0021 | - |
| 2.5709 | 3900 | 0.0014 | - |
| 2.6038 | 3950 | 0.0014 | - |
| 2.6368 | 4000 | 0.0013 | - |
| 2.6697 | 4050 | 0.0014 | - |
| 2.7027 | 4100 | 0.0016 | - |
| 2.7357 | 4150 | 0.0016 | - |
| 2.7686 | 4200 | 0.0019 | - |
| 2.8016 | 4250 | 0.0014 | - |
| 2.8345 | 4300 | 0.0015 | - |
| 2.8675 | 4350 | 0.0012 | - |
| 2.9005 | 4400 | 0.0011 | - |
| 2.9334 | 4450 | 0.0013 | - |
| 2.9664 | 4500 | 0.0016 | - |
| 2.9993 | 4550 | 0.0014 | - |
### Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- Transformers: 4.40.1
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```