microsoft/DialogRPT-human-vs-rand

Demo

Please try this ➤➤➤ Colab Notebook Demo (click me!)

Context	Response	human_vs_rand score
I love NLP!	He is a great basketball player.	0.027
I love NLP!	Can you tell me how it works?	0.754
I love NLP!	Me too!	0.631

The human_vs_rand score predicts how likely the response is corresponding to the given context, rather than a random response.

DialogRPT-human-vs-rand

Dialog Ranking Pretrained Transformers

How likely a dialog response is upvoted 👍 and/or gets replied 💬?

This is what DialogRPT is learned to predict. It is a set of dialog response ranking models proposed by Microsoft Research NLP Group trained on 100 + millions of human feedback data. It can be used to improve existing dialog generation model (e.g., DialoGPT) by re-ranking the generated response candidates.

Quick Links:

• EMNLP'20 Paper
• Dataset, training, and evaluation
• Colab Notebook Demo

We considered the following tasks and provided corresponding pretrained models.

Task	Description	Pretrained model
Human feedback	given a context and its two human responses, predict...
updown	... which gets more upvotes?	model card
width	... which gets more direct replies?	model card
depth	... which gets longer follow-up thread?	model card
Human-like (human vs fake)	given a context and one human response, distinguish it with...
human_vs_rand	... a random human response	this model
human_vs_machine	... a machine generated response	model card

Contact:

Please create an issue on our repo

Citation:

@inproceedings{gao2020dialogrpt,
    title={Dialogue Response RankingTraining with Large-Scale Human Feedback Data},
    author={Xiang Gao and Yizhe Zhang and Michel Galley and Chris Brockett and Bill Dolan},
    year={2020},
    booktitle={EMNLP}
}