Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
@@ -1,71 +1,10 @@
|
|
1 |
-
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
<table class="tg" style="padding-left: 30px;">
|
12 |
-
<tr>
|
13 |
-
<th class="tg-0pky">Method</th>
|
14 |
-
<th class="tg-0pky">5-Model Generations ↑</th>
|
15 |
-
<th class="tg-0pky">ChatGPT/GPT-4 Generations ↑</th>
|
16 |
-
<th class="tg-0pky">Speedup ↑</th>
|
17 |
-
</tr>
|
18 |
-
<tr>
|
19 |
-
<td class="tg-0pky">DetectGPT</td>
|
20 |
-
<td class="tg-0pky">0.9554</td>
|
21 |
-
<td class="tg-0pky">0.7225</td>
|
22 |
-
<td class="tg-0pky">1x</td>
|
23 |
-
</tr>
|
24 |
-
<tr>
|
25 |
-
<td class="tg-0pky">Fast-DetectGPT</td>
|
26 |
-
<td class="tg-0pky">0.9887 (relative↑ <b>74.7%</b>)</td>
|
27 |
-
<td class="tg-0pky">0.9338 (relative↑ <b>76.1%</b>)</td>
|
28 |
-
<td class="tg-0pky"><b>340x</b></td>
|
29 |
-
</tr>
|
30 |
-
</table>
|
31 |
-
The table shows detection accuracy (measured in AUROC) and computational speedup for machine-generated text detection. The <b>white-box setting</b> (directly using the source model) is used for detecting generations produced by five source models (5-model), whereas the <b>black-box
|
32 |
-
setting</b> (utilizing surrogate models) targets ChatGPT and GPT-4 generations. AUROC results are averaged across various datasets and source models. Speedup assessments were conducted on a Tesla A100 GPU.
|
33 |
-
|
34 |
-
|
35 |
-
## Environment
|
36 |
-
* Python3.8
|
37 |
-
* PyTorch1.10.0
|
38 |
-
* Setup the environment:
|
39 |
-
```bash setup.sh```
|
40 |
-
|
41 |
-
(Notes: our experiments are run on 1 GPU of Tesla A100 with 80G memory.)
|
42 |
-
|
43 |
-
## Local Demo
|
44 |
-
Please run following command locally for an interactive demo:
|
45 |
-
```
|
46 |
-
python scripts/local_infer.py
|
47 |
-
```
|
48 |
-
where the default reference and sampling models are both gpt-neo-2.7B.
|
49 |
-
|
50 |
-
We could use gpt-j-6B as the reference model to obtain more accurate detections:
|
51 |
-
```
|
52 |
-
python scripts/local_infer.py --reference_model_name gpt-j-6B
|
53 |
-
```
|
54 |
-
|
55 |
-
|
56 |
-
An example (using gpt-j-6B as the reference model) looks like
|
57 |
-
```
|
58 |
-
Please enter your text: (Press Enter twice to start processing)
|
59 |
-
Disguised as police, they broke through a fence on Monday evening and broke into the cargo of a Swiss-bound plane to take the valuable items. The audacious heist occurred at an airport in a small European country, leaving authorities baffled and airline officials in shock.
|
60 |
-
|
61 |
-
Fast-DetectGPT criterion is 1.9299, suggesting that the text has a probability of 87% to be machine-generated.
|
62 |
-
```
|
63 |
-
|
64 |
-
## Workspace
|
65 |
-
Following folders are created for our experiments:
|
66 |
-
* ./exp_main -> experiments for 5-model generations (main.sh).
|
67 |
-
* ./exp_gpt3to4 -> experiments for GPT-3, ChatGPT, and GPT-4 generations (gpt3to4.sh).
|
68 |
-
|
69 |
-
(Notes: we share <b>generations from GPT-3, ChatGPT, and GPT-4</b> in exp_gpt3to4/data for convenient reproduction.)
|
70 |
-
|
71 |
-
|
|
|
1 |
+
---
|
2 |
+
title: "Fast Detect GPT"
|
3 |
+
emoji: "⚡"
|
4 |
+
colorFrom: "blue"
|
5 |
+
colorTo: "purple"
|
6 |
+
sdk: "streamlit"
|
7 |
+
sdk_version: "1.41.0"
|
8 |
+
app_file: app.py
|
9 |
+
pinned: false
|
10 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|