byroneverson
commited on
Commit
•
4da0890
1
Parent(s):
4827840
Update README.md
Browse files
README.md
CHANGED
@@ -14,20 +14,13 @@ library_name: transformers
|
|
14 |
---
|
15 |
|
16 |
|
17 |
-
NOTE: This is a current WIP (work in progress).
|
18 |
|
19 |
-
|
20 |
-
|
21 |
-
1. Obtain refusal direction with llama-cpp-python.
|
22 |
-
2. Orthogonalization performed with torch directly to .safetensors. (one at a time)
|
23 |
-
|
24 |
-
It is a rather larger model so it may take me another day or two to figure out which layer I should be using for the direction vector.
|
25 |
-
|
26 |
-
First attempt: Layer 20 was used to obtain refusal direction vector. Refusal mitigation sort of worked but not perfect.
|
27 |
|
28 |
-
|
|
|
|
|
29 |
|
30 |
-
# gemma-2-27b-it-abliterated
|
31 |
Check out the <a href="https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/blob/main/abliterate-gemma-2-27b-it.ipynb">jupyter notebook</a> for details of how this model was abliterated from glm-4-9b-chat.
|
32 |
|
33 |
![Logo](https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/resolve/main/logo.png "Logo")
|
|
|
14 |
---
|
15 |
|
16 |
|
|
|
17 |
|
18 |
+
# gemma-2-27b-it-abliterated
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
|
20 |
+
This is a novel approach to abliterating a model on CPU only.
|
21 |
+
1. Obtain refusal direction vector using a quant model with llama.cpp (llama-cpp-python and ggml-python).
|
22 |
+
2. Orthogonalize each .safetensors files directly from original repo and upload to a new repo. (one at a time)
|
23 |
|
|
|
24 |
Check out the <a href="https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/blob/main/abliterate-gemma-2-27b-it.ipynb">jupyter notebook</a> for details of how this model was abliterated from glm-4-9b-chat.
|
25 |
|
26 |
![Logo](https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/resolve/main/logo.png "Logo")
|