byroneverson commited on
Commit
4da0890
1 Parent(s): 4827840

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -11
README.md CHANGED
@@ -14,20 +14,13 @@ library_name: transformers
14
  ---
15
 
16
 
17
- NOTE: This is a current WIP (work in progress).
18
 
19
- Abliteration method:
20
-
21
- 1. Obtain refusal direction with llama-cpp-python.
22
- 2. Orthogonalization performed with torch directly to .safetensors. (one at a time)
23
-
24
- It is a rather larger model so it may take me another day or two to figure out which layer I should be using for the direction vector.
25
-
26
- First attempt: Layer 20 was used to obtain refusal direction vector. Refusal mitigation sort of worked but not perfect.
27
 
28
- Second attempt: (Current) Layer 23 was used (mid-point of model). Half-way has proven to work with other model so this should be fine.
 
 
29
 
30
- # gemma-2-27b-it-abliterated
31
  Check out the <a href="https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/blob/main/abliterate-gemma-2-27b-it.ipynb">jupyter notebook</a> for details of how this model was abliterated from glm-4-9b-chat.
32
 
33
  ![Logo](https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/resolve/main/logo.png "Logo")
 
14
  ---
15
 
16
 
 
17
 
18
+ # gemma-2-27b-it-abliterated
 
 
 
 
 
 
 
19
 
20
+ This is a novel approach to abliterating a model on CPU only.
21
+ 1. Obtain refusal direction vector using a quant model with llama.cpp (llama-cpp-python and ggml-python).
22
+ 2. Orthogonalize each .safetensors files directly from original repo and upload to a new repo. (one at a time)
23
 
 
24
  Check out the <a href="https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/blob/main/abliterate-gemma-2-27b-it.ipynb">jupyter notebook</a> for details of how this model was abliterated from glm-4-9b-chat.
25
 
26
  ![Logo](https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated/resolve/main/logo.png "Logo")