Update README.md

README.md

@@ -1,3 +1,382 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+# The GDPR Input Detection and Anonymization model
+
+The **The GDPR Input Detection and Anonymization model**  is designed to protect sensitive information locally before it is processed by larger AI models in external clouds. 
+
+External cloud suppliers store every input for at least 30 days!
+
+Inference costs of small or big model differ by factor 100!
+
+You can't give LLM access to employees without caring about GDPR and company compliance.
+
+
+## Intended Use
+
+The model is made to bridge the user inputs to external LLM input like a firewall or proxy
+
+The model analysis the user prompts and computes two scores. 
+
+The first score helps to identify if it needs a small or more cabable model to process the user input. The second score rates the sensitivity of the prompt. When it detects sensitive information, the further cloud processing of the prompt can be blocked or at least be replaced by an anonymized version.
+
+
+## Complexity Scoring
+
+The score rates the complexity of a task on a scale from 1 to 10, where **1 represents simple information retrieval of world knowledge** and **10 involves highly complex, expert-level analysis of domain specific content**. With lower scores, smaller models like **LLaMA** are sufficient, while mid to high scores suggest using more powerful models like from OpenAI or Anthropic. 
+
+Also the score incresease, if the number of instructions and contraints in a prompt increase.
+
+This scoring system provides guidance for selecting the right model, helping to **save costs** and **optimize resources** by aligning the task’s complexity with the appropriate computational power.
+
+
+| **Score** | **Description**                             | **Example Tasks**                                                | Number of Instructions and Constraints* | **Suggested Models**               |
+|:---------:|--------------------------------------------|-------------------------------------------------------------------|-|------------------------------------|
+| **1**     | Basic fact retrieval or identification      | "What is the capital of France?"                                 | 1 | Llama, smaller models              |
+| **2**     | Simple rephrasing or minor adjustments      | "Paraphrase the sentence: 'The dog barked loudly.'"              | 1 | Llama, GPT-4o Mini                 |
+| **3**     | Summaries or brief overviews                | "Summarize the key points of a short text."                      | 1 | Llama, GPT-4o Mini                 |
+| **4**     | List creation or short reasoning            | "List three reasons why the following exercise is more beneficial in Greek than in Italy: Here is the exercise: ..."                 | 2 | GPT-4o Mini                        |
+| **5**     | Key point extraction, organized info        | "Exctract challenges and solutions as bullet points. no summary, no intro. Here is the text: ..."       | 2 | GPT-4o Mini                        |
+| **6**     | Basic narrative or simple analysis          | "Write a short story about a character learning a lesson. Written in Harry Potter style, but not about Harry potter, but for coding amateur, humorous"       | 3 | GPT-4o Mini                        |
+| **7**     | Multi-step reasoning, scenario-based tasks  | "Base on the following sprint reports, describe the challenges for project manager, team-building, psychologically, deadline-restriction, building technical doubt etc."          | 3+ | GPT-4o Mini or GPT-4o              |
+| **8**     | In-depth explanations or complex analysis   | "I will give you 3 text snippets. Explain how climate change affects ocean currents, find overlapping arugments and disargreements of the authors, and use the Harvard Concept to solve the discrepancies"             | 3+ | GPT-4o                             |
+| **9**     | Advanced analysis or cross-disciplinary     | "Check the next 200 mails and filter out conversations between engineering and sales that are helpfull and not helpfull for the company. Give me the result as CSV-Table."         | 3+ | GPT-4o                             |
+| **10**    | Expert-level analysis and synthesis         | "Write a report on AI’s impact on real estate invest decision in the middle class, covering the following the main trends from 2025 to 2030 in consumption development, given by the following reports of ten different organizations. Have cultural differences in mind around the world and convert to german culture"| 3+ | GPT-4o                         |
+
+
+
+## Sensitivity Scoring
+
+The sensitivity score rates the **confidentiality** of a task on a scale from 0 to 3, where **0 represents non-sensitive, public information** (e.g., event announcements) and **3 involves highly confidential, critical data** (e.g., medical reports). Lower scores indicate content that can be shared freely, while higher scores require **secure handling** and **anonymization**. This system ensures sensitive content is properly protected, preventing any **unauthorized exposure**.
+
+Here's the revised table with the addition of a GDPR-related example for each sensitivity level:
+
+| **Score** | **Sensitivity and Description**                                                                                 | **Example 1**                                                                                             | **Example 2**                                                                                         |
+|:---------:|-----------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|
+| **0**     | **Public, Non-Sensitive** — Free to share; contains no confidential or personal information.                    | “Community Picnic this Saturday at Greenfield Park, 11 AM. RSVP by Thursday!”                            | "Company Press Release: New sustainability initiatives for 2024."                                     |
+| **1**     | **Internal, Low Sensitivity** — For internal use only but only minor impact if disclosed.                       | “New office hours starting next month: 9 AM - 6 PM Mon-Thurs, half-day Fri.”                             | "Employee List for Training Day: Names only, no personal data."                                       |
+| **2**     | **Confidential, Medium Sensitivity** — For managers or leadership; could impact finances or reputation if disclosed. | “Q3 Budget Breakdown: Marketing: $150K, R&D: $200K”                                                      | "Customer Analysis Report: Aggregate data only, segmented by region."                                 |
+| **3**     | **Highly Confidential, Critical Sensitivity** — Restricted to top executives or medical professionals; severe harm if disclosed. | “Merger Negotiation: Acme to acquire 75% of BetaTech for $2.5B…”                                         | "Employee Health Records: Only accessible to HR and necessary medical staff in compliance with GDPR." |
+
+
+## Annonymization and Re-Annonymization
+
+The model can be used locally before and after the processing data with external models
+
+| Step                                | Content                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+|-------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Anonymization and Re-Anonymization** |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+| **User Input**                      | We have a community picnic this Saturday at Greenfield Park, 11 AM. RSVP by Thursday. Write me an e-mail announcement.                                                                                                                                                                                                                                                                                                                                                                   |
+| **Anonymized Version by Local Model** | We have a community picnic this Saturday at Maplewood Park, Late Morning. RSVP by Thursday. Write me an e-mail announcement.                                                                                                                                                                                                                                                                                                                                                             |
+| **Processed with Main LLM (GPT-4)**  | **Subject:** 🍃 Community Picnic This Saturday at Maplewood Park! 🌞  <br>  <br> Hello [Community/Neighborhood] Friends, We’re excited to announce our Community Picnic happening this Saturday at Maplewood Park! This is a wonderful opportunity to relax, connect with neighbors, and enjoy some outdoor fun. We’ll gather late morning (around 10:30 AM), so bring your favorite snacks, picnic blankets, and plenty of smiles!  <br>  <br> **Details:**  <br> 📅 **Date:** Saturday, [Exact Date]  <br> 📍 **Location:** Maplewood Park  <br> ⏰ **Time:** Late Morning (around 10:30 AM)  <br>  <br> We’ll have games, music, and a few surprises, so it’ll be a great time for all ages! Feel free to bring family, friends, and any special treats you’d like to share with the group.  <br>  <br> **RSVP**  <br> To help us plan, please RSVP by this Thursday by replying to this email or contacting [Contact Information]. We can’t wait to see you all there and make this picnic a memorable one for our community. Don’t forget your sunscreen and a lawn chair if you’d like some extra comfort!  <br>  <br> **Warm regards,** [Your Name/Community Group] |
+| **Entities Replaced by Local Model** | **Subject:** 🍃 Community Picnic This Saturday at Greenfield Park! 🌞  <br>  <br> Hello [Community/Neighborhood] Friends, We’re excited to announce our Community Picnic happening this Saturday at Greenfield Park! This is a wonderful opportunity to relax, connect with neighbors, and enjoy some outdoor fun. We’ll gather late morning (around 10:30 AM), so bring your favorite snacks, picnic blankets, and plenty of smiles!  <br>  <br> **Details:**  <br> 📅 **Date:** Saturday, [Exact Date]  <br> 📍 **Location:** Greenfield Park  <br> ⏰ **Time:** Late Morning (around 10:30 AM)  <br>  <br> We’ll have games, music, and a few surprises, so it’ll be a great time for all ages! Feel free to bring family, friends, and any special treats you’d like to share with the group.  <br>  <br> **RSVP**  <br> To help us plan, please RSVP by this Thursday by replying to this email or contacting [Contact Information]. We can’t wait to see you all there and make this picnic a memorable one for our community. Don’t forget your sunscreen and a lawn chair if you’d like some extra comfort!  <br>  <br> **Warm regards,** [Your Name/Community Group] |
+
+
+
+## How to Use the Model
+
+This model provides functionalities for sensitivity assessment, complexity assesment, and entity anonymization. Each function is illustrated below with example code snippets.
+
+<details>
+<summary>Model setup and inference</summary>
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+model_path = "metricspace/GDPR_Input_Detection_and_Anonymization"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16 if device == "cuda" else torch.float32).to(device)
+
+import re
+import json
+
+# Set tokenizer tokens and model padding
+tokenizer.pad_token = "<|im_start|>"
+tokenizer.eos_token = "<|im_end|>"
+tokenizer.padding_side = "left"
+model.generation_config.pad_token_id = tokenizer.pad_token_id
+
+# Define formats for different analysis types
+formats = {
+    "sensitivity": """<|im_start|>system\nSensitivity<|im_end|>\n<|im_start|>user\n{text}<|im_end|>\n<|im_start|>assistant\n""",
+    "complexity": """<|im_start|>system\nComplexity<|im_end|>\n<|im_start|>user\n{text}<|im_end|>\n<|im_start|>assistant\n""",
+    "entity_detection": """<|im_start|>system\nEntity Detection<|im_end|>\n<|im_start|>user\n{text}<|im_end|>\n<|im_start|>assistant\n""",
+    "entity_swapping": """<|im_start|>system\nEntity Swapping<|im_end|>\n<|im_start|>user\nentities:{entities}\ntext:\n{text}<|im_end|>\n<|im_start|>assistant\n"""
+}
+
+def model_inference(text, mode="anonymization", max_new_tokens=50, config=None, entity_mapping=None):
+    if mode not in formats and mode != "anonymization":
+        raise ValueError("Invalid mode. Choose from 'sensitivity', 'complexity', 'entity_detection', 'anonymization'.")
+
+    
+    # Configuration for anonymization
+    # The `config` dictionary specifies the anonymization behavior for each type of entity detected.
+    # Each key in `config` represents an entity type (e.g., "LOC" for location, "PERSON" for personal names),
+    # and the value assigned to that key determines how entities of that type should be anonymized:
+    # 
+    # - "RANDOM": Replaces the entity with a randomly selected placeholder.
+    # - "GENERAL LOW", "GENERAL MEDIUM", "GENERAL HIGH": Replaces the entity with a generalized label,
+    #   with the intensity level (LOW, MEDIUM, HIGH) controlling the specificity. For example, 
+    #   "GENERAL LOW" might use a more specific label ("Local Park") while "GENERAL HIGH" would use
+    #   a broader label ("Recreational Area").
+    # 
+    # This allows fine-grained control over anonymization, ensuring that different types of sensitive 
+    # information can be replaced in ways that are appropriate for the context. For example:
+    #   - "LOC": "RANDOM" replaces any detected location with a random placeholder.
+    #   - "DATETIME": "GENERAL LOW" uses a lower-intensity generalization for dates and times.
+    # 
+    # This flexibility enables custom anonymization policies to suit different privacy or obfuscation needs.
+
+    if config is None:
+        config = {
+            "LOC": "RANDOM",
+            "PERSON": "RANDOM",
+            "DEM": "RANDOM",
+            "CODE": "RANDOM",
+            "ORG": "GENERAL MEDIUM",
+            "DATETIME": "GENERAL LOW",
+            "QUANTITY": "RANDOM",
+            "MISC": "RANDOM",
+        }
+
+    # Anonymization Mode
+    if mode == "anonymization":
+        # Step 1: Entity detection
+        detection_prompt = formats["entity_detection"].format(text=text)
+        detection_inputs = tokenizer(detection_prompt, return_tensors="pt").to(device)
+        detection_output = model.generate(
+            **detection_inputs,
+            max_new_tokens=max_new_tokens,
+            use_cache=True,
+            eos_token_id=151645
+        )
+        detection_text = tokenizer.decode(detection_output[0], skip_special_tokens=True)
+        detected_entities = postprocess_entity_recognition(detection_text)
+        
+        # Step 2: Select entities based on config
+        selected_entities = select_entities_based_on_json(detected_entities, config)
+        entities_str = "\n".join([f"{entity} : {label}" for entity, label in selected_entities])
+
+        # Step 3: Entity swapping for anonymization
+        swapping_prompt = formats["entity_swapping"].format(entities=entities_str, text=text)
+        swapping_inputs = tokenizer(swapping_prompt, return_tensors="pt").to(device)
+        swapping_output = model.generate(
+            **swapping_inputs,
+            max_new_tokens=max_new_tokens,
+            use_cache=True,
+            eos_token_id=151645
+        )
+        
+        anonymized_text = tokenizer.decode(swapping_output[0], skip_special_tokens=True)
+        anonymized_text = anonymized_text.split("assistant\n", 1)[-1].strip()  # Extract only the assistant's response
+
+        return anonymized_text, detected_entities
+
+    # Entity Restoration Mode using entity_swapping
+    elif mode == "entity_swapping" and entity_mapping:
+        # Aggregate RANDOM and GENERAL replacements for restoration
+        reversed_entities = []
+        for original, details in entity_mapping.items():
+            # Include RANDOM replacement
+            reversed_entities.append(f"{details['RANDOM']} : {original}")
+            # Include GENERAL replacements
+            for general_label, _ in details["GENERAL"]:
+                reversed_entities.append(f"{general_label} : {original}")
+        
+        # Combine all replacement mappings for the prompt
+        reversed_entities_str = "\n".join(reversed_entities)
+
+        # Create the swapping prompt with the aggregated reversed mappings
+        swapping_prompt = formats["entity_swapping"].format(entities=reversed_entities_str, text=text)
+        swapping_inputs = tokenizer(swapping_prompt, return_tensors="pt").to(device)
+        swapping_output = model.generate(
+            **swapping_inputs,
+            max_new_tokens=max_new_tokens,
+            use_cache=True,
+            eos_token_id=151645
+        )
+
+        # Decode and extract the restored text
+        output_text = tokenizer.decode(swapping_output[0], skip_special_tokens=True)
+        output_text = output_text.split("assistant\n", 1)[-1].strip()  # Extract only the assistant's response
+
+        return output_text
+
+    # Other modes (sensitivity, complexity, entity_detection)
+    else:
+        prompt = formats[mode].format(text=text)
+        model_inputs = tokenizer(prompt, return_tensors="pt").to(device)
+        generation_output = model.generate(
+            **model_inputs,
+            max_new_tokens=max_new_tokens,
+            use_cache=True,
+            eos_token_id=151645
+        )
+        full_output = tokenizer.decode(generation_output[0], skip_special_tokens=True)
+        
+        if mode in ["sensitivity", "complexity"]:
+            assistant_text = full_output.split("assistant\n", 1)[-1].strip()
+            return assistant_text
+        elif mode == "entity_detection":
+            return postprocess_entity_recognition(full_output)
+
+# Function to parse entity detection output
+def postprocess_entity_recognition(detection_output: str) -> dict:
+    output_json = {}
+    entity_pattern = re.compile(
+        r'(?P<entity>[\w\s]+)--(?P<type>[\w]+)--(?P<random>[\w\s]+)--(?P<generalizations>.+)'
+    )
+    generalization_pattern = re.compile(r'(\d+)::([\w\s]+)')
+    
+    lines = detection_output.strip().split("\n")
+    for line in lines:
+        match = entity_pattern.search(line)
+        if match:
+            entity_name = match.group("entity").strip()
+            entity_type = match.group("type").strip()
+            random_replacement = match.group("random").strip()
+            
+            generalizations = []
+            for gen_match in generalization_pattern.findall(match.group("generalizations")):
+                score, label = gen_match
+                generalizations.append([label.strip(), score.strip()])
+
+            output_json[entity_name] = {
+                "TYPE": entity_type,
+                "RANDOM": random_replacement,
+                "GENERAL": generalizations
+            }
+    return output_json
+
+# Function to select entities based on config
+def select_entities_based_on_json(prediction_json, entity_json):
+    entities = []
+    for key, value in prediction_json.items():
+        entity_type = value["TYPE"]
+        if entity_type in entity_json:
+            anonymization_type = entity_json[entity_type]
+            if anonymization_type == "RANDOM":
+                entities.append([key, value["RANDOM"]])
+            elif "GENERAL" in anonymization_type:
+                intensity = anonymization_type.split(" ")[1]
+                if intensity == "LOW" and value["GENERAL"]:
+                    entities.append([key, value["GENERAL"][0][0]])
+                elif intensity == "MEDIUM":
+                    for gen in value["GENERAL"]:
+                        if int(gen[1]) >= 4:
+                            entities.append([key, gen[0]])
+                            break
+                elif intensity == "HIGH":
+                    if value["GENERAL"]:
+                        entities.append([key, value["GENERAL"][0][0]])
+    return entities
+```
+</details>
+
+### 1. Sensitivity and Complexity Analysis
+
+Example text
+```
+We have a community picnic this Saturday at Greenfield Park, 11 AM. RSVP by Thursday, Write me an e-mail annoucment!
+```
+
+The sensitivity analysis feature evaluates the sensitivity of a given text and the complexitivity feature rates the complexity.
+```python
+text = "We have a community picnic this Saturday at Greenfield Park, 11 AM. RSVP by Thursday, Write me an e-mail annoucment!"
+
+# Generate sensitivity score
+sensitivity_score = model_inference(text, mode="sensitivity")
+print(f"Sensitivity Score: {sensitivity_score}" "\n")
+
+# Generate complexity score
+complexity_score = model_inference(text, mode="complexity")
+print(f"Complexity: {complexity_score}" "\n")
+```
+
+Output
+```
+Sensitivity Score: 0
+Complexity Score: 3
+```
+
+### 3. Anonymization and Re-Anonymization
+
+To protect sensitive information, the model detects specific entities in the text and anonymizes them based on a pre-configured setting.
+
+
+```python
+# Anonymize the text
+anonymized_text, entity_mapping = model_inference(text, mode="anonymization")
+print(f"Anonymized Text: {anonymized_text}\n")
+
+# Restore the original text
+restored_text = model_inference(anonymized_text, mode="entity_swapping", entity_mapping=entity_mapping)
+print(f"Restored Text: {restored_text}")
+```
+
+Output
+```
+Anonymized Text: We have a community picnic this Saturday at Maplewood Park, Late Morning. RSVP by Thursday, Write me an e-mail announcement.
+
+Restored Text: We have a community picnic this Saturday at Greenfield Park, 11 AM. RSVP by Thursday, Write me an e-mail announcement.
+```
+
+This is how the stored entitiy maps looks like
+```python
+print(f"{json.dumps(entity_mapping, indent=4)}")
+```
+Output
+```
+{
+    "Greenfield Park": {
+        "TYPE": "LOC",
+        "RANDOM": "Maplewood Park",
+        "GENERAL": [
+            [
+                "Local Park",
+                "3"
+            ],
+            [
+                "Public Park",
+                "5"
+            ],
+            [
+                "Recreational Area",
+                "7"
+            ]
+        ]
+    },
+    "11 AM": {
+        "TYPE": "DATETIME",
+        "RANDOM": "1 PM",
+        "GENERAL": [
+            [
+                "Late Morning",
+                "2"
+            ],
+            [
+                "A",
+                "4"
+            ]
+        ]
+    }
+}
+```
+
+Normally you would process the annonymized version with a LLM and than reanonymize the result back.
+
+
+
+## Limitations
+
+For **complexity and sensitivity scoring**, the model can process inputs up to **2,048 tokens**. However, for **entity detection**, the combined limit for input and output is **3,000 tokens**. If a text is too long or contains numerous entities, this limit may be exceeded, which could result in truncated outputs or inconsistent behavior. To ensure accurate results, it's recommended to keep input lengths within these token limits.
+
+## Language Support
+
+The model supports 29 language. It was trained with a mixture of (80% english examples, 20% multilanguage examples)
+
+## Licence
+
+Apache license 2.0