WPAI-INC/WPAIGPT-sql-01

WPAIGPT-SQL-01 is a specialized text-to-SQL model designed for WordPress and WordPress plugins. It generates SQL queries based on natural language requests, with a focus on WordPress-specific database structures and popular plugins.

Model Details

Model Description

WPAIGPT-SQL-01 is a fine-tuned version of the Qwen2.5-Coder-7B model, optimized for generating SQL queries for WordPress databases. It can handle queries related to core WordPress tables as well as tables added by various plugins.

• Developed by: WPAI Inc, James LePage
• Funded by: WPAI Inc
• Model type: Text-to-SQL Language Model
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model: Qwen2.5-Coder-7B-Instruct

Uses

Direct Use

The model is designed for direct text-to-SQL generation for WordPress databases. Users can input natural language requests, optionally including plugin names, versions, and table descriptions, to generate SQL queries. This is particularly useful for:

1. Retrieving information from WordPress databases
2. Adding functionality to existing WordPress plugins by generating SQL queries
3. Assisting developers in creating database queries for WordPress projects

Downstream Use

1. Integration into WPAI products, primarily AgentWP, for real-time information retrieval from WordPress websites
2. Use in code generation tools to create queries for more complete WordPress systems like plugins
3. Incorporation into agent pipelines for WordPress-related tasks

Out-of-Scope Use

While there are no strict out-of-scope uses, users should be aware that as a Transformer-based model, it can potentially hallucinate or generate incorrect queries. All generated SQL should be verified before execution against a live WordPress database.

Bias, Risks, and Limitations

• The model may be biased towards more popular WordPress plugins and those with more extensive database interactions.
• There's a bias towards SELECT and read-only operations over database-modifying queries.
• The model's knowledge is limited to the training data, which may not cover all possible WordPress plugins or database structures.
• As with any language model, there's a risk of generating syntactically correct but logically incorrect or potentially harmful SQL queries.

Recommendations

• Always verify and test generated SQL queries before executing them on a live WordPress database.
• Use in conjunction with proper access controls and user authentication to prevent unauthorized database access.
• Regularly update the model to include knowledge of new WordPress versions and popular plugins.
• Implement additional safety checks and validations when using the model in automated systems.

Training Details

Training Data

The training data consists of hundreds of thousands of instruction-to-SQL examples, structured as follows:

• 25% include described tables that WordPress plugins may add, along with plugin name, version, and instruction
• 25% include only the plugin name, version, and instruction
• 50% include only the instruction

The queries are derived from popular WordPress plugins, both from the official WordPress repository and premium plugins. The data generation process involves:

1. Indexing plugin codebases
2. Extracting code that manipulates the WordPress database
3. Synthetically generating SQL queries
4. Verifying queries by running them against a WordPress installation with the plugin installed

There's a bias towards the most popular WordPress plugins and those with significant database interactions. Additional manual data has been included for specific plugins like WooCommerce, LearnDash, and Gravity Forms.

Training Procedure

The training procedure details are available in the provided Python notebook. For specific information about hyperparameters, preprocessing steps, and other training details, please refer to the notebook.

Evaluation

Testing Data, Factors & Metrics

Formal evaluations have not been conducted. The model's performance is primarily assessed through:

1. A/B testing in WPAI products
2. User rankings on end systems (AgentWP, CodeWP, and other WPAI products)

Technical Specifications

Model Architecture and Objective

The model is based on the Qwen2.5-Coder-7B architecture, fine-tuned for the specific task of WordPress SQL generation. It uses a causal language modeling objective to generate SQL queries based on natural language inputs.

Key features of the base Qwen2.5-Coder-7B model include:

• Number of Parameters: 7.61B
• Number of Layers: 28
• Number of Attention Heads: 28 for Q and 4 for KV (using Grouped-Query Attention)
• Context Length: Full 131,072 tokens (with the ability to handle long contexts using YaRN technique)

The model has been specifically fine-tuned to understand WordPress database structures and generate appropriate SQL queries, maintaining its coding capabilities while focusing on the WordPress ecosystem.