📡 LWM: Large Wireless Model

🚀 Try the Interactive Demo on Hugging Face!

Welcome to the LWM (Large Wireless Model) repository! LWM is a powerful pre-trained model designed to extract rich and high-quality features from wireless communication datasets, such as the DeepMIMO dataset. This model leverages advanced neural architectures to efficiently handle wireless channel data, making it applicable in a wide range of tasks like channel prediction, classification, and beamforming.

LWM was built with robust generalization capabilities, ensuring it performs well even on datasets it hasn’t seen before, making it an ideal tool for both research and real-world applications in wireless communications. Read on to learn how to set up, run, and explore the features of LWM.

✨ Key Features

Wireless Channel Embeddings: LWM is trained to extract meaningful embeddings from wireless channel data, capturing complex features that can be used in various downstream tasks.
Flexible Input: Whether you're working with raw channel data or compressed embeddings, LWM supports different data formats, offering versatility in wireless data processing.
Efficient Inference: LWM's architecture is optimized for quick and scalable inference, providing fast results even on large datasets.
Generalization Power: Tested on several unseen datasets, LWM maintains high-quality performance without overfitting, proving its effectiveness in diverse environments.

🛠 How to Use

1. Install Conda or Mamba

To begin, install Conda or Mamba for managing Python environments and packages.

Conda: Download and install Miniconda for a lightweight environment.
Mamba (via Miniforge): Miniforge is a faster alternative to Conda, with Mamba pre-installed for quicker package installations.

2. Set Up Your Environment

Step 1: Create a new environment

Create a new Python environment named lwm_env (or any name you prefer).

# If you're using Conda:
conda create -n lwm_env python=3.12

# If you're using Mamba:
mamba create -n lwm_env python=3.12

Step 2: Activate the environment

Activate the new environment:

conda activate lwm_env

3. Install Required Packages

Install the necessary Python packages for LWM.

# Using Conda or Mamba to install PyTorch
conda install pytorch torchvision torchaudio -c pytorch

# Install additional dependencies with pip
pip install -r requirements.txt

Note: If requirements.txt includes all necessary dependencies, you can install everything in one go by running pip install -r requirements.txt.

Main package requirements include:

import torch
import numpy as np
import pandas as pd
import DeepMIMOv3
import os
import pickle
import shutil
import warnings
from tqdm import tqdm
from datetime import datetime
from torch.utils.data import Dataset, DataLoader

4. Clone the Model Repository

# Step 1: Clone the model repository (if not already cloned)
model_repo_url = "https://huggingface.co/sadjadalikhani/lwm"
model_repo_dir = "./LWM"

if not os.path.exists(model_repo_dir):
    print(f"Cloning model repository from {model_repo_url}...")
    subprocess.run(["git", "clone", model_repo_url, model_repo_dir], check=True)

5. Clone the Desired Datasets

LWM is designed to process datasets from various environments, such as the DeepMIMO dataset. Before running inference, ensure the datasets are cloned into your local environment. Below is an overview of some of the available datasets and their respective links:

📊 Dataset Overview

📊 Dataset	🏙️ City	👥 Number of Users	🔗 DeepMIMO Page
Dataset 0	🌆 Denver	1354	DeepMIMO City Scenario 18
Dataset 1	🏙️ Indianapolis	3248	DeepMIMO City Scenario 15

# Function to clone specific dataset scenario
def clone_dataset_scenario(scenario_name, repo_url, model_repo_dir="./LWM", scenarios_dir="scenarios"):
    # Logic for cloning datasets
    pass

6. Tokenize and Load the Model

Once you have cloned the model and the datasets, you can preprocess the data and load the model.

from input_preprocess import tokenizer
from lwm_model import lwm

# Tokenizing the dataset
preprocessed_chs = tokenizer(selected_scenario_names=["city_18_denver"], manual_data=None, gen_raw=True)

# Load LWM model
device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(f"Loading the LWM model on {device}...")
model = lwm.from_pretrained(device=device)

7. Perform Inference

After tokenizing the data and loading the model, you're ready to perform inference with LWM.

from inference import lwm_inference, create_raw_dataset

input_types = ['cls_emb', 'channel_emb', 'raw']
selected_input_type = 'cls_emb'  # Choose the type of input

if selected_input_type in ['cls_emb', 'channel_emb']:
    dataset = lwm_inference(preprocessed_chs, selected_input_type, model, device)
else:
    dataset = create_raw_dataset(preprocessed_chs, device)

LWM Architecture and Usage in Wireless Tasks

LWM employs a robust neural network architecture to extract valuable features from wireless channel data. Its deep layers, combined with advanced tokenization, make it suitable for challenging wireless communication tasks, such as beamforming, interference management, and channel state prediction.

LWM is particularly effective in generalizing across diverse environments without overfitting, and its results are consistent across raw channel data and embeddings.

By using LWM, you can enhance your wireless communication systems by utilizing data-driven models that outperform traditional approaches, offering faster and more accurate results.