app.py

import streamlit as st
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
from statsmodels.stats.outliers_influence import variance_inflation_factor
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import cross_validate
import time
from sklearn.model_selection import learning_curve

def show_intro():
    with st.expander(f"➡️ What is Regression?"):
        st.markdown("""
        **Regression** is a fundamental statistical technique used to understand and quantify the relationship between a **dependent variable (what you want to predict)** and one or more **independent variables (predictors)**.

        ---
        ###### 🔍 Everyday Examples of Regression:
        - 📈 Predicting **house prices** based on size, location, and number of bedrooms.
        - 🎓 Estimating a student’s **final grade** based on hours of study and attendance.
        - 🚗 Forecasting **fuel efficiency** based on engine size and weight of the car.
        - 🧠 Predicting **IQ scores** or **height** based on parental traits (enter Galton! 👇)
        
        ---

        ###### 👨‍👩‍👧‍👦 Galton’s Theory – *Regression to the Mean*
        Sir Francis Galton, a 19th-century statistician and cousin of Charles Darwin, studied the heights of parents and their children. 

        He observed:
        - Very tall parents tended to have children **shorter** than themselves.
        - Very short parents tended to have children **taller** than themselves.
        
        🧠 He coined the term **"regression to the mean"**, which means:
        > "Extreme traits tend to be followed by traits closer to the average in the next generation."

        ---
        ###### 👶 Real-Life Example:
        - If both parents are exceptionally tall (say, 6'5"), their child is **likely tall**, but **closer to the average height** than the parents — maybe 6'2".
        - Similarly, if parents are very short, the child’s height tends to “regress” toward the average population height.

        This pattern **doesn't mean height is random**, just that genetics and environment **pull traits toward typical values** over time.

        ---
        Regression models in ML extend this idea — instead of modeling parent-child height, we model **any continuous outcome** based on relevant input variables.
        """)
        
    with st.expander("➡️ Industry Use-Cases of Regression Models"):
        st.markdown("""
        ###### 🏥 Healthcare
        - 🔬 Estimating **patient recovery time** based on age, treatment type, and initial condition.
        - 💉 Predicting **blood glucose levels** based on dietary habits and medication dosage.
        - 🫀 Forecasting **hospital readmission rates** based on prior health records and discharge details.

        ###### 🛒 Retail
        - 📦 Predicting **sales volume** based on pricing, seasonality, and promotional campaigns.
        - 🛍️ Estimating **inventory demand** for specific SKUs using historical sales and trends.
        - 👗 Forecasting **customer churn** likelihood using past purchase behavior and returns.

        ###### 🛍️ E-commerce
        - 💸 Predicting **customer lifetime value (CLV)** based on purchase frequency and basket size.
        - 🚚 Estimating **delivery time** based on warehouse location, item type, and order volume.
        - 🧾 Forecasting **return probability** of products based on description, images, and reviews.

        ###### 💰 Finance
        - 📊 Predicting **stock prices** or **bond yields** based on historical trends and market indicators.
        - 🏦 Estimating **credit risk** or **loan default probability** using income, credit history, etc.
        - 💳 Forecasting **spending patterns** on credit cards based on customer behavior.

        ###### 💊 Pharma & Life Sciences
        - 🧪 Predicting **drug efficacy** based on dosage and patient demographics in clinical trials.
        - 🦠 Estimating **disease progression** timelines based on early symptoms and test results.
        - 💊 Forecasting **adverse drug reactions** from formulation and patient profiles.
        """)
        
def simple_regression_example():
    with st.expander("➡️ Single Variable Regression (Manual Calculation)"):
        # Sample data
        advertising_spend = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
        sales_revenue = np.array([2.1, 3.9, 5.2, 6.0, 7.1, 8.1, 9.0, 10.2, 10.8, 12.0])

        # Regression coefficients (manual calculation)
        x_mean = np.mean(advertising_spend)
        y_mean = np.mean(sales_revenue)
        b1 = np.sum((advertising_spend - x_mean) * (sales_revenue - y_mean)) / np.sum((advertising_spend - x_mean)**2)
        b0 = y_mean - b1 * x_mean
        predicted_sales = b0 + b1 * advertising_spend

        # Two-column layout
        col1, col2 = st.columns(2)

        with col1:
            st.markdown("###### 📊 Sample Data")
            df = pd.DataFrame({
                'Advertising Spend (in lakhs)': advertising_spend,
                'Sales Revenue (in lakhs)': sales_revenue
            })
            st.dataframe(df)

        with col2:
            st.markdown("###### 📉 Linear Regression Formula")
            st.markdown(f"""
            The linear regression equation is:  
            **Sales Revenue = {b0:.2f} + {b1:.2f} × Advertising Spend**
            
            Where:  
            - **b₀ (Intercept)**: Sales revenue when advertising spend is zero.  
            - **b₁ (Slope)**: Increase in revenue for each additional lakh spent.

            ###### Formula for Computing Coefficients
            - **b₁ (Slope)** = (Σ(xᵢ - x̄)(yᵢ - ȳ)) / Σ(xᵢ - x̄)²  
            - **b₀ (Intercept)** = ȳ - b₁ × x̄
            """)

        # Plotting the regression line
        fig, ax = plt.subplots(figsize=(9,4))
        ax.scatter(advertising_spend, sales_revenue, color='blue', label='Actual')
        ax.plot(advertising_spend, predicted_sales, color='red', label='Fitted Line')
        ax.set_xlabel("Advertising Spend (in lakhs)", fontsize=10)
        ax.set_ylabel("Sales Revenue (in lakhs)", fontsize=10)
        ax.set_title("Linear Regression: Advertising Spend vs Sales Revenue", fontsize=10)
        ax.tick_params(axis='both', labelsize=8)
        ax.legend()
        st.pyplot(fig)

    with st.expander("➡️ Predict Sales Revenue from Advertising Spend"):
        st.markdown(f"Use the trained regression model to forecast expected sales revenue 📈")

        user_input = st.number_input(
            "Enter Advertising Spend (in lakhs)", 
            min_value=1.0, 
            max_value=20.0, 
            value=5.0, 
            step=0.5, 
            format="%.1f"
        )

        if user_input:
            predicted_value = b0 + b1 * user_input
            st.success(f"🔮 Predicted Sales Revenue: **{predicted_value:.2f} lakhs**")

            # Visualize prediction on the regression chart
            fig, ax = plt.subplots(figsize=(9,4))
            ax.scatter(advertising_spend, sales_revenue, color='blue', label='Actual')
            ax.plot(advertising_spend, predicted_sales, color='red', label='Fitted Line')

            # Add dashed lines for prediction
            ax.axvline(x=user_input, color='red', linestyle='--', linewidth=1)
            ax.axhline(y=predicted_value, color='red', linestyle='--', linewidth=1)
            ax.plot(user_input, predicted_value, 'ro')  # predicted point

            ax.set_xlabel("Advertising Spend (in lakhs)", fontsize=10)
            ax.set_ylabel("Sales Revenue (in lakhs)", fontsize=10)
            ax.set_title("Prediction on Regression Line", fontsize=10)
            ax.tick_params(axis='both', labelsize=8)
            ax.legend()
            st.pyplot(fig)
            
    with st.expander("➡️ Key Takeaways ..."):
        st.markdown("""
        - 🔍 **Simplicity with Impact**: Even a simple linear model offers valuable foresight—linking investments (like ad spend) directly to outcomes (like sales revenue).
        - 📊 **Data-Driven Decisions**: Enables leadership to make **objective** decisions, backed by quantitative evidence rather than gut feel.
        - 🎯 **Budget Optimization**: Helps identify how much to invest to hit revenue targets—minimizing under or over-spending on campaigns.
        - 📈 **Trend Insights**: Understanding whether returns from increased spending are **linear**, diminishing, or plateauing over time.
        - 🧪 **Foundation for More Advanced Models**: This simple regression builds the base for multivariable models involving seasonality, regions, or digital channels.
        """)


def load_ca_data():
    data = fetch_california_housing(as_frame=True)
    X = data.frame.drop(['MedHouseVal'], axis=1)
    y = data.frame['MedHouseVal']
    return data.frame, X,y

def vif_check(df):
    X = df.drop(columns=['MedHouseVal'])
    vif_data = pd.DataFrame()
    vif_data["feature"] = X.columns
    vif_data["VIF"] = [variance_inflation_factor(X.values, i) for i in range(X.shape[1])]
    return vif_data, X, df['MedHouseVal']

def build_model(X_train, y_train):
    model = LinearRegression()
    model.fit(X_train, y_train)
    return model

def main():
    st.markdown(f"**🧠 Regression Intuitions - Linear Regression Demo**")
    
    show_intro()
    
    simple_regression_example()
    
    with st.expander("➡️ Load & View California Housing Dataset"):
        df, X, y = load_ca_data()
        st.dataframe(df.head())
        
        st.markdown("""
            The **California Housing Dataset** is based on data from the 1990 U.S. Census.  
            It contains information collected from block groups across California and is often used for regression tasks to predict housing values.

            ###### 📌 Columns Description:

            - **MedInc** *(Median Income)*: Median income of households in the block group (in tens of thousands of dollars).
            - **HouseAge** *(Median House Age)*: Median age of houses in the area.
            - **AveRooms** *(Average Rooms)*: Average number of rooms per household.
            - **AveBedrms** *(Average Bedrooms)*: Average number of bedrooms per household.
            - **Population**: Total population of the block group.
            - **AveOccup** *(Average Occupancy)*: Average number of people per household.
            - **Latitude**: Geographical latitude of the block group.
            - **Longitude**: Geographical longitude of the block group.

            ###### 🎯 Target Column:
            - **MedHouseVal** *(Median House Value)*: This is the target variable to be predicted.  
            It represents the **median house value** in the block group (in hundreds of thousands of dollars).
            """)
        
        st.markdown("###### 🗺️ California Housing: Prices by Location")

        fig, ax = plt.subplots(figsize=(12, 5))
        scatter = ax.scatter(
            df["Longitude"], 
            df["Latitude"], 
            c=df["MedHouseVal"], 
            cmap="viridis", 
            s=10, 
            alpha=0.5
        )

        ax.set_title("Median House Value across California", fontsize=14)
        ax.set_xlabel("Longitude")
        ax.set_ylabel("Latitude")
        ax.grid(True)

        # Add color bar to represent house value
        cbar = plt.colorbar(scatter, ax=ax)
        cbar.set_label("Median House Value ($100,000s)")
        
        # Annotate major cities
        ax.annotate("Los Angeles", xy=(-118.25, 34.05), xytext=(-121, 33.8),
                    arrowprops=dict(facecolor='red', arrowstyle="->"), fontsize=10, color='red')
        ax.annotate("San Francisco", xy=(-122.42, 37.77), xytext=(-125, 38.5),
                    arrowprops=dict(facecolor='blue', arrowstyle="->"), fontsize=10, color='blue')

        # Shade ocean region (rough approximation: west of longitude -123)
        ax.axvspan(-125, -123, color='lightblue', alpha=0.3, label="Pacific Ocean")

        # Add legend
        ax.legend(loc="lower right")

        st.pyplot(fig)
        
        st.write(f"""
                 - Color represents housing value: darker → cheaper, lighter → more expensive.
                 - notice high-value clusters around coastal regions (e.g., around the Bay Area and Los Angeles).
                  """
                  )
    
    with st.expander("➡️ Key Challenges of California Housing Dataset (Regression vs Rule-Based Models)"):
        st.markdown("""
        Understanding the limitations of both data and modeling approaches is vital for leaders making data-driven decisions. Below are the key challenges when using this dataset for **regression modeling**, especially compared to traditional **rule-based systems**:

        ###### 🔍 Data Challenges (Specific to Regression):
        - **Non-linear Relationships**: Housing prices may not increase proportionally with income, age, or other features, making simple linear models insufficient.
        - **Geographic Bias**: Locations like LA and SF have unique dynamics not captured by standard features—housing is expensive due to factors beyond income or age.
        - **Data Outliers**: Some neighborhoods may have unusually high or low prices, skewing the model's predictions.
        - **Capped Target Values**: The `MedHouseVal` was capped at $500,000 in the dataset, which can limit the model's ability to predict higher-end housing.

        ###### 🤖 Compared to Rule-Based Models:
        - **Rule-based systems lack adaptability**: Rules like "if income > X, price > Y" cannot account for regional nuances, housing density, or socio-economic patterns.
        - **Hard to scale**: Adding new rules for every edge case becomes complex and unmanageable over time.
        - **Not data-driven**: Rule-based logic does not improve from historical data or learn from new patterns.

        ###### 🧭 Key Takeaway:
        > Regression models offer adaptability and learning from patterns across vast geographies and populations. However, they require clean, unbiased data and continuous validation—unlike rule-based systems, which are simple but brittle and not future-proof.
        """)
    
    # with st.expander("➡️Linearity Check & VIF"):
    #     vif_data, X, y = vif_check(df)
    #     st.dataframe(vif_data)
    
    with st.expander("➡️ Prepare Data for the regression model"):
        
        st.markdown("""
        Creating training and test datasets is a fundamental step in building machine learning models. It ensures the model learns patterns **only from part of the data**, and is then **evaluated on unseen data** to measure its performance.

        ###### 🔧 Why Prepare Data?
        - **Ensures Model Quality**: Models need structured and clean data to learn effectively.
        - **Prevents Overfitting**: By separating training from testing, we prevent the model from simply memorizing the data.
        - **Enables Generalization**: A well-prepared dataset ensures the model can make accurate predictions on new, real-world data.

        ###### 📦 Train-Test Split
        - **Training Set**: Used by the model to learn patterns and relationships between input (features) and output (target).
        - **Test Set**: Held back during training and used solely to evaluate model performance. It simulates how the model would perform in production.

        ###### ✅ Best Practices
        - **Use an 80/20 or 70/30 split** depending on dataset size.
        - **Stratify** if your target variable is imbalanced (more applicable in classification).
        - **Set a random seed** (e.g., `random_state=42`) for reproducibility.
        - **Clean and preprocess** before splitting to avoid data leakage.
        - **Avoid using test data during model training or tuning**—this ensures an unbiased evaluation.

        > 🔍 **Key point**: Proper data preparation is like setting the foundation of a building—without it, even the most advanced models can crumble in production.
        """)
            
        X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
        
        # Display the number of samples in training and testing sets
        st.write(f"Number of samples in training set: {X_train.shape[0]}")
        st.write(f"Number of samples in testing set: {X_test.shape[0]}")
        st.write("Train and test sets created.")
    
    with st.expander("➡️ Build Linear Regression Model"):
        #model = build_model(X_train, y_train)
        
        st.write("Training the Linear Regression model...")

        # Simulate training with progress bar
        progress_bar = st.progress(0)
        for i in range(100):
            time.sleep(0.01)
            progress_bar.progress(i + 1)

        # Train the model
        model = LinearRegression()
        model.fit(X_train, y_train)
        st.success("Model trained successfully.")

        # Predict and compute metrics
        y_pred = model.predict(X_test)
        mae = mean_absolute_error(y_test, y_pred)
        mse = mean_squared_error(y_test, y_pred)
        rmse = np.sqrt(mse)
        r2 = r2_score(y_test, y_pred)

        st.markdown("###### 📊 Model Evaluation on Test Set")
        st.write(f"**MAE**: {mae:.2f}")
        st.write(f"**MSE**: {mse:.2f}")
        st.write(f"**RMSE**: {rmse:.2f}")
        st.write(f"**R² Score**: {r2:.2f}")

        # Cross-validation to detect overfitting
        st.markdown("###### 🔁 Cross-Validation Performance")
        cv_results = cross_validate(model, X, y, cv=10, return_train_score=True, scoring='r2')
        train_r2 = cv_results['train_score']
        test_r2  = cv_results['test_score']

        r2_df = pd.DataFrame({
            'Fold': list(range(1, 11)),
            'Training R²': train_r2,
            'Test R²': test_r2
        })

        fig, ax = plt.subplots(figsize=(9,5))
        ax.plot(r2_df['Fold'], r2_df['Training R²'], marker='o', label='Training R²', color='blue')
        ax.plot(r2_df['Fold'], r2_df['Test R²'], marker='o', label='Test R²', color='green')
        ax.set_title("Cross-Validation R² Scores")
        ax.set_xlabel("Fold")
        ax.set_ylabel("R² Score")
        ax.legend()
        ax.grid(True)
        st.pyplot(fig)

        st.dataframe(r2_df.style.format({'Training R²': '{:.2f}', 'Test R²': '{:.2f}'}))
        
        st.write("""
            - ✅ **Consistent Training Performance**:  
            The training R² scores range from **0.59 to 0.63**, indicating a fairly **consistent learning pattern** across all 10 folds.  
            This means the model generalizes reasonably well on the training data.

            - ⚠️ **Test Set Variability**:  
            The test R² scores range from **0.42 to 0.61**, showing **slightly higher variance** across folds.  
            Some folds show strong performance (e.g., Fold 2), while others drop noticeably (e.g., Fold 3).

            - 🔁 **No Severe Overfitting Detected**:  
            If the training R² was very high (e.g., 0.9) and test R² was low (e.g., 0.3), that would indicate **overfitting**.  
            In this case, **training and test R² are fairly close**, suggesting the model is **not overfitting significantly**.

            - 📉 **Room for Improvement**:  
            An average test R² around **0.52** implies that the model explains **just over 50% of the variance** in house values.  
            For business-critical applications like real estate pricing or policy decisions, we may consider:
                - **Feature engineering** (e.g., regional segmentation),
                - **Model tuning**, or
                - **Trying more expressive models** like decision trees or gradient boosting.

            """)
    
    # learning curve
    with st.expander("➡️ Was Training Data Sufficient? (Learning Curve Analysis)"):
        st.markdown("###### 📊 Learning Curve Analysis")

        # Generate learning curves
        train_sizes, train_scores, test_scores = learning_curve(
            model, X, y, cv=5, scoring='r2', train_sizes=np.linspace(0.1, 1.0, 10), shuffle=True, random_state=42
        )

        # Calculate mean and std deviation
        train_scores_mean = np.mean(train_scores, axis=1)
        test_scores_mean = np.mean(test_scores, axis=1)

        # Plotting
        fig, ax = plt.subplots(figsize=(9,4))
        ax.plot(train_sizes, train_scores_mean, 'o-', color="blue", label="Training R²")
        ax.plot(train_sizes, test_scores_mean, 'o-', color="green", label="Validation R²")
        ax.set_title("Learning Curve: Linear Regression")
        ax.set_xlabel("Number of Training Samples")
        ax.set_ylabel("R² Score")
        ax.legend(loc="best")
        ax.grid(True)
        st.pyplot(fig)

        # Interpret results
        st.write("""
        - ✅ **Training R² is high initially** (indicating the model learns patterns even with fewer samples).
        - 📉 **Validation R² improves as training size increases**, then plateaus.
        - 🧠 This suggests the model **benefits from more training data**, but after a certain point, **additional data does not significantly improve generalization**.
        - 🔍 The **gap between training and validation curves** is relatively small, indicating **no severe overfitting**.
        - 📌 **Conclusion**: The current dataset size seems **adequate**, and the model is learning well with the data provided.
        """)    
        
    with st.expander("📊 Understand Feature Impact: Coefficients of the Linear Regression Model"):
        importance = model.coef_
        features = X.columns

        fig, ax = plt.subplots(figsize=(9,5))
        ax.barh(features, importance, color='skyblue')
        ax.set_title("Feature Importance (Linear Regression Coefficients)")
        ax.set_xlabel("Coefficient Value")
        st.pyplot(fig)

        st.markdown("""
        ###### 🔍 Interpretation:
        - Features with larger **absolute values** have a stronger effect on the predicted house value.
        - A **positive coefficient** increases the predicted value.
        - A **negative coefficient** decreases the predicted value.

        ###### 🧠 What it means for decision-makers:
        - **Median Income** is a strong positive driver — wealthier areas tend to have higher housing values.
        - **Latitude** has a negative coefficient — northern areas may have lower house prices.
        - Helps focus strategic decisions on what really influences prices across California.
        """)
    
    with st.expander("🧠 Why Linear Regression Still Matters: Foundation for Deep Learning & Transformers"):
        st.markdown("""
        Linear Regression may look simple, but it's far from trivial — it’s the **first building block** in the ladder to advanced AI models like **Deep Learning** and **Transformers**.

        ###### 📚 Conceptual Foundations:
        - **Weights & Bias**: The core of linear regression is about learning weights and biases — which is exactly what **every neural network layer** does, just at scale.
        - **Loss Minimization**: Linear regression minimizes **Mean Squared Error** — a principle used in training neural networks to adjust weights through **backpropagation**.
        - **Linear Combinations**: Deep learning models, at their core, are just multiple layers of **linear transformations + non-linear activations**.

        ###### 🤖 Connect to Transformers:
        - Transformer architectures (like GPT, BERT) use **linear projections** in attention mechanisms.
        - Every layer in these models performs matrix multiplications — which is, again, just advanced **linear algebra and regression-like operations**.

        ###### 🏗️ Strategic Insight:
        - A solid grasp of linear regression builds the intuition needed to understand more complex systems.
        - Senior leaders can better evaluate ML and AI project feasibility and interpret outcomes by understanding these **fundamentals that scale**.

        🔄 *"From Linear Regression to Transformers, it's all about modeling relationships and optimizing parameters — just with different levels of complexity and abstraction."*
        """)


if __name__ == "__main__":
    main()