Merge branch 'milestone-3' of https://github.com/aye-thuzar/CS634Project into milestone-3

README.md

@@ -1,10 +1,16 @@
 # CS634Project
 
-Milestone-3 notebook: https://colab.research.google.com/drive/17-7A0RkGcwqcJw0IcSvkniDmhbn5SuXe
+Milestone-3 notebook: [[https://colab.research.google.com/drive/17-7A0RkGcwqcJw0IcSvkniDmhbn5SuXe]](https://github.com/aye-thuzar/CS634Project/blob/milestone-3/CS634Project_Milestone3_AyeThuzar.ipynb)(https://colab.research.google.com/drive/1BeoZ4Dxhgd6OcUwPhk6rKCeFnDFMUCmt#scrollTo=TZ4Ci-YXOSl6)
 
 Hugging Face App: 
 
-Results:
+App Demonstration Video: 
+
+***********
+
+Results
+
+***********
 
 XGBoost Model's RMSE: 28986  (Milestone-2)
 
@@ -14,15 +20,15 @@ Optuna optimized LGBM's RMSE: 13799.282803291926
 
 ***********
 
-Totalnumber of trials:  120
+Hyperparameter Tuning with Optuna
 
-Best RMSE score on validation data: 12338.665498601415
+************
 
-------------------------------
+Total number of trials:  120
 
-Best params:
+Best RMSE score on validation data: 12338.665498601415
 
-------------------------------
+**Best params:**
 
 boosting_type :	 goss
 
@@ -44,6 +50,62 @@ min_child_samples :	 1
 
 ***********
 
-Reference:
+## Documentation for Milestone 4
+
+***********
+
+Dataset: https://www.kaggle.com/competitions/house-prices-advanced-regression-techniques/overview
+
+**Data Processing and Feature Selection:**
+
+For the feature selection, I started by dropping columns with a low correlation (< 0.4) with SalePrice. I then dropped columns with low variances (< 1). After that, I checked the correlation matrix between columns to drop selected columns that have a correlation greater than 0.5 but with consideration for domain knowledge. After that, I checked for NAs in the numerical columns. Then, based on the result, I used domain knowledge to fill the NAs with appropriate values. In this case, I used 0 to fill the NAs as it was the most relevant value. As for the categorical NAs, they were replaced with ‘None’. Once, all the NAs were taken care of, I used LabelEncoder to encode the categorical values. I, then, checked for a correlation between columns and dropped them based on domain knowledge.
+
+Here are the 10 features I selected:
+
+ 'OverallQual': Overall material and finish quality
+ 
+ 'YearBuilt': Original construction date
+ 
+ 'TotalBsmtSF': Total square feet of basement area
+ 
+ 'GrLivArea': Above grade (ground) living area square feet
+ 
+ 'MasVnrArea': Masonry veneer area in square feet
+ 
+ 'BsmtFinType1': Quality of basement finished area
+ 
+ 'Neighborhood': Physical locations within Ames city limits
+ 
+ 'GarageType': Garage location
+ 
+ 'SaleCondition': Condition of sale
+ 
+ 'BsmtExposure': Walkout or garden-level basement walls
+
+All the attributes are encoded and normalized before splitting into train and test with 80% train and 20% test.
+
+**Milestone 2:**
+
+For milestone 2, I ran an XGBoost Model with objective="reg:squarederror" and max_depth=3. The RMSE score is 28986.
+
+**Milestone 3:**
+
+
+
+**References:**
+
+https://towardsdatascience.com/analysing-interactions-with-shap-8c4a2bc11c2a
+
+https://towardsdatascience.com/introduction-to-shap-with-python-d27edc23c454
+
+https://www.aidancooper.co.uk/a-non-technical-guide-to-interpreting-shap-analyses/
+
+https://www.kaggle.com/code/rnepal2/lightgbm-optuna-housing-prices-regression/notebook
+
+https://www.kaggle.com/code/rnepal2/lightgbm-optuna-housing-prices-regression/notebook
+
+https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/
+
+https://towardsdatascience.com/why-is-everyone-at-kaggle-obsessed-with-optuna-for-hyperparameter-tuning-7608fdca337c
 
 https://github.com/adhok/streamlit_ames_housing_price_prediction_app/tree/main