How to improve my regression models results more accurate in random forest regression

Question

Issue: Getting r2 near to 0.64. Want to improve my results more. Don't know what's the issue of these results. Have done Removing outliers, Converting String -> Numerical, normalization. Wanna know is there any issue with my output? Please ask me anything if I didn't ask the question correctly. It's just my starting on Stack overflow.

y.value_counts()

This is histogram of my outputs. I am not professional in Regression need super help from your side.

Removing Collinearity in my inputs


import seaborn as sns
# data=z_scores(df)
data=df
correlation=data.corr()

k=22
cols=correlation.nlargest(k,'Please enter your Subjects GPA which you have studied? (CS) [Introduction to ICT]')['Please enter your Subjects GPA which you have studied? (CS) [Introduction to ICT]'].index
cm=np.corrcoef(data[cols].values.T)
f,ax=plt.subplots(figsize=(15,15))
sns.heatmap(cm,vmax=.8,linewidths=0.01,square=True,annot=True,cmap='viridis',
            linecolor="white",xticklabels=cols.values,annot_kws={'size':12},yticklabels=cols.values)

cols=pd.DataFrame(cols)
cols=cols.set_axis(["Selected Features"], axis=1)
cols=cols[cols['Selected Features'] != 'Please enter your Subjects GPA which you have studied? (CS) [Introduction to ICT]']
cols=cols[cols['Selected Features'] != 'Your Fsc/Ics marks percentage?']
X=df[cols['Selected Features'].tolist()]
X

Then applied Random Forest Regressor and got these results

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=42)

from sklearn.ensemble import RandomForestRegressor
regressor = RandomForestRegressor(n_estimators = 10, random_state = 0)
model=regressor.fit(X_train, y_train)
y_pred = model.predict(X_test)

print("MAE Score: ", mean_absolute_error(y_test, y_pred))
print("MSE Score: ", mean_squared_error(y_test, y_pred))
print("RMSE Score: ", math.sqrt(mean_squared_error(y_test, y_pred)))
print("R2 score : %.2f" %r2_score(y_test,y_pred))

Got these Results.

MAE Score:  0.252967032967033
MSE Score:  0.13469450549450546
RMSE Score:  0.36700750059706605
R2 score : 0.64

nitin kalra · Accepted Answer · 2022-05-24 08:28:34Z

in order to get better results you need to do hyper-parameter tuning try to focus on these

n_estimators = number of trees in the forest
max_features = max number of features considered for splitting a node
max_depth = max number of levels in each decision tree
min_samples_split = min number of data points placed in a node before the node is split
min_samples_leaf = min number of data points allowed in a leaf node
bootstrap = method for sampling data points (with or without replacement)

Parameters currently in use(random forest regressor )
{'bootstrap': True,
'criterion': 'mse',
'max_depth': None,
'max_features': 'auto',
'max_leaf_nodes': None,
'min_impurity_decrease': 0.0,
'min_impurity_split': None,
'min_samples_leaf': 1,
'min_samples_split': 2,
'min_weight_fraction_leaf': 0.0,
'n_estimators': 10,
'n_jobs': 1,
'oob_score': False,
'random_state': 42,
'verbose': 0,
'warm_start': False}

k fold cross validation
use grid search cv

Collectives™ on Stack Overflow

How to improve my regression models results more accurate in random forest regression

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related