Feature Selection With Practical Approach



Original Source Here

Remove Constant Features Using Variance Threshold

The very first features that we should remove from our dataset are constant ones. It might seem easy to perform manually but suppose you have 200–300 features. In that case, using some technique make sense.

Firstly, let import all the required libraries. Pandas is used to create and manipulate the dataset. VarianceThreshold is used to remove the features that have low variance. Variance is just a measure of variability. 0 variance implies all values are same/constant. Train_test_split is used for splitting the data for training and testing purposes.

import pandas as pd
from sklearn.feature_selection import VarianceThreshold
from sklearn.model_selection import train_test_split

Here, we have just created a dummy dataframe to understand this feature selection easily. We can see that columns C and D have constant values hence they should be removed.

df = pd.DataFrame({"A":[2,3,5,2,6],
                  "B":[4,6,4,2,8],
                  "C":[5,5,5,5,5],
                  "D":[0,0,0,0,0]})
df

Now, we have initialized the object var of VarianceThreshold with the threshold as 0(constant). We can vary this according to our needs. Then, we have printed the array of boolean in which true and false represent non-constant and constant features respectively.

var = VarianceThreshold(threshold=0.0)
var.fit(df)
var.get_support()

We have used list comprehension in which we are looping through all the columns and inserting the constant features into the list named constant_features . Finally, we have dropped the constant features from the dataset.

constant_features = [i for i in df.columns
                    if i not in df.columns[var.get_support()]]
print(constant_features)
df.drop(constant_features,axis=1)

The previous example may seem too easy and can be easily detected. But in real-world scenarios, there will be 100’s or 1000’s of features. This technique will help in those cases. Let us see the same using the bigger and real dataset.

Here, we have loaded our dataset that is available on GitHub. The dataset consists of 371 features.

df = pd.read_csv("train.csv")
print(df.shape)

Here, X represents input variables and Y represents the target variable. We should always apply any Feature Selection technique to the training set to avoid any kind of overfitting. That’s why we are splitting the dataset into training and testing. After that, the process is the same as explained above. Here, 371–332 = 39 features are constant. So, we drop these features from both training and testing sets.

X = df.drop(labels=['TARGET'],axis=1)
Y = df['TARGET']
x_train,x_test,y_train,y_test = train_test_split(X,Y,test_size = 0.2)
var = VarianceThreshold(threshold=0.0)
var.fit(x_train)
sum(var.get_support())constant_features = [i for i in x_train
                    if i not in x_train.columns[var.get_support()]]
x_train.drop(constant_features,axis=1)

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot



via WordPress https://ramseyelbasheer.wordpress.com/2021/02/12/feature-selection-with-practical-approach/

Popular posts from this blog

I’m Sorry! Evernote Has A New ‘Home’ Now

Image
Original Source Here My Apologies! I was critical about the grandfather of all apps — Evernote a few months back when I wrote Why is Evernote losing its Ground? on Medium. With that said, I also wrote another story on Evernote in November of last year — Has Evernote Made A Comeback After Years Of Stagnation? This was the time when Ian Small and his team were making inroads into the technology and strategically making enhancements and revamping the app across all platforms. Evernote Is At A War-front And Wants To Prove A Point. I must h ave written this in my earlier blogs but just to re-iterate — I have been and am an avid user of Evernote since 2009 and is a part of my life for planning my daily life, projects, content writing, and almost everything. Recently, Evernote has launched its HOME or you may call it a DASHBOARD that provides its users a one-stop place to get access to their important and critical notes and also provides a few AI-enabled capabilities that I wil
Read more

Jensen Huang: Racism is one flywheel we must stop

Image
https://venturebeat.com/wp-content/uploads/2021/07/jensen-huang.jpg?w=1200&strip=all Original Source Here All the sessions from Transform 2021 are available on-demand now. Watch now. Jensen Huang , CEO of Nvidia , received the distinguished lifetime achievement award by the Asian American Engineer of the Year from the Chinese Institute of Engineers (CIE) group. And in his acceptance speech, he made a rare comment beyond Nvidia’s business matters: “Racism is one flywheel we must stop.” The comments came at the end of an acceptance speech where he thanked the group for its award, an annual gathering that recognizes outstanding Asian American scientists, engineers and role models. In a virtual ceremony, Huang was awarded for his contributions as “a visionary and innovator in parallel computing technology that accelerates the realization of AI computing.” He also spoke of his experience as an immigrant and an Asian American. “It is strange to accept a lifetime achievement
Read more

Fully Explained DBScan Clustering Algorithm with Python

Image
Original Source Here Fully Explained DBScan Clustering Algorithm with Python Unsupervised learning in machine learning on density-based clusters Clustering In unsupervised Learning. A photo by Author In this article, we will discuss the machine learning clustering-based algorithm that is the DBScan cluster. The approach in this cluster algorithm is density-based than another distance-based approach. The other cluster which is distance-based looks for closeness in data points but also misclassifies if the point belongs to another class. So, density-based clustering is suited in this kind of scenario. The cluster algorithms come in unsupervised learning in which we don’t rely on target variables to make clusters. In a cluster, the main concern is on maximum dense connected points. The density-based clustering can be useful in arbitrary shapes even with noise. The working function of the density-based cluster is shown below: We select a random data point. Then it will
Read more