Tuning the Hyperparameter thus automating Deep Learning model



Original Source Here

Tuning the Hyperparameter thus automating Deep Learning model

ML model to train ML model… sounds cool..?

Introduction

This is the era where every Computer Science Student is looking forward to be a Data Scientist. Their job heavily involves using the data to make predictions. But there is a lot of manual things which needs to be done like hyperparamater tuning. So why not just automate it?

This blog contains automation of hyperparameter tuning using python and devops idea.

What are Hyperparameters ?

Hyperparameters are the variables which determines the network structure (Eg: Number of Hidden Units) and the variables which determine how the network is trained (Eg: Kernel Size). Hyperparameters are set before training (before optimizing the weights and bias).

Now Since we are aware with the objective of the main objective of the blog, let us continue.

Here the blog is divided into 5 separate jobs that will do the work for us, so let’s get started.

Job 1: Get the data from the GitHub Repository.

Whenever the developer pushes any code to GitHub repository, with the help of GitHub plugin installed on Jenkins we will be able to monitor the changes in the repository and thus if any change is found we copy the code from remote to our local directory. The technique used is Poll SCM that keeps on monitoring the remote repository at every minute.

Job 2: Check the type of model and launch the appropriate container.

Using a python script, we will we check what type of model is to be trained and accordingly we will launch a container that has the requirements satisfied. Once the container is launched the model will be trained and we predict its accuracy. The accuracy predicted by the model will be saved in CSV file named as realrecords.csv with all the hyperparamaters provided initially by a input.txt file.

Job 3: Check the accuracy — predict new input parameters.

A python script named CheckAccuracy.py checks on the accuracy of the last trained model’s accuracy, if desired is attained then the model runs a python script name SuccessEmail.py and sends the Success Mail to the client. And if the desired accuracy is not attained then a container will be launched with Predict_Model_Test.py. On running this container, a pretrained model that I had created earlier on the dataset (generated by me) will be loaded and will predict the new hyperparameters with accuracy and these values will be stored a CSV file named labtest.csv. And when a accuracy comes greater than that of the accuracy given be the Deep Learning model then we give these parameters as an input to that Original model.

Job 4: Change the input parameters.

A python script reads the last row from the labtest.csv file i.e. the new hyperparameters and thus changes the content of input.txt file.

Job 5: Push the input file.

When all the above jobs are built successfully the input.txt file is pushed to remote repository.

As soon as the change in some file is detected at the GitHub repo by Jenkins it again copies the files to local repo and all the process from job 1 repeats in a loop till the desired accuracy is achieved. The loop will break when no change is observed in input.txt file.

So, basically an ML model is being used to train another ML model.

Thank you guys for reading this post.

GitHub Link for the above codes: https://github.com/a19singh/MlopsTrain

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot



via WordPress https://ramseyelbasheer.io/2021/05/31/tuning-the-hyperparameter-thus-automating-deep-learning-model/

Popular posts from this blog

I’m Sorry! Evernote Has A New ‘Home’ Now

Image
Original Source Here My Apologies! I was critical about the grandfather of all apps — Evernote a few months back when I wrote Why is Evernote losing its Ground? on Medium. With that said, I also wrote another story on Evernote in November of last year — Has Evernote Made A Comeback After Years Of Stagnation? This was the time when Ian Small and his team were making inroads into the technology and strategically making enhancements and revamping the app across all platforms. Evernote Is At A War-front And Wants To Prove A Point. I must h ave written this in my earlier blogs but just to re-iterate — I have been and am an avid user of Evernote since 2009 and is a part of my life for planning my daily life, projects, content writing, and almost everything. Recently, Evernote has launched its HOME or you may call it a DASHBOARD that provides its users a one-stop place to get access to their important and critical notes and also provides a few AI-enabled capabilities that I wil
Read more

Jensen Huang: Racism is one flywheel we must stop

Image
https://venturebeat.com/wp-content/uploads/2021/07/jensen-huang.jpg?w=1200&strip=all Original Source Here All the sessions from Transform 2021 are available on-demand now. Watch now. Jensen Huang , CEO of Nvidia , received the distinguished lifetime achievement award by the Asian American Engineer of the Year from the Chinese Institute of Engineers (CIE) group. And in his acceptance speech, he made a rare comment beyond Nvidia’s business matters: “Racism is one flywheel we must stop.” The comments came at the end of an acceptance speech where he thanked the group for its award, an annual gathering that recognizes outstanding Asian American scientists, engineers and role models. In a virtual ceremony, Huang was awarded for his contributions as “a visionary and innovator in parallel computing technology that accelerates the realization of AI computing.” He also spoke of his experience as an immigrant and an Asian American. “It is strange to accept a lifetime achievement
Read more

5 Best Machine Learning Books for ML Beginners

Image
Original Source Here Reading books is important because they books you endless knowledge. If you plan to become a Data Scientist or Machine Learning Engineer, reading machine learning books will help you gain the knowledge and skills required in this field. In a simple definition, machine learning is known as the subset of AI that has the ability to automatically learn from the data without explicitly being programmed or assisted by domain expertise. Email spam and malware filtering, product recommendations, image recognition, automatic language translation — these are all examples of real-life machine learning applications. The Futu r e of Jobs Report 2020 predicts that artificial intelligence (of which machine learning is a subset) will create 12 million new jobs across 26 countries by 2025. There are many books available in the market to learn machine learning. In this article, I have compiled the 5 best machine learning books that will help you start building your care
Read more