Increase your machine learning success with AWS ML services and AWS Machine Learning Embark

Original Source Here

This is a guest post from Mikael Graindorge, Sales Operations Leader at Thermo Fisher Scientific.

In the life sciences industry, data is growing in abundance and is getting increasingly complex, which makes it challenging to use traditional analytics methodologies. At Thermo Fisher Scientific, our mission is to make the world healthier, cleaner, and safer, and to realize this vision, we need to make optimal decisions by extracting insights from the large volume and variety of data available to us. To do this effectively, we need to empower team members with machine learning (ML) skills so we can achieve our vision as an organization.

ML, as a capability, has the transformative power to enable people from all backgrounds to effectively use data in their decision-making processes. However, using ML isn’t the same thing as using ML effectively. To share an analogy, having access to a kitchen doesn’t necessarily mean you can cook well. There is a big difference between making food and cooking a tasty meal. It takes training and experience to concoct something that most people would agree tastes amazing. And ML isn’t that different. With training and experience, we can build the necessary skills to apply ML to various businesses and operational needs within an organization.

In this post, we start by looking at the prerequisites necessary for ML novices to undertake the journey, and the steps necessary to build confidence and proficiency. To start the journey, you don’t need experience in ML nor have advanced statistical background. If you have an open mind, and an appetite and willingness to learn new ways to process information, you’re ready to start.

To begin the journey at Thermo Fisher Scientific, we got help from the AWS Machine Learning Embark program, which provides a structured pathway to learn ML. The program includes a discovery workshop, business leader training, technical training, and a hands-on proof of concept solution that our team developed alongside ML experts from AWS. AWS ML Embark provides the training necessary to build your foundational knowledge, establish processes for success, and launch your very first ML solution.

What does it take to use machine learning?

You should know two important things before getting started with building ML solutions. First, developing an ML solution isn’t just limited to software engineers and scientists with years of experience in the field. Second, the ML solution development process consists of several steps, requiring people from many backgrounds to jump in and contribute at various points along the way. Depending on your individual background, you may need to level-set on certain skills more than others.

Many professions are involved with data processing, from business analysts to data engineers or even data scientists. Building comprehensive AI/ML solutions often requires a multi-functional team that consists of business leaders, scientists, and engineers. Typical roles required to support the full lifecycle of AI/ML solutions are shown in the following diagram. However, these roles all share something in common: data. As long as you build and use your technical skills around your subject matter expertise, possess in-depth knowledge of your input data, and have a good understanding of the required business output, developing ML solutions simply becomes a process, with steps to follow, just like a recipe.

Craftsmanship over passion

AWS offers a variety of tools and recommendations necessary to become a successful ML professional. However, the ones that truly succeed are those who come with a craftsman’s mindset, so you can develop and continue to refine a valuable skillset. In other words, although it’s easy to start and get short-term wins, success comes to those who invest time and effort to truly hone their skills.

As an ML professional, you can accomplish many tasks by using the datasets available to you within your organization. Solutions that you develop may empower the organization to save money, drive productivity gains, improve customer experience, or simply expand the boundaries of your knowledge. You need to determine which of these reasons will be your drivers to succeed with AI/ML.

Increase your chance of success with the AWS ML Embark program

To hit the ground running, you need to know where to start and get a perspective about the journey ahead. Teams struggle to launch ML initiatives and get meaningful adoption because they don’t have well-understood and proven patterns to follow. Challenges include identifying the most impactful projects to tackle, and developing the right skills to solve these problems. Without a blueprint for success and data science know-how, projects can stall. Even if you somehow deliver these projects into production, they often fail to catalyze the change you expected because they don’t necessarily align to the right business goals.

The AWS ML Embark program is designed to help teams (and organizations) overcome these common challenges and start on the journey to ML success. By working backwards through an interactive workshop, business and technical leaders in the organization come together to jointly identify business opportunities and specific use cases in which ML can have meaningful impact. Then, led by expert instructors from AWS, the technical training sessions ramp up attendee’s ML skills through introducing practical applications, and business enablement sessions, designed specifically for business leaders, dive into strategic topics on how to successfully lead ML initiatives and build an AI-powered organization. Lastly, to add some fun to the practical learning, the program also includes an optional corporate AWS DeepRacer event to excite the broader technical staff to embrace ML through hands-on training and racing fully autonomous 1/18th scale race cars.

The AWS ML Embark program’s technical training, developed by Amazon’s own Machine Learning University, covers major topics in ML that you need to get started. Over the course of the training, you gain hands-on experience and learn from the basic to complex neural networks. Through an interactive classroom setting, you have the unique opportunity to learn (or relearn) different ML approaches with a multitude of applicable use cases. The AWS ML Embark program offers a safe environment to practice, fail, and gain experience that will pave your ML journey. The training helps individuals and teams to build a culture of learning and collaboration.

Adopt a framework for success

Looking back, it’s clear to me that it would have taken us a lot longer to get started without the perspectives brought by instructors of the AWS ML Embark program. The program offered helpful information and inspired our new ML professionals to take on AI/ML projects, advance their careers, and discover new horizons. But don’t just take it from me; here is a quote from one of the engineers who took the training:

“I really liked the ML Embark training and I benefited greatly from it. I was able to directly apply the methodology and even the exact ML Python code from the training to my forecasting project at work. For example, I used the code for KNN, linear regression, logistic regression, and decision tree from the training class. By doing so, I have a deeper understanding about ML. This training has saved me hours and even days of time by demonstrating the most cutting-edge tools and options for ML. The trainers are very proficient and patient to help us and they are top notch in their fields. I deeply appreciate that our organization gave us the opportunity to participate in the training, thanks for organizing the event!!!”

As a manager and mentor, I took an additional step to combine AWS ML Embark learnings with my adaptation of the CRoss Industry Standard Process for Data Mining (CRISP-DM) framework. This approach helped us break down ML projects into the following manageable steps and speed up project delivery:

Understanding the “why” – Gathering the motivation behind the initiative and the outcomes and benefits expected to be achieved at the end of the project
Data preparation – Identifying and collecting data, including any data cleansing, enrichment, and preparation necessary to efficiently and accurately train and validate ML models
Modeling – Training, testing, and tuning ML models
Measuring success – Evaluating model outputs against original business goals, and depending on the results, productionizing the solution or going back to refine the approach and try again
Documentation and production – Developing the process necessary to summarize the project to share with technical leadership and non-technical business stakeholders, including technical documentation and storytelling for the outputs

Although the exact process depends on your specific requirements, this process should fit most scenarios. The key is having the right data and understanding it—this will likely be the main reason for why your own ML project succeeds or fails.

Flying on your own as an ML professional

After you complete the AWS ML Embark training, you can begin building your ML solution. With AWS, you can take advantage of the broadest and deepest set of AI/ML services, and the supporting cloud infrastructure:

Amazon Simple Storage Service (Amazon S3) provides scalable storage for your data lake and training data, running with AWS data analytics services such as AWS Glue and Amazon Athena.
Services such as AWS Lambda and AWS Step Functions provide tools to move data in and out of your ML workflow and orchestrate model training and deployment processes.
Depending on the use case, for many AI-powered applications such as forecasting and personalization, you can use the fully managed and easy-to-use AI services from AWS such as Amazon Forecast and Amazon Personalize.
For use cases that require custom ML model development, you can use Amazon SageMaker to build, train, and deploy ML models at scale. SageMaker removes the complexity of many steps in the ML workflow; for example, you can use SageMaker Data Wrangler for preprocessing, and SageMaker Pipelines for automation. The SageMaker ecosystem allows developers, engineers, and scientists to accelerate ML development and adoption.

The AWS ML Embark program introduces you to these services and gets you started with the AWS AI/ML landscape.

Before you get started with AWS services, the key to unlocking the potential of ML is to understand the business problem and available data, and then formulate the business problem into an appropriate ML problem. Each ML problem formulation is unique and requires an understanding of what the output of the ML model will be, and how it will be evaluated. AWS ML Embark gives you the training to translate your business problem into an ML problem, and assess the cost of errors from your ML model in order to set clear and quantifiable measures of success.

The last factor to consider when building your solution is how to determine whether your model achieves the right value for your business. ML model inputs, loss functions, and optimization parameters are covered during the AWS ML Embark training, but result interpretation and measure of success is unique to your specific use case. Therefore, my recommendation is to take the time at the beginning of the project to understand your input data, while setting clear and quantifiable measures of success. This will eliminate personal bias and opinions from project implementation, enable you to experiment quickly with different ML approaches, and find the right solution for your business.

Getting aspiring ML professionals started with AWS AI/ML

When you’re ready to get started, you can pick from several options. For beginners, we recommended two paths: AWS AI services such as Forecast and Amazon Personalize for ease of use, and SageMaker as a sandbox environment to learn how to develop custom ML models.

In our case, business intelligence specialists chose the Forecast-based forecasting solution, mainly for its simplicity in implementation. Although the technology was straightforward, the entire architecture had to be very robust because the solution must forecast weekly revenue performance for the upcoming quarters, across hundreds of thousands of customers and product categories, with a relatively high accuracy. Beyond the prediction itself, the process would save hundreds of hours for the sales representatives and their analysts, who could rely on a highly efficient and automated Forecast service.

The team’s expertise with the data and the understanding of implications of the predictions that the ML solution will generate allowed them to quickly focus on setting up the technical environment for the solution. The following diagram is a visual representation of their first project. The team spent a few weeks’ worth of effort and used multiple AWS services. Although the diagram contains multiple steps, it essentially achieves their goal by using five managed AWS services: Step Functions, Lambda, Amazon S3, Forecast, and Amazon Redshift.

The second project, using SageMaker, was assigned to an individual with foundational knowledge of ML and AWS services. Their prior experience enabled them to quickly combine AWS services and develop a custom ML solution. The project had two focus areas: data processing with Step Functions, and behavior predictions with SageMaker.

Step Functions is a serverless function orchestrator that makes it easy to sequence Lambda functions and create event-driven workflows defined by business logic. In our case, we used Step Functions to centralize, orchestrate, and enrich over 20 data sources, thereby eliminating the need for manual data preparation. We set up SageMaker ML batch inference to start making predictions after data is preprocessed with Step Functions and placed by Lambda functions into S3 buckets. Currently, this simple process generates hundreds of thousands of predictions used within our organization.

Although the details and the use case of this custom ML solution are proprietary to our business, we believe that this AWS architecture (as shown in the following diagram) is simple and can be easily adopted for different types of ML applications. Unlike the Forecast-based project, this approach centralized the level of effort around the data, where rigorous measures of success must be put in place in order to ensure the validity of the solutions.

Conclusion

AWS ML Embark and AWS services offer a comprehensive suite of enablement services and tools to help you develop your career, learn new skills, and develop your craftsmanship. As a business leader, it empowers me with a wide range of solutions to advance my team’s careers while driving innovative solutions for the organization. The path to becoming an ML professional is open to anyone, regardless of your background. For organizations like Thermo Fisher Scientific, it’s the game changer necessary to help our customers continue advancing science.

What about you? Have you been wondering about developing the ML career path at your organization? If so, check out the AWS Machine Learning Embark program. Only then will you know if this can open as many doors for you and your organization as it did for our Thermo Fisher Scientific ML engineers.

About the Author

Mikael Graindorge is a Sales Operations Leader at Thermo Fisher Scientific. His passion is to combine his craftsmanship with modern technology by developing new global solutions to drive sales conversion rates, advance life science research, and enable others to reach their full potential. He is also known for his cooking and carpentry skills, and is committed to lifelong learning. Mikael holds a master’s and a doctorate specialized in digital commerce growth by utilizing cognitive psychology stimulation.

AI/ML

Trending AI/ML Article Identified & Digested via Granola by Ramsey Elbasheer; a Machine-Driven RSS Bot

via WordPress https://ramseyelbasheer.io/2021/08/31/increase-your-machine-learning-success-with-aws-ml-services-and-aws-machine-learning-embark/