Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Mastering Machine Learning Algorithms
Mastering Machine Learning Algorithms

Mastering Machine Learning Algorithms: Expert techniques for implementing popular machine learning algorithms, fine-tuning your models, and understanding how they work , Second Edition

Arrow left icon
Profile Icon Bonaccorso Profile Icon Giuseppe Bonaccorso
Arrow right icon
€18.99 per month
Full star icon Full star icon Full star icon Full star icon Empty star icon 4 (12 Ratings)
Paperback Jan 2020 798 pages 2nd Edition
eBook
€20.98 €29.99
Paperback
€36.99
Subscription
Free Trial
Renews at €18.99p/m
Arrow left icon
Profile Icon Bonaccorso Profile Icon Giuseppe Bonaccorso
Arrow right icon
€18.99 per month
Full star icon Full star icon Full star icon Full star icon Empty star icon 4 (12 Ratings)
Paperback Jan 2020 798 pages 2nd Edition
eBook
€20.98 €29.99
Paperback
€36.99
Subscription
Free Trial
Renews at €18.99p/m
eBook
€20.98 €29.99
Paperback
€36.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with a Packt Subscription?

Free for first 7 days. €18.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing
Table of content icon View table of contents Preview book icon Preview Book

Mastering Machine Learning Algorithms

Loss Functions and Regularization

Loss functions are proxies that allow us to measure the error made by a machine learning model. They define the very structure of the problem to solve, and prepare the algorithm for an optimization step aimed at maximizing or minimizing the loss function. Through this process, we make sure that all our parameters are chosen in order to reduce the error as much as possible. In this chapter, we're going to discuss the fundamental loss functions and their properties. I've also included a dedicated section about the concept of regularization; regularized models are more resilient to overfitting, and can achieve results beyond the limits of a simple loss function.

In particular, we'll discuss:

  • Defining loss and cost functions
  • Examples of cost functions, including mean squared error and the Huber and hinge cost functions
  • Regularization
  • Examples of regularization, including Ridge, Lasso, ElasticNet, and early...

Defining loss and cost functions

Many machine learning problems can be expressed throughout a proxy function that measures the training error. The obvious implicit assumption is that, by reducing both training and validation errors, the accuracy increases, and the algorithm reaches its objective.

If we consider a supervised scenario (many considerations hold also for semi-supervised ones), with finite datasets X and Y:

We can define the generic loss function for a single data point as:

J is a function of the whole parameter set and must be proportional to the error between the true label and the predicted label.

A very important property of a loss function is convexity. In many real cases, this is an almost impossible condition; however, it's always useful to look for convex loss functions, because they can be easily optimized through the gradient descent method. We're going to discuss this topic in Chapter 10, Introduction...

Regularization

When a model is ill-conditioned or prone to overfitting, regularization offers some valid tools to mitigate the problems. From a mathematical viewpoint, a regularizer is a penalty added to the cost function, to impose an extra condition on the evolution of the parameters:

The parameter controls the strength of the regularization, which is expressed through the function . A fundamental condition on is that it must be differentiable so that the new composite cost function can still be optimized using SGD algorithms. In general, any regular function can be employed; however, we normally need a function that can contrast the indefinite growth of the parameters.

To understand the principle, let's consider the following diagram:

https://packt-type-cloud.s3.amazonaws.com/uploads/sites/3717/2019/05/IMG_49.png

Interpolation with a linear curve (left) and a parabolic one (right)

In the first diagram, the model is linear and has two parameters, while in the second one, it is quadratic and has three parameters. We already...

Summary

In this chapter, we introduced the loss and cost functions, first as proxies of the expected risk, and then we detailed some common situations that can be experienced during an optimization problem. We also exposed some common cost functions, together with their main features and specific applications.

In the last part, we discussed regularization, explaining how it can mitigate the effects of overfitting and induce sparsity. In particular, the employment of Lasso can help the data scientist to perform automatic feature selection by forcing all secondary coefficients to become equal to 0.

In the next chapter, Chapter 3, Introduction to Semi-Supervised Learning, we're going to introduce semi-supervised learning, focusing our attention on the concepts of transductive and inductive learning.

Further reading

  • Darwiche A., Human-Level Intelligence or Animal-Like Abilities?, Communications of the ACM, Vol. 61, 10/2018
  • Crammer K., Kearns M., Wortman J., Learning from Multiple Sources, Journal of Machine Learning Research, 9/2008
  • Mohri M., Rostamizadeh A., Talwalkar A., Foundations of Machine Learning, Second edition, The MIT Press, 2018
  • Valiant L., A theory of the learnable, Communications of the ACM, 27, 1984
  • Ng A. Y., Feature selection, L1 vs. L2 regularization, and rotational invariance, ICML, 2004
  • Dube S., High Dimensional Spaces, Deep Learning and Adversarial Examples, arXiv:1801.00634 [cs.CV]
  • Sra S., Nowozin S., Wright S. J. (edited by), Optimization for Machine Learning, The MIT Press, 2011
  • Bonaccorso G., Machine Learning Algorithms, Second Edition, Packt, 2018
Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Updated to include new algorithms and techniques
  • Code updated to Python 3.8 & TensorFlow 2.x
  • New coverage of regression analysis, time series analysis, deep learning models, and cutting-edge applications

Description

Mastering Machine Learning Algorithms, Second Edition helps you harness the real power of machine learning algorithms in order to implement smarter ways of meeting today's overwhelming data needs. This newly updated and revised guide will help you master algorithms used widely in semi-supervised learning, reinforcement learning, supervised learning, and unsupervised learning domains. You will use all the modern libraries from the Python ecosystem – including NumPy and Keras – to extract features from varied complexities of data. Ranging from Bayesian models to the Markov chain Monte Carlo algorithm to Hidden Markov models, this machine learning book teaches you how to extract features from your dataset, perform complex dimensionality reduction, and train supervised and semi-supervised models by making use of Python-based libraries such as scikit-learn. You will also discover practical applications for complex techniques such as maximum likelihood estimation, Hebbian learning, and ensemble learning, and how to use TensorFlow 2.x to train effective deep neural networks. By the end of this book, you will be ready to implement and solve end-to-end machine learning problems and use case scenarios.

Who is this book for?

This book is for data science professionals who want to delve into complex ML algorithms to understand how various machine learning models can be built. Knowledge of Python programming is required.

What you will learn

  • Understand the characteristics of a machine learning algorithm
  • Implement algorithms from supervised, semi-supervised, unsupervised, and RL domains
  • Learn how regression works in time-series analysis and risk prediction
  • Create, model, and train complex probabilistic models
  • Cluster high-dimensional data and evaluate model accuracy
  • Discover how artificial neural networks work – train, optimize, and validate them
  • Work with autoencoders, Hebbian networks, and GANs

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jan 31, 2020
Length: 798 pages
Edition : 2nd
Language : English
ISBN-13 : 9781838820299
Vendor :
Google
Category :
Languages :
Tools :

What do you get with a Packt Subscription?

Free for first 7 days. €18.99 p/m after that. Cancel any time!
Product feature icon Unlimited ad-free access to the largest independent learning library in tech. Access this title and thousands more!
Product feature icon 50+ new titles added per month, including many first-to-market concepts and exclusive early access to books as they are being written.
Product feature icon Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.
Product feature icon Thousands of reference materials covering every tech concept you need to stay up to date.
Subscribe now
View plans & pricing

Product Details

Publication date : Jan 31, 2020
Length: 798 pages
Edition : 2nd
Language : English
ISBN-13 : 9781838820299
Vendor :
Google
Category :
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 122.97
Mastering Machine Learning Algorithms
€36.99
Python Machine Learning
€41.99
Machine Learning for Algorithmic Trading
€43.99
Total 122.97 Stars icon
Banner background image

Table of Contents

27 Chapters
Machine Learning Model Fundamentals Chevron down icon Chevron up icon
Loss Functions and Regularization Chevron down icon Chevron up icon
Introduction to Semi-Supervised Learning Chevron down icon Chevron up icon
Advanced Semi-Supervised Classification Chevron down icon Chevron up icon
Graph-Based Semi-Supervised Learning Chevron down icon Chevron up icon
Clustering and Unsupervised Models Chevron down icon Chevron up icon
Advanced Clustering and Unsupervised Models Chevron down icon Chevron up icon
Clustering and Unsupervised Models for Marketing Chevron down icon Chevron up icon
Generalized Linear Models and Regression Chevron down icon Chevron up icon
Introduction to Time-Series Analysis Chevron down icon Chevron up icon
Bayesian Networks and Hidden Markov Models Chevron down icon Chevron up icon
The EM Algorithm Chevron down icon Chevron up icon
Component Analysis and Dimensionality Reduction Chevron down icon Chevron up icon
Hebbian Learning Chevron down icon Chevron up icon
Fundamentals of Ensemble Learning Chevron down icon Chevron up icon
Advanced Boosting Algorithms Chevron down icon Chevron up icon
Modeling Neural Networks Chevron down icon Chevron up icon
Optimizing Neural Networks Chevron down icon Chevron up icon
Deep Convolutional Networks Chevron down icon Chevron up icon
Recurrent Neural Networks Chevron down icon Chevron up icon
Autoencoders Chevron down icon Chevron up icon
Introduction to Generative Adversarial Networks Chevron down icon Chevron up icon
Deep Belief Networks Chevron down icon Chevron up icon
Introduction to Reinforcement Learning Chevron down icon Chevron up icon
Advanced Policy Estimation Algorithms Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
(12 Ratings)
5 star 50%
4 star 25%
3 star 8.3%
2 star 8.3%
1 star 8.3%
Filter icon Filter
Top Reviews

Filter reviews by




hawkinflight Aug 13, 2020
Full star icon Full star icon Full star icon Full star icon Full star icon 5
The book covers a lot of methods, as can be seen in the table of contents. This provides a nice breadth. What is really nice, is that before getting into any of the methods, the author starts with a chapter on ML Model Fundamentals. In this chapter, he presents not just the how's but also the why's of scaling data before modeling. He also talks about model capacity, bias and variance of an estimator, and how these relate to under or over fitting a model. Recently, I have heard two things during presentations: 1) "I won't go over scaling, because I think we all know when to apply these transformations" 2) "I don't even know what you're talking about" when asked about a bias/variance trade-off. I think it's great to have the opportunity to read a practitioner's explanation of these matters. He also talks about the options for splitting a data set into portions: 1) training, validation, and test portions, or 2) just into training and test portions, or 3) using a cross-validation approach. I have taken an ML class on Coursera where the first option was used, and the others weren't mentioned. It's nice to see all three here. He helps the reader by noting that some topics he's introducing will be discussed in more detail in the next few chapters. That's nice because questions start arising in your head, and then you read that and know there's more detail and answers coming, so just relax, and look forward.There are nice plots and Python code snippets throughout. It's helpful that each chapter ends with a summary and list of references.A lot of times we hear "supervised" or "unsupervised". It's nice that the book has three chapters on "semi-supervised" learning. In the Graph-Based Semi-Supervised Learning chapter, there is a "t-distributed stochastic neighbor embedding" (t-SNE) example, which is a topic I was curious about.
Amazon Verified review Amazon
Alfred H. Mar 06, 2020
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I look at algorithms like appliances in a department store. Each has a particular use-case and purpose. For instance, depending on your individual needs while shopping for appliances most often stop to glance at the specs then at the cost to see if a particular unit suits their needs.Few ever think to build the appliance from scratch but rather by one already made to serve their needs progressively adopting it as an augmentation to everyday life. Even easier is when these items can be cataloged in one place, shopped, and put into use immediately (think Sears catalog of 1888). In its first catalog, Sears sold jewelry and watches. The directories grew in popularity, and with time different products were added and tested, even whole houses!While algorithms are not a new thing, thanks to the father of algebra: Abdullah Muhammad bin Musa al-Khwarizmi, it should NOT be challenging to catalog them. This book one of the best, like it, does that facilitating faster solutioning to get to the point of solving problems using built appliances (machine intelligence: algorithms).
Amazon Verified review Amazon
Thom Ives, Ph.D. May 26, 2020
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Our exciting field of data science (DS) is exploding, and it’s hard to keep up with all of it. We each become extremely focused on our specific work in our current roles, and we are each funneled into specialized areas to solve specific challenges. After a long battle to deliver your great DS tool to production, your next challenge arrives. It seems this problem will require a DS method that you haven’t used since college. Maybe you’re not even sure, which DS method will best address your problem. Regardless, once you decide on a method of modeling, you have limited time to master it. You still need to collect, refine, and condition your data for that method. You must also master how to fight through the training of your chosen model. You’d seek help from your fellow DS’s, but they’re in the situation you just left.Now enters Giuseppe Bonaccorso - a DS friend with a corpus of DS methods that provide adequate mathematical overviews, explanations, and python code applied to substantial examples to get you up to speed quickly. I believe in keeping multiple sources at hand for learning / reviewing any methods. In that spirit, I'm relieved to have Giuseppe's book in my library. In a mere 750+ pages, he takes you on a tour of important methods that, while they won't make you an expert in the foundational mathematics for each method, they won't leave you blind in those respects either. I especially appreciate the references to foundational papers for each method following every group of methods for when we might need or want to go deeper. If you estimate that your library could be enhanced by such a substantial book as I've described, I believe you'd be well benefited by Giuseppe's book. He even manages to provide a good review of reinforced learning that, in my opinion, is extremely challenging to teach clearly.An important note is that this book does not cover natural language processing. In my experience, NLP would require another 750+ pages if it were to be covered as well as the methods that Giuseppe has covered. However, this is the only major field that he skips, and he does relate areas of that field to the techniques that he covers.
Amazon Verified review Amazon
TD59 Jun 13, 2021
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I have used this book (1st and 2nd edition) in my Machine Learning class for a couple of years. Together with the Raschka and Mirjalili book (Python Machine Learning), Bonnacorso's provides a solid foundation for understanding the key algorithms, how they work, and how to fine-tune them. Both amanuals re required in my class and my students have only had positive feedback about both books. They are complementary as Bonnacorso does not cover some topics such as Natural Language Processing compared with Raschka, for instance.
Amazon Verified review Amazon
Duubar Villalobos Feb 19, 2020
Full star icon Full star icon Full star icon Full star icon Full star icon 5
As Giuseppe Bonaccorso expresses in this book, "It's always possible to join scientific rigor with an artistic approach." This book offers a great resource to dust off those concepts for the advance practitioners or to learn the fundamentals for those willing to master their ML algorithms artistically.I've been reading this book for over a week by now, and I like how Giuseppe explains important theoretical concepts related to machine learning models, bias, variance, overfitting, underfitting, data normalization, scaling, and so on.Even though this book requires a solid knowledge of essential machine learning topics and familiarity with Python programming language, don't be discouraged. I found out that this book provides the best first-hand experiential advice that someone can provide for someone willing to learn. Moreover, given the complexity of some subjects, proper mathematical training is desirable, but the willingness to learn outweighs it.If you are looking to get great advice and insights from an expert, this book is for you. My recommendation is not to rush over the concepts. As I am reading --still not finished since it's 800 pages, it makes me reflect on my problem-solving styles and helps me identify areas of opportunity for improvement.I have to thank Giuseppe for taking his time to think, sort his thoughts, write, and for sharing his knowledge and experiences for us to become better practitioners.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is included in a Packt subscription? Chevron down icon Chevron up icon

A subscription provides you with full access to view all Packt and licnesed content online, this includes exclusive access to Early Access titles. Depending on the tier chosen you can also earn credits and discounts to use for owning content

How can I cancel my subscription? Chevron down icon Chevron up icon

To cancel your subscription with us simply go to the account page - found in the top right of the page or at https://subscription.packtpub.com/my-account/subscription - From here you will see the ‘cancel subscription’ button in the grey box with your subscription information in.

What are credits? Chevron down icon Chevron up icon

Credits can be earned from reading 40 section of any title within the payment cycle - a month starting from the day of subscription payment. You also earn a Credit every month if you subscribe to our annual or 18 month plans. Credits can be used to buy books DRM free, the same way that you would pay for a book. Your credits can be found in the subscription homepage - subscription.packtpub.com - clicking on ‘the my’ library dropdown and selecting ‘credits’.

What happens if an Early Access Course is cancelled? Chevron down icon Chevron up icon

Projects are rarely cancelled, but sometimes it's unavoidable. If an Early Access course is cancelled or excessively delayed, you can exchange your purchase for another course. For further details, please contact us here.

Where can I send feedback about an Early Access title? Chevron down icon Chevron up icon

If you have any feedback about the product you're reading, or Early Access in general, then please fill out a contact form here and we'll make sure the feedback gets to the right team. 

Can I download the code files for Early Access titles? Chevron down icon Chevron up icon

We try to ensure that all books in Early Access have code available to use, download, and fork on GitHub. This helps us be more agile in the development of the book, and helps keep the often changing code base of new versions and new technologies as up to date as possible. Unfortunately, however, there will be rare cases when it is not possible for us to have downloadable code samples available until publication.

When we publish the book, the code files will also be available to download from the Packt website.

How accurate is the publication date? Chevron down icon Chevron up icon

The publication date is as accurate as we can be at any point in the project. Unfortunately, delays can happen. Often those delays are out of our control, such as changes to the technology code base or delays in the tech release. We do our best to give you an accurate estimate of the publication date at any given time, and as more chapters are delivered, the more accurate the delivery date will become.

How will I know when new chapters are ready? Chevron down icon Chevron up icon

We'll let you know every time there has been an update to a course that you've bought in Early Access. You'll get an email to let you know there has been a new chapter, or a change to a previous chapter. The new chapters are automatically added to your account, so you can also check back there any time you're ready and download or read them online.

I am a Packt subscriber, do I get Early Access? Chevron down icon Chevron up icon

Yes, all Early Access content is fully available through your subscription. You will need to have a paid for or active trial subscription in order to access all titles.

How is Early Access delivered? Chevron down icon Chevron up icon

Early Access is currently only available as a PDF or through our online reader. As we make changes or add new chapters, the files in your Packt account will be updated so you can download them again or view them online immediately.

How do I buy Early Access content? Chevron down icon Chevron up icon

Early Access is a way of us getting our content to you quicker, but the method of buying the Early Access course is still the same. Just find the course you want to buy, go through the check-out steps, and you’ll get a confirmation email from us with information and a link to the relevant Early Access courses.

What is Early Access? Chevron down icon Chevron up icon

Keeping up to date with the latest technology is difficult; new versions, new frameworks, new techniques. This feature gives you a head-start to our content, as it's being created. With Early Access you'll receive each chapter as it's written, and get regular updates throughout the product's development, as well as the final course as soon as it's ready.We created Early Access as a means of giving you the information you need, as soon as it's available. As we go through the process of developing a course, 99% of it can be ready but we can't publish until that last 1% falls in to place. Early Access helps to unlock the potential of our content early, to help you start your learning when you need it most. You not only get access to every chapter as it's delivered, edited, and updated, but you'll also get the finalized, DRM-free product to download in any format you want when it's published. As a member of Packt, you'll also be eligible for our exclusive offers, including a free course every day, and discounts on new and popular titles.