Topics Covered:
- Regularization in Machine Learning
- Data Noise in Machine Learning
- Overfitting in Machine Learning
There are many factors involved when it comes to training a machine learning model. One of the most critical things to keep in mind is how to avoid overfitting. Overfitting can drastically decrease the accuracy of the model when it captures too much unnecessary noise from the overall dataset.
Noise, in regards to data, refers to all the extra information that is not useful to the model. It can also be referred to as ‘corrupt data’ and includes anomalies, information the system doesn’t understand, or data it cannot correctly interpret for some other reason.
When your machine learning model uses this noise, it treats it with the same importance as any other data points in the dataset. This can add to the overall robustness of the data being used and it may be necessary when using very small datasets, but all too often, it causes the model to learn from data it shouldn’t include. This is overfitting and is one of the two causes for machine learning models to underperform (the other being underfitting).
Let’s put this into simpler terms.
Imagine you are trying to teach a machine learning model to predict the height of a baby giraffe before it is born. Your training data consists of tons of information about baby giraffes who have already been born and their parents. Your machine learning model’s goal is to look at the parents’ information, such as their height, weight, their diet, their geographic location, etc., and try to find patterns that correlate with the height of their newborn. While many different traits (data points) of the parent giraffes will affect the outcome, their height is likely the most important. The taller the parent, the taller the child, right?
Usually, but not always.
There will almost certainly be some outliers in the dataset. There may be one or two parent giraffes who were abnormally tall yet had abnormally short children. When your machine learning model learns from these anomalies and gives them the same level of importance as all of the other information, it will likely skew or reduce the accuracy of the results. In machine learning, this is overfitting, and regularization attempts to remedy the problem.
There are a variety of methods used to regularize data in machine learning.
To learn the math behind regularization, using scikit-learn in Python, take a look at this article.
Are you looking for a job in Information Technology?
See all of our current openings here!
Check out our latest video on YouTube!
About the Company:
Peterson Technology Partners (PTP) has been Chicago’s premier Information Technology (IT) staffing, consulting, and recruiting firm for over 22+ years. Named after Chicago’s historic Peterson Avenue, PTP has built its reputation by developing lasting relationships, leading digital transformation, and inspiring technical innovation throughout Chicagoland.
Based in Park Ridge, IL, PTP’s 250+ employees have a narrow focus on a single market (Chicago) and expertise in 4 innovative technical areas;
- Cybersecurity
- Artificial Intelligence
- Data Science
- Cloud & DevOps
PTP exists to ensure that all of our partners (clients and candidates alike) make the best hiring and career decisions.
Connect: LinkedIn | Facebook | Twitter | YouTube
Review: Google | Glassdoor | Yelp | All Review Links
Apply: Stack Overflow | Dice | LinkedIn | Glassdoor | All Job Openings
Peterson Technology Partners is an equal opportunity employer.
