E-Learn Knowledge Base
Machine learning has become a household term in recent years as the concept moved from science fiction to a key driver of how businesses and organizations process information. With the pace of data creation continuing to grow exponentially, machine learning tools are pivotal for organizations looking to discover patterns, tease out trends, and chart the most profitable path forward.
How commonplace is machine learning? If you’ve clicked on a recommendation from an ecommerce website or streaming platform, been notified of potential misuse of a credit card, or used transcription software, you’ve benefited from machine learning. It’s used in finance, healthcare, marketing, retail, and many other industries to extract valuable insights from data and automate processes.
What Is Machine Learning?
Machine learning (ML) is the subset of artificial intelligence that focuses on building systems that learn—and improve—as they consume more data. Artificial intelligence is a broader term that refers to systems or machines that mimic human intelligence. Machine learning and AI are often discussed together, and the terms are sometimes used interchangeably, but they don’t mean the same thing.
In short, all machine learning is AI, but not all AI is machine learning.
Key Takeaways
- Machine learning is a subset of AI.
- The four most common types of machine learning are supervised, unsupervised, semi-supervised, and reinforced.
- Popular types of machine learning algorithms include neural networks, decision trees, clustering, and random forests.
- Common machine learning use cases in business include object identification and classification, anomaly detection, document processing, and predictive analysis.
Machine Learning Explained
Machine learning is a technique that discovers previously unknown relationships in data by searching potentially very large data sets to discover patterns and trends that go beyond simple statistical analysis. Machine learning uses sophisticated algorithms that are trained to identify patterns in data, creating models. Those models can be used to make predictions and categorize data.
Note that an algorithm isn’t the same as a model. An algorithm is a set of rules and procedures used to solve a specific problem or perform a particular task, while a model is the output or result of applying an algorithm to a data set.
Before training, you have an algorithm. After training, you have a model.
For example, machine learning is widely used in healthcare for tasks including medical imaging analysis, predictive analytics, and disease diagnosis. Machine learning models are ideally suited to analyze medical images, such as MRI scans, X-rays, and CT scans, to identify patterns and detect abnormalities that may not be visible to the human eye or that an overworked diagnostician might miss. Machine learning systems can also analyze symptoms, genetic information, and other patient data to suggest tests for conditions such as cancer, diabetes, and heart disease.
The key features of machine learning are the
- Automatic discovery of patterns
- Prediction of likely outcomes
- Creation of actionable information
- Ability to analyze very large volumes of data
Types of Machine Learning
There are four main types of machine learning. Each has its own strengths and limitations, making it important to choose the right approach for the specific task at hand.
- Supervised machine learning is the most common type. Here, labeled data teaches the algorithm what conclusions it should make. Just as a child learns to identify fruits by memorizing them in a picture book, in supervised learning the algorithm is trained by a data set that’s already labeled. An example of supervised machine learning is a spam email filter, where the algorithm is trained on a labeled data set in which each email is tagged as either spam or not spam. The model learns from these labeled examples and then can predict whether new incoming emails are likely spam or not based on the patterns it identified. This type of supervised learning requires a human expert to provide the correct answers by labeling data so the algorithm can learn and make accurate predictions in the future.
Note that the goal here isn’t to train using pristine data. You want to mimic what the system will see in the real world—some spam is easy to spot, but other examples are stealthy or borderline. Overly clean data leads to overfitting, meaning the model will identify only other pristine samples. - Unsupervised machine learning employs a more independent approach, in which a computer learns to identify complex processes and patterns without relying on previously labeled data. Unsupervised machine learning not only involves training based on data that doesn’t have labels; there’s also no specific, defined output, such as whether an email is likely spam.
Unsupervised machine learning tends to spot groupings of similar data, creating clusters. Once trained, the model can identify similar patterns and put that data into its appropriate group. A common use of unsupervised machine learning is recommendation engines, which are used in consumer applications to provide “customers who bought that also bought this” suggestions. When dissimilar patterns are found, the algorithm can identify them as anomalies, which is useful in fraud detection. - Semi-supervised machine learning addresses the problem of not having enough labeled data to fully train a model. For instance, you might have large training data sets but don’t want to incur the time and cost of labeling the entire set. By using a combination of supervised and unsupervised methods, you can often achieve a fully trained model. The training process starts off like supervised learning, using labeled data to get initial results and establish guidelines for the algorithm. When labeled data is exhausted, the semi-trained model is given the unlabeled data sets. It uses the training it has and looks at the unlabeled data, with the goal of adding to the labeled data set. If the model can find an appropriate label for a sample with high certainty, that sample is added to the labeled data. The learning process starts again, but now with a larger set of labeled samples. By iterating, more samples are labeled with what’s often called pseudo-labels, and the model can be further refined.
Reinforcement machine learning, like unsupervised machine learning, uses unlabeled data sets and allows algorithms to evaluate the data. However, reinforcement learning differs in that it’s working toward a set goal rather than exploring data to discover whatever patterns might exist. With an objective in mind, the algorithm proceeds in a trial-and-error process. Each move receives positive, negative, or neutral feedback, which the algorithm uses to hone its overall decision-making process. Reinforcement learning algorithms can work on a macro level toward the project goal, even if that means dealing with short-term negative consequences. In that way, reinforcement learning handles more complex and dynamic situations than other methods because it allows the context of the project goal to influence the risk in choices. Teaching a computer to play chess is a good example. The overall goal is to win the game, but that may require sacrificing pieces as the game goes on.
Which is best for your needs?
Choosing a supervised approach or one of the other three methods usually depends on the structure and volume of your data, the budget and hours that can be devoted to training, and the use case to which you want to apply the final model. Whiffing on a suggestion for a blouse to go with a skirt may be inconsequential. Missing a tumor, less so.
How Machine Learning Works
As its name indicates, machine learning works by creating computer-based statistical models that are refined for a given purpose by evaluating training data, rather than by the classical approach where programmers develop a static algorithm that attempts to solve a problem. As data sets are put through the ML model, the resulting output is judged on accuracy, allowing data scientists to adjust the model through a series of established variables, called hyperparameters, and algorithmically adjusted variables, called learning parameters.
Because the algorithm adjusts as it evaluates training data, the process of exposure and calculation around new data trains the algorithm to become better at what it does. The algorithm is the computational part of the project, while the term “model” is a trained algorithm that can be used for real-word use cases.
The scope, resources, and goals of machine learning projects will determine the most appropriate path, but most involve a series of steps.
1. Gather and compile data
Training ML models requires a lot of high-quality data. Finding it is sometimes difficult, and labeling it, if necessary, can be very resource intensive. After identifying potential data sources, evaluate them to determine overall quality and alignment with the project’s existing data integration/repository resources. Those sources form the training foundation of a machine learning project.
2. Select an appropriate algorithm to yield the desired model
Depending on whether the project plans to use supervised, unsupervised, or semi-supervised learning, data scientists can select the most appropriate algorithms. For example, a simpler project with a labeled data set can use a decision tree, while clustering—dividing data samples into groups of similar objects—requires more compute resources as the algorithm works unsupervised to determine the best path to a goal.
3. Refine and prepare data for analysis
Chances are that incoming data won’t be ready to go. Data preparation cleans up data sets to ensure that all records can be easily ingested during training. Preparation includes a range of transformation tasks, such as establishing date and time formats, joining or separating columns as needed, and setting other format parameters, such as acceptable significant digits in real number data. Other key tasks include cleaning out duplicate records, also called data deduplication, and identifying and possibly removing outliers.
4. Educate the model through training
Once the desired final model has been selected, the training process begins. In training, a curated data set, either labeled or unlabeled, is fed to the algorithm. In initial runs, outcomes may not be great, but data scientists will tweak as needed to refine performance and increase accuracy. Then the algorithm is shown data again, usually in larger quantities to tune it more precisely. The more data the algorithm sees, the better the final model should become at delivering the desired results.
5. Assess model performance and accuracy
After the model has been trained to sufficient accuracy, it’s time to give it previously unseen data to test how it performs. Often, the data used for testing is a subset of the training data set aside for use after initial training.
6. Fine-tune and enhance model parameters
The model now is most likely close to deployment. Runs with test data sets should produce highly accurate results. Enhancements happen through additional training with specific data—often unique to a company’s operations—to supplement the generalized data used in the original training.
7. Launch the model
With results optimized, the model is now ready to tackle previously unseen data in normal production use. When the model is live, project teams will collect data on how the model performs in real-world scenarios. This can be done by monitoring key performance metrics, such as accuracy, the overall correctness of the model’s predictions, and recall, the ratio of correctly predicted positive observations. Also consider how the model’s predictions are affecting business outcomes on the ground—is it generating value, whether in increased sales of blouses or better diagnostics?
Conducting regular audits and reviews of the model’s performance can help identify issues or distortions that may have arisen post-deployment and are essential to ensure that the model performs effectively and meets the desired objectives.
Machine Learning Algorithms
Algorithms are the computational part of a machine learning project. Once trained, algorithms produce models with a statistical probability of answering a question or achieving a goal. That goal might be finding certain features in images, such as “identify all the cats,” or it might be to spot anomalies in data that could indicate fraud, spam, or a maintenance issue with a machine. Still other algorithms might attempt to make predictions, such as which clothing items a buyer might also like based on what’s currently in a shopping cart.
Some of the most common algorithms used in machine learning are as follows:
- Neural networks: The human brain works by creating networks of nodes that perform functions as varied as processing images from our eyes, storing memories, and controlling our muscles. Computer-based neural networks are modeled after this brain architecture, creating layers of nodes that weigh the relationships between data they’ve analyzed and data in adjacent nodes. Working as a network, these nodes can determine features of data, such as elements within a picture.
- Linear regression: Linear regression algorithms take data points and build a mathematical equation for a line that best supports predicted outcomes. This is sometimes known as the “line of best fit.” Linear regression works by tweaking variables in the equation to minimize the errors in predictions. An example of linear regression is seen in pediatric care, where different data points can predict a child’s height and weight based on historical data. Similarly, BMI is linear regression that attempts to correlate height and weight to overall body fat. Because the algorithm uses a simple line for its predictions, error rates can be high. For instance, a BMI of 30 indicates obesity. That’s often true for the general public but definitely not for strength athletes such as NFL linebackers.
- Logistic regression: Best used for binary outcomes, logistic regression is like linear regression but with special considerations at the boundaries of possible data ranges. An example of logistic regression includes pass/fail analysis on the likelihood of converting a potential customer into a paying one. Logistic regression is often used in medical diagnoses—for instance, plasma glucose concentrations over a certain range are used as a strong indicator of diabetes. Logistic regression also can be used to predict whether an email is spam or not, or if a credit card transaction is fraudulent.
- Clustering: Clustering is a form of unsupervised learning that exposes an algorithm to unlabeled data sets in which data may fall into distinct groups, or clusters. As the algorithm evaluates training data, it searches for patterns and overlapping details between the data and creates groups. Say the algorithm is looking at the nutritional value of a wide variety of fruits. It might find that citrus fruits form one group, berries form another, and melons form a third. This allows for easier understanding of what makes each cluster unique. Clustering is particularly useful for any sort of categorization project, such as market segmentation.
- Decision trees: Decision trees use supervised learning and basic if-then progressions to make predictions. Depending on the complexity of the project, decision trees can be ideal as resource-light algorithms that produce straightforward results. For example, if a college wanted to determine which students could skip freshman English, it might use a decision tree that first asked if the student had taken four years of English in high school and, if so, whether the student had at least a 3.6 GPA in those classes. Another path might simply ask if the student had scored better than a certain threshold on the reading, writing, and language SAT sections.
- Random forests: On their own, decision trees come with limitations due to their inherent rigid workflows and requirement that all evaluation questions be answered. In our decision tree example above, the college might require that both conditions be true, even though meeting just one might be sufficient. And there could be other factors to consider, such as how well a student performed on an admissions essay. If all three factors were required, the college would have a problem of requiring an “overfit” to the decision tree. Students who fared well on all three criteria would highly likely be fine without taking English 101. However, others who also would be fine still will be required to take the class. Random forests bring together collections of decision trees that cumulatively weigh outcomes to present a broader perspective. With random forests, projects can still use the core mechanics of decision trees while considering nuanced relationships between relevant data points. So, our college might split these three criteria into separate decision trees and then use weights to decide who can skip English 101 based on finding, perhaps, that doing well in high school English is the most predictive indicator and that performance on the essay is the least.
Beyond Neural Networks
Machine learning uses a vast array of algorithms. While the ones discussed above reign supreme in popularity, here are five less common but still useful algorithms.
| Gradient boosting | Builds models sequentially by focusing on previous errors in the sequence. Useful for fraud and spam detection. |
| K-nearest neighbors (KNN) | A simple yet effective model that classifies data points based on the labels of their nearest neighbors in the training data. |
| Principal component analysis (PCA) | Reduces data dimensionality by identifying the most significant features. It’s useful for visualization and data compression for, for example, anomaly detection. |
| Q-learning | Employs and agent that learns through trial and error, receiving rewards for desired actions and penalties for making the wrong move. |
| Support vector machines (SVM) | Creates a hyperplane to effectively separate data points belonging to different classes, such as image classification. |
Benefits of Machine Learning
Machine learning lets organizations extract insights from their data that they might not be able to find any other way. Some of the most common benefits from integrating machine learning into processes include the following:
- Streamlining Decision-Making and Predictive Analysis: Data-driven decisions start with data analysis. That’s an obvious statement, but when done manually the analysis process is time- and resource-intensive and may not yield rich enough insights to justify the cost. Machine learning can comb through large volumes of data to identify trends and patterns so that users can focus on queries and actionable results rather than optimizing manual data processing. Depending on the analytics tool, machine learning can generate predictions and identify hard-to-find insights in the data, allowing for a greater depth of analysis and more value to the organization.
- Boosting Efficiency and Automating Tasks: Machine learning is at the root of many of the technologies that make workers more efficient. Many low-cognition, repetitive tasks—including spell-checking as well as document digitization and classification—are now done by computers, thanks to machine learning.
Machine learning also excels at the lightning fast, in-the-moment data analysis that’s extremely difficult for humans. Is that transaction fraudulent, or is that email a phishing scam? Machine learning systems can often accurately determine the answer in seconds and automatically take appropriate measures. By combining ML technologies, predictions can be made from data accompanied by explanations of the factors that influenced the prediction, helping executives chart the best paths for their organizations. - Personalization and Innovation in Services: Machine learning has opened a new door for customer experiences through personalization. Purchase history, browsing history, demographic data, and additional information can be used to build an individual customer profile, which then can be cross-referenced against similar profiles to make predictions about customer interests. This allows for suggestion engine offerings, auto-generated discounts, and other types of personalized engagement to keep customers returning.
Using the nomenclature discussed here: Decision trees can help classify customers. Perhaps one customer likes vintage and second-hand fashion while another likes cutting-edge designer garments. Clustering each customer with similar ones can help identify the products they’ll likely click on or whether they’re likely to be highly motivated by sales. ML then can predict the best offers for each customer by considering which clusters they fit in.
Machine Learning Challenges
Machine learning projects are only as effective as the system and resources they’re built with. That highlights the need to invest in proper planning and preparation.
The following are some of the most common challenges facing machine learning projects:
- Data quality: The adage “garbage in, garbage out” applies to machine learning—the quality of data is critical, during both the training phase and in production. High-quality data can lead to more accurate results delivered in a timely, efficient manner; low-quality data can create inaccuracies and distortion in resultant models. Note that “quality” can mean different things to different projects. For training image recognition systems, the data should represent what the model will see in the real world. That includes subjects in shadows, slightly out of focus, and not looking directly into the camera. For training purposes, the more data resembles what the system will see in production, the better.
To that end, organizations should vet data sources, transform data sets for consistent and compatible formats, run cleansing and deduplication procedures, train users on process and protocols, and integrate tools to assess quality and fit. - Bias: Data may be clean, but is it free from bias? As an obvious case, let’s say you wanted to train a machine learning system to detect dogs in pictures, and you’ve got a robust data set of only Labrador and poodle photos. After training, the model is great at detecting these dogs—you could say it’s biased to do so. But when shown a picture of a bulldog, it says it can’t find a dog. Of course not—it wasn’t trained for that.
Creating the right training data set is one of the trickiest and most expensive aspects of building ML tools that work the way you want them to. The difficulty of finding certain types of data can often generate an unintended source of bias. For example, the Tibetan mastiff is a rare breed, but the dogs look like an orangish brown Newfoundland. So, if insufficient data samples exist of the Tibetan mastiff, it won’t be too surprising if an ML model identifies it as a Newfie. - Data security: Despite its many benefits, machine learning can introduce a range of security issues. The data used in ML analysis may contain sensitive or proprietary information not meant for public consumption. Similarly, data may be the target of a cyberattack meant to poison the model and build misinformation into the results. Data preparation steps can both expose and address security vulnerabilities, particularly when data sets go through export or import processes between systems. To help mitigate security issues, companies must employ an array of security policies, procedures, and controls, including practical staff training.
- Data privacy: Ensuring sensitive data isn’t disclosed is an ongoing effort. Data anonymization is an emerging practice, but it may not always be available or sufficient. For example, say a company wants to offer a service that allows its business customers to learn more about their end consumers based on data it has collected. This type of information would need to be protected and used consistent with legal requirements, and companies would need to carefully consider new threat vectors that may attempt to compromise the data through an ML system.
Machine Learning Use Cases
Machine learning can provide significant benefits for nearly every industry and every department within an organization. If numbers are crunched and data exists, machine learning offers a way to increase efficiency and derive new kinds of engagement. Common machine learning use cases across industries include the following:
- Retail: For retailers, machine learning can help operations, sales, and more. On an operations level, machine learning can analyze supply chain data to help optimize inventory management and identify possible delays early on. To help increase sales, machine learning can examine a customer’s search and browse history along with demographics to build a profile that acts as the basis for further engagement.
- Streaming media: Like retailers, both audio and video streamers can build customer profiles based on a user’s engagement and browsing history and demographic data. This profile then can power recommendation engines that help with discovery and further engagement.
- Finance: One of the most powerful, yet simple, uses of machine learning is fraud detection for the finance industry. With machine learning, algorithms can identify general account behavior, then begin flagging anomalous behavior for potential fraud investigations.
- Healthcare: The healthcare industry is primed for using machine learning in nearly every facet of operations. Patient profiles through electronic health records can identify possible issues ahead of time based on patterns compared to similar demographics. Data from Internet of Things devices, such as a smart medication dispenser, can flag errors quickly, and operational data from patient foot traffic or hospital bed use can inform staffing scalability.
Faster, More Secure Machine Learning with Oracle
Machine Learning in Oracle Database offers a spectrum of capabilities and features to accelerate the machine learning process. With the ability to keep data within the database, data scientists can simplify their workflow and increase security while taking advantage of more than 30 built-in, high performance algorithms; support for popular languages, including R, SQL, and Python; automated machine learning capabilities; and no-code interfaces.
For organizations with large data sets, in-database machine learning with HeatWave MySQL negates the need to move data to a separate system for machine learning, which can help increase security, reduce costs, and save time. HeatWave AutoML automates the machine learning lifecycle, including algorithm selection, intelligent data sampling for training, feature selection, and tuning, often saving even more time and effort.
The payoff for machine learning is the ability to analyze and interpret large amounts of data quickly and accurately. Once trained, machine learning models can identify in seconds or minutes patterns, trends, and insights that could take humans weeks to detect—or that might never see the light of day. The result is more informed decision-making, improved problem-solving, and the ability to make data-driven predictions. In addition, machine learning models can automate rote processes, saving time and resources. Machine learning is realizing its potential to revolutionize the workplace and drive innovation.
Machine Learning FAQs
What’s the difference between AI and ML?
Artificial intelligence is the name given to the broad computing subject focusing on building and refining systems to think like humans. Machine learning is a subset of this field that focuses specifically on the computational aspect of the learning process. The two terms are often used interchangeably and face similar challenges, but they exist separately despite this connection.
What are the four main types of machine learning?
The four types of machine learning are as follows:
- Supervised. Supervised learning uses labeled data sets to train the algorithm toward a specific goal.
- Unsupervised. Unsupervised learning uses unlabeled data sets that provide the algorithm space to explore and identify patterns.
- Semi-supervised. Semi-supervised learning uses labeled data sets for initial training to establish the broad parameters of the project. Then the algorithm uses that training to evaluate unlabeled samples to see if it can label them with a high probability. That process can be repeated—with the labeled sample set growing larger on each iteration.
- Reinforcement. Reinforcement learning acts similarly to unsupervised learning in that it uses unlabeled data sets. However, reinforcement learning focuses on learning the best path to achieve a specific goal through positive, negative, and neutral feedback rather than searching for patterns.
Is it hard to learn machine learning?
Like any technical craft, learning the ins and outs of machine learning is an iterative process that requires time and dedication. A good starting point for machine learning is to have a foundation in programming languages, such as Python or R, along with an understanding of statistics. Many elements involved with evaluating machine learning output require understanding statistical concepts, such as regression, classification, fitting, and parameters.
What is an example of machine learning?
One of the most common examples of machine learning is a suggestion engine. In ecommerce, this is seen as a “you may also like…” product suggestion. In video streaming media, this is seen as ideas for what to watch next. In these cases, the algorithm takes a user’s history and creates predictions for what the user may find interesting—and the more the user adds in data points, the more the algorithm can refine predictions.
Register for this course: Enrol Now
What is machine learning?
Machine learning (ML) is a branch of artificial intelligence (AI) focused on enabling computers and machines to imitate the way that humans learn, to perform tasks autonomously, and to improve their performance and accuracy through experience and exposure to more data.
UC Berkeleybreaks out the learning system of a machine learning algorithm into three main parts.
- A Decision Process: In general, machine learning algorithms are used to make a prediction or classification. Based on some input data, which can be labeled or unlabeled, your algorithm will produce an estimate about a pattern in the data.
- An Error Function: An error function evaluates the prediction of the model. If there are known examples, an error function can make a comparison to assess the accuracy of the model.
- A Model Optimization Process: If the model can fit better to the data points in the training set, then weights are adjusted to reduce the discrepancy between the known example and the model estimate. The algorithm will repeat this iterative “evaluate and optimize” process, updating weights autonomously until a threshold of accuracy has been met.
-
Machine learning versus deep learning versus neural networks
Since deep learning and machine learning tend to be used interchangeably, it’s worth noting the nuances between the two. Machine learning, deep learning, and neural networks are all sub-fields of artificial intelligence. However, neural networks is actually a sub-field of machine learning, and deep learning is a sub-field of neural networks.
The way in which deep learning and machine learning differ is in how each algorithm learns. "Deep" machine learning can use labeled datasets, also known as supervised learning, to inform its algorithm, but it doesn’t necessarily require a labeled dataset. The deep learning process can ingest unstructured data in its raw form (e.g., text or images), and it can automatically determine the set of features which distinguish different categories of data from one another. This eliminates some of the human intervention required and enables the use of large amounts of data. You can think of deep learning as "scalable machine learning"
Classical, or "non-deep," machine learning is more dependent on human intervention to learn. Human experts determine the set of features to understand the differences between data inputs, usually requiring more structured data to learn.
Neural networks, or artificial neural networks (ANNs), are comprised of node layers, containing an input layer, one or more hidden layers, and an output layer. Each node, or artificial neuron, connects to another and has an associated weight and threshold. If the output of any individual node is above the specified threshold value, that node is activated, sending data to the next layer of the network. Otherwise, no data is passed along to the next layer of the network by that node. The “deep” in deep learning is just referring to the number of layers in a neural network. A neural network that consists of more than three layers—which would be inclusive of the input and the output—can be considered a deep learning algorithm or a deep neural network. A neural network that only has three layers is just a basic neural network.
Deep learning and neural networks are credited with accelerating progress in areas such as computer vision, natural language processing, and speech recognition.
-
Machine learning methods
Machine learning models fall into three primary categories.
Supervised machine learning
Supervised learning, also known as supervised machine learning, is defined by its use of labeled datasets to train algorithms to classify data or predict outcomes accurately. As input data is fed into the model, the model adjusts its weights until it has been fitted appropriately. This occurs as part of the cross validation process to ensure that the model avoids overfitting or underfitting. Supervised learning helps organizations solve a variety of real-world problems at scale, such as classifying spam in a separate folder from your inbox. Some methods used in supervised learning include neural networks, naïve bayes, linear regression, logistic regression, random forest, and support vector machine (SVM).
Unsupervised machine learning
Unsupervised learning, also known as unsupervised machine learning, uses machine learning algorithms to analyze and cluster unlabeled datasets (subsets called clusters). These algorithms discover hidden patterns or data groupings without the need for human intervention. This method’s ability to discover similarities and differences in information make it ideal for exploratory data analysis, cross-selling strategies, customer segmentation, and image and pattern recognition. It’s also used to reduce the number of features in a model through the process of dimensionality reduction. Principal component analysis (PCA) and singular value decomposition (SVD) are two common approaches for this. Other algorithms used in unsupervised learning include neural networks, k-means clustering, and probabilistic clustering methods.
Semi-supervised learning
Semi-supervised learning offers a happy medium between supervised and unsupervised learning. During training, it uses a smaller labeled data set to guide classification and feature extraction from a larger, unlabeled data set. Semi-supervised learning can solve the problem of not having enough labeled data for a supervised learning algorithm. It also helps if it’s too costly to label enough data.
Reinforcement machine learning
Reinforcement machine learning is a machine learning model that is similar to supervised learning, but the algorithm isn’t trained using sample data. This model learns as it goes by using trial and error. A sequence of successful outcomes will be reinforced to develop the best recommendation or policy for a given problem.
The IBM Watson® system that won the Jeopardy! challenge in 2011 is a good example. The system used reinforcement learning to learn when to attempt an answer (or question, as it were), which square to select on the board, and how much to wager—especially on daily doubles.
Common machine learning algorithms
A number of machine learning algorithms are commonly used. These include:
- Neural networks
- Linear regression
- Logistic regression
- Clustering
- Decision trees
- Random forests
Neural networks
Neural networks simulate the way the human brain works, with a huge number of linked processing nodes. Neural networks are good at recognizing patterns and play an important role in applications including natural language translation, image recognition, speech recognition, and image creation.
Linear regression
This algorithm is used to predict numerical values, based on a linear relationship between different values. For example, the technique could be used to predict house prices based on historical data for the area.
Logistic regression
This supervised learning algorithm makes predictions for categorical response variables, such as “yes/no” answers to questions. It can be used for applications such as classifying spam and quality control on a production line.
Clustering
Using unsupervised learning, clustering algorithms can identify patterns in data so that it can be grouped. Computers can help data scientists by identifying differences between data items that humans have overlooked.
Decision trees
Decision trees can be used for both predicting numerical values (regression) and classifying data into categories. Decision trees use a branching sequence of linked decisions that can be represented with a tree diagram. One of the advantages of decision trees is that they are easy to validate and audit, unlike the black box of the neural network.
Random forests
In a random forest, the machine learning algorithm predicts a value or category by combining the results from a number of decision trees.
Advantages and disadvantages of machine learning algorithms
Depending on your budget, need for speed and precision required, each algorithm type—supervised, unsupervised, semi-supervised, or reinforcement—has its own advantages and disadvantages. For example, decision tree algorithms are used for both predicting numerical values (regression problems) and classifying data into categories. Decision trees use a branching sequence of linked decisions that may be represented with a tree diagram. A prime advantage of decision trees is that they are easier to validate and audit than a neural network. The bad news is that they can be more unstable than other decision predictors.
Overall, there are many advantages to machine learning that businesses can leverage for new efficiencies. These include machine learning identifying patterns and trends in massive volumes of data that humans might not spot at all. And this analysis requires little human intervention: just feed in the dataset of interest and let the machine learning system assemble and refine its own algorithms—which will continually improve with more data input over time. Customers and users can enjoy a more personalized experience as the model learns more with every experience with that person.
On the downside, machine learning requires large training datasets that are accurate and unbiased. GIGO is the operative factor: garbage in / garbage out. Gathering sufficient data and having a system robust enough to run it might also be a drain on resources. Machine learning can also be prone to error, depending on the input. With too small a sample, the system could produce a perfectly logical algorithm that is completely wrong or misleading. To avoid wasting budget or displeasing customers, organizations should act on the answers only when there is high confidence in the output.Real-world machine learning use cases
Here are just a few examples of machine learning you might encounter every day:
Speech recognition: It is also known as automatic speech recognition (ASR), computer speech recognition, or speech-to-text, and it is a capability which uses natural language processing (NLP) to translate human speech into a written format. Many mobile devices incorporate speech recognition into their systems to conduct voice search—e.g. Siri—or improve accessibility for texting.
Customer service:Online chatbots are replacing human agents along the customer journey, changing the way we think about customer engagement across websites and social media platforms. Chatbots answer frequently asked questions (FAQs) about topics such as shipping, or provide personalized advice, cross-selling products or suggesting sizes for users. Examples include virtual agents on e-commerce sites; messaging bots, using Slack and Facebook Messenger; and tasks usually done by virtual assistants and voice assistants.
Computer vision: This AI technology enables computers to derive meaningful information from digital images, videos, and other visual inputs, and then take the appropriate action. Powered by convolutional neural networks, computer vision has applications in photo tagging on social media, radiology imaging in healthcare, and self-driving cars in the automotive industry.
Recommendation engines: Using past consumption behavior data, AI algorithms can help to discover data trends that can be used to develop more effective cross-selling strategies. Recommendation engines are used by online retailers to make relevant product recommendations to customers during the checkout process.
Robotic process automation (RPA): Also known as software robotics, RPA uses intelligent automation technologies to perform repetitive manual tasks.Automated stock trading: Designed to optimize stock portfolios, AI-driven high-frequency trading platforms make thousands or even millions of trades per day without human intervention.
Fraud detection: Banks and other financial institutions can use machine learning to spot suspicious transactions. Supervised learning can train a model using information about known fraudulent transactions. Anomaly detection can identify transactions that look atypical and deserve further investigation.
Challenges of machine learning
As machine learning technology has developed, it has certainly made our lives easier. However, implementing machine learning in businesses has also raised a number of ethical concerns about AI technologies. Some of these include:
Technological singularity
While this topic garners a lot of public attention, many researchers are not concerned with the idea of AI surpassing human intelligence in the near future. Technological singularity is also referred to as strong AI or superintelligence. Philosopher Nick Bostrum defines superintelligence as “any intellect that vastly outperforms the best human brains in practically every field, including scientific creativity, general wisdom, and social skills.” Despite the fact that superintelligence is not imminent in society, the idea of it raises some interesting questions as we consider the use of autonomous systems, like self-driving cars. It’s unrealistic to think that a driverless car would never have an accident, but who is responsible and liable under those circumstances? Should we still develop autonomous vehicles, or do we limit this technology to semi-autonomous vehicles which help people drive safely? The jury is still out on this, but these are the types of ethical debates that are occurring as new, innovative AI technology develops.
AI impact on jobs
While a lot of public perception of artificial intelligence centers around job losses, this concern should probably be reframed. With every disruptive, new technology, we see that the market demand for specific job roles shifts. For example, when we look at the automotive industry, many manufacturers, like GM, are shifting to focus on electric vehicle production to align with green initiatives. The energy industry isn’t going away, but the source of energy is shifting from a fuel economy to an electric one.
In a similar way, artificial intelligence will shift the demand for jobs to other areas. There will need to be individuals to help manage AI systems. There will still need to be people to address more complex problems within the industries that are most likely to be affected by job demand shifts, such as customer service. The biggest challenge with artificial intelligence and its effect on the job market will be helping people to transition to new roles that are in demand.
Privacy
Privacy tends to be discussed in the context of data privacy, data protection, and data security. These concerns have allowed policymakers to make more strides in recent years. For example, in 2016, GDPR legislation was created to protect the personal data of people in the European Union and European Economic Area, giving individuals more control of their data. In the United States, individual states are developing policies, such as the California Consumer Privacy Act (CCPA), which was introduced in 2018 and requires businesses to inform consumers about the collection of their data. Legislation such as this has forced companies to rethink how they store and use personally identifiable information (PII). As a result, investments in security have become an increasing priority for businesses as they seek to eliminate any vulnerabilities and opportunities for surveillance, hacking, and cyberattacks.
Bias and discrimination
Instances of bias and discrimination across a number of machine learning systems have raised many ethical questions regarding the use of artificial intelligence. How can we safeguard against bias and discrimination when the training data itself may be generated by biased human processes? While companies typically have good intentions for their automation efforts, Reuters2 highlights some of the unforeseen consequences of incorporating AI into hiring practices. In their effort to automate and simplify a process, Amazon unintentionally discriminated against job candidates by gender for technical roles, and the company ultimately had to scrap the project. Harvard Business Review3 has raised other pointed questions about the use of AI in hiring practices, such as what data you should be able to use when evaluating a candidate for a role.
Bias and discrimination aren’t limited to the human resources function either; they can be found in a number of applications from facial recognition software to social media algorithms.
As businesses become more aware of the risks with AI, they’ve also become more active in this discussion around AI ethics and values. For example, IBM has sunset its general purpose facial recognition and analysis products. IBM CEO Arvind Krishna wrote: “IBM firmly opposes and will not condone uses of any technology, including facial recognition technology offered by other vendors, for mass surveillance, racial profiling, violations of basic human rights and freedoms, or any purpose which is not consistent with our values and Principles of Trust and Transparency.”
Accountability
Since there isn’t significant legislation to regulate AI practices, there is no real enforcement mechanism to ensure that ethical AI is practiced. The current incentives for companies to be ethical are the negative repercussions of an unethical AI system on the bottom line. To fill the gap, ethical frameworks have emerged as part of a collaboration between ethicists and researchers to govern the construction and distribution of AI models within society. However, at the moment, these only serve to guide. Some research shows that the combination of distributed responsibility and a lack of foresight into potential consequences aren’t conducive to preventing harm to society.
How to choose the right AI platform for machine learning
Selecting a platform can be a challenging process, as the wrong system can drive up costs, or limit the use of other valuable tools or technologies. When reviewing multiple vendors to select an AI platform, there is often a tendency to think that more features = a better system. Maybe so, but reviewers should start by thinking through what the AI platform will be doing for their organization. What machine learning capabilities need to be delivered and what features are important to accomplish them? One missing feature might doom the usefulness of an entire system. Here are some features to consider.
- MLOps capabilities. Does the system have:
- a unified interface for ease of management?
- automated machine learning tools for faster model creation with low-code and no-code functionality?
- decision optimization to streamline the selection and deployment of optimization models?
- visual modeling to combine visual data science with open-source libraries and notebook-based interfaces on a unified data and AI studio?
- automated development for beginners to get started quickly and more advanced data scientists to experiment?
- synthetic data generator as an alternative or supplement to real-world data when real-world data is not readily available?
- Generative AI capabilities. Does the system have:
- a content generator that can generate text, images and other content based on the data it was trained on?
- automated classification to read and classify written input, such as evaluating and sorting customer complaints or reviewing customer feedback sentiment?
- a summary generator that can transform dense text into a high-quality summary, capture key points from financial reports, and generate meeting transcriptions?
- a data extraction capability to sort through complex details and quickly pull the necessary information from large documents?
Register for this course: Enrol Now