Machine learning

Machine learning (ML) is a field of study in artificial intelligence that focuses on creating statistical algorithms able to learn from data and apply that learning to new, unseen situations. Instead of being explicitly programmed for specific tasks, these algorithms improve over time as they process more information.

A key area within machine learning is deep learning, which uses neural networks — special types of algorithms — to achieve better performance than older methods. These advances have led to significant improvements in many technologies.

The foundations of machine learning rest on statistics and mathematical optimisation. It is closely related to data mining, which focuses on discovering patterns in data through techniques like unsupervised learning. Together, these fields help computers make sense of large amounts of information and support many modern tools and services we use every day.

History

Relationships to other fields

Artificial intelligence

Machine learning grew from the idea of creating artificial intelligence (AI). Early AI researchers wanted machines to learn from data, trying methods like neural networks and probabilistic reasoning. But by the 1980s, AI focused more on expert systems, leaving machine learning to develop separately. Machine learning became its own field in the 1990s, focusing on practical problems using methods from statistics and probability theory.

Data compression

Data mining

Machine learning and data mining use similar methods but have different goals. Machine learning aims to predict based on known information, while data mining looks for new patterns in data. Both fields help each other, with machine learning using data mining techniques to improve accuracy.

Generalization

Statistics

Machine learning and statistics are related but different. Statistics makes conclusions from samples, while machine learning finds patterns that can predict new data. Unlike traditional statistics, machine learning lets the data shape the model, using many variables to improve accuracy.

Statistical physics

Techniques from statistical physics are being used to study large machine learning problems, such as understanding deep neural networks. This helps improve areas like medical diagnostics.

Theory

Main articles: Computational learning theory and Statistical learning theory

Machine learning aims to help computers learn from data so they can make good guesses about new information they haven't seen before. This is called generalisation. Since we can't know everything ahead of time, scientists study how well these learning methods work using probability and math.

One important idea is making sure the computer’s model isn’t too simple or too complex. If it’s too simple, it might not catch important patterns. If it’s too complex, it might only remember the data it was trained on and not work well with new data. Researchers also study how fast these learning methods can work and what kinds of problems they can solve efficiently.

Approaches

Machine learning is a way for computers to learn from data. There are three main ways this learning happens:

In supervised learning, the training data is labelled with the expected answers, while in unsupervised learning, the model identifies patterns or structures in unlabelled data.

Supervised learning: The computer gets examples with answers, like learning from a teacher. It tries to find rules that match the answers to the examples.
Unsupervised learning: The computer finds patterns in data without any answers given. It looks for groups or structures in the information.
Reinforcement learning: The computer learns by trying things and getting rewards or penalties, like playing a game where it tries to win.

Each method has its own strengths and is used for different kinds of problems. For example, supervised learning is good for predicting things like weather, while reinforcement learning can help a robot learn to move.

Models

A machine learning model is a type of mathematical model that can make predictions or classifications on new data after being "trained" on a dataset. During training, a learning algorithm adjusts the model's internal settings to reduce errors in its predictions.

There are many types of models used in machine learning. One common type is called an artificial neural network, which is inspired by how animal brains work. These networks learn by processing examples without specific programming rules. Other models include decision trees, which use branches to make decisions, and support-vector machines, which sort data into categories. Each model has its own strengths and is chosen based on the task it needs to perform.

Applications

Machine learning has many uses. It helps computers learn from data so they can make decisions or predictions. Some areas where machine learning is used include agriculture, banking, healthcare, and even space science.

People have used machine learning in fun ways too. For example, a contest was held to improve movie suggestions, and machine learning has helped study art and even fight diseases like COVID-19. It can also help predict weather patterns and make smartphones work better.

Limitations

Machine learning can sometimes fail to work as expected. This can happen for many reasons, such as not having enough good data, privacy issues, or choosing the wrong tools. One big challenge is that some machine learning systems can be like a "black box," meaning even the people who made them can't always understand how the system makes its decisions.

The blue line could be an example of overfitting a linear function due to random noise.

There are also problems like overfitting, where a system learns the training data too well and doesn't work well with new data. Sometimes, machine learning can also be tricked into making mistakes by small changes to images or data. These challenges show that while machine learning is powerful, it still has limits and needs careful use.

Main article: Explainable artificial intelligence

Main article: Overfitting

Model assessments

To check how well a machine learning model works, we can use different methods. One way is the holdout method, where we split the data into two parts: most of it for training the model and a smaller part to test it. Another method is K-fold cross-validation, where we divide the data into several small groups and use each group, one at a time, to test the model.

We also look at how often the model is correct or makes mistakes. Tools like the Receiver Operating Characteristic (ROC) help us understand how good a model is, with higher values meaning better performance.

Ethics

Bias

Main article: Algorithmic bias

Machine learning can sometimes show biases because of the data it is trained on. If the data includes human biases, the machine learning system might repeat those biases. For example, a program used for college admissions once unfairly treated people with non-European-sounding names. Similarly, some systems have mislabeled people from certain backgrounds.

Because of these issues, many scientists are working to make machine learning fair and helpful for everyone. They believe it is important to include people from all backgrounds in developing these technologies.

Hardware

Since the 2010s, better computer chips and software have made it easier to teach computers to learn by themselves. Big projects now use special chips called GPUs to handle the tough work. These chips help computers learn faster and better.

There are also special chips called Tensor Processing Units (TPUs) made by Google. These TPUs are designed just for teaching computers and work very well for big projects. Some computers are built to work more like our brains, which helps them learn in new ways. Machines can also learn right on small devices like watches or phones, keeping information safe without needing to send it far away.

Software

Machine learning uses special computer programs to help computers learn from data. There are many different tools and programs that people use for this. Some of these tools are free for anyone to use and change, while others are made by companies and might cost money.

Here are some of the popular free tools:

Some tools are made by companies but also have free versions:

And here are some tools made by companies that you need to buy:

Journals

Some important journals where researchers share their work on machine learning include the Journal of Machine Learning Research, Machine Learning, Nature Machine Intelligence, Neural Computation, and IEEE Transactions on Pattern Analysis and Machine Intelligence. These journals help scientists keep up with the latest discoveries and ideas in this exciting field.

Conferences

Some important conferences where researchers share their latest ideas about machine learning include the AAAI Conference on Artificial Intelligence, the Association for Computational Linguistics (ACL), and the International Conference on Machine Learning (ICML). Others, like the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), the Conference on Neural Information Processing Systems (NeurIPS), and the Conference on Knowledge Discovery and Data Mining (KDD), also bring together experts to discuss new discoveries and tools in this exciting field.