In this quick overview, we introduce you to the concepts of one-versus-one and one-versus-all in classification. In classification models, you will often want to predict one class from another. This is called binary classification, or one-versus-one. But what if you have more than two classes to predict? This is where one-versus-all is introduced.
What You'll Learn
- Difference between one-versus-one and one-versus-all classification techniques
- Where to use a particular classification technique
Welcome to this short introduction to one-versus-one vs. one-versus-all in classification.
In classification models, you often want to predict one class from the other, such as predicting apples from bananas. This is binary classification, where one class is a positive class and the other class is a negative class. For example, the presence of aggressive cancer versus passive cancer, or returning customers versus new customers, or successful applicants versus non-successful applicants. Here, we have our classic one-versus-one classification. You simply predict one class from another class or one-versus-one. But what if you have more than 2 classes to predict, such as many types of cancer, or many types of customers, or different levels in a game such as easy, intermediate, and advanced?
Many classification algorithms are limited to binary classification, where it only allows one-versus-one class prediction. You need a way of being able to predict more than 2 classes. This is where one-versus-all is introduced. It treats one of the multiple classes, such as easy level in a game, and predicts against all other classes, such as intermediate and advanced levels in a game. This means the easy class is treated as the positive class, and all other classes (intermediate and advanced) are treated as the negative class - effectively turning this into binary, 2-class classification.
But there is a problem with the one-versus-all approach to multi-class classification. You often have an imbalance in the number of examples of subjects in one class versus all other classes. For example, if you have 100 easy level subjects, 100 intermediate level subjects, and 100 advanced level subjects, a one-versus-all approach says that you now have only 100 easy level subjects versus 200 (double the amount) of all other subjects. When there are less examples for a model to train on, the model usually doesn’t predict as well on those examples.
If you feed your model a ton of examples of one class and a very small number of examples of the other class - well, guess what, it’s not going to predict well on the other class. So, we go back to one-versus-one again. It might be better to do predictions of all pair combinations of classes. So, easy-versus-intermediate, and intermediate-versus-advanced, and advanced-versus-easy. This way you always have an even number of examples - 100 from one class, 100 from the other. And that sums one-versus-one vs one-versus-all in classification.
Thanks for watching. If you found this video useful, give us a like, or you can check out our other videos at online.datasciencedojo.com.
Rebecca Merrett - Rebecca holds a bachelor’s degree of information and media from the University of Technology Sydney and a post graduate diploma in mathematics and statistics from the University of Southern Queensland. She has a background in technical writing for games dev and has written for tech publications.