What is a Learning Curve in machine learning?

I want to know what a learning curve in machine learning is. What is the standard way of plotting it? I mean what should be the x and y-axis of my plot.

In Andrew’s machine learning class, a learning curve is the plot of the training/cross-validation error versus the sample size. The learning curve can be used to detect whether the model has a high bias or high variance. If the model suffers from a high bias problem, as the sample size increases, training error will increase and the cross-validation error will decrease and at last, they will be very close to each other but still at a high error rate for both training and classification error. And increasing the sample size will not help much for the high bias problem.

A machine learning curve allows you to find the point from which the algorithm starts to learn. If you take a curve and then slice a slope tangent for derivative at the point that it starts to reach constantly is when it starts to build its learning ability.

Again, It usually refers to a plot of the prediction accuracy/error vs. the training set size (i.e: how better does the model get at predicting the target as you the increasing number of instances used to train it)

Learning Curve in machine learning

Usually both the training and test/validation performance are plotted together so we can diagnose the bias-variance tradeoff (i.e determine if we benefit from adding more training data, and assess the model complexity by controlling regularization or number of features).

Depending on how your x and y axis are mapped, one of your axes will start to approach a constant value while the other axis’s values will keep increasing. This is when you start seeing some learning. The whole curve pretty much allows you to measure the rate at which your algorithm can learn. The maximum point is usually when the slope starts to recede. You can take several derivative measures to the maximum/minimum point.
So from the above examples, you can see that the curve is gradually tending towards a constant value. It initially starts to harness its learning through the training examples and the slope widens at maximum/minimum point where it tends to approach closer and closer towards the constant state. At this point, it can pick up new examples from test data and find new and unique results from data. You would have such x/y axis measures for epochs vs error.

Is learning curve and ROC curve are synonymous?

 The learning curve and ROC curve are not synonymous.

As indicated in the other answers to this question, a learning curve conventionally depicts improvement in performance on the vertical axis when there are changes in another parameter (on the horizontal axis), such as training set size (in machine learning) or iteration/time (in both machine and biological learning). One salient point is that many parameters of the model are changing at different points on the plot. Other answers here have done a great job of illustrating learning curves.

(There is also another meaning of the learning curve in industrial manufacturing, originating in an observation in the 1930s that the number of labor hours needed to produce an individual unit decreases at a uniform rate as the number of units manufactured doubles. It isn’t really relevant but is worth noting for completeness and to avoid confusion in web searches.)

In contrast, the Receiver Operating Characteristic curve, or the ROC curve, does not show learning; it shows performance. A ROC curve is a graphical depiction of classifier performance that shows the trade-off between increasing true positive rates (on the vertical axis) and increasing false positive rates (on the horizontal axis) as the discrimination threshold of the classifier is varied. Thus, only a single parameter (the decision/discrimination threshold) associated with the model is changing at different points on the plot. This ROC curve (from Wikipedia) shows the performance of three different classifiers.

ROC curve, see previous link for CC licensing

There is no learning being depicted here, but rather performance with respect to two different classes of success/error as the classifier’s decision threshold is made more lenient/strict. By looking at the area under the curve, we can see an overall indication of the ability of the classifier to distinguish the classes. This area-under-the-curve metric is insensitive to the number of members in the two classes, so it may not reflect actual performance if class membership is unbalanced

Similar Posts