code
share




6.6 Which Approach Produces the Best Results?*




* The following is part of an early draft of the second edition of Machine Learning Refined. The published text (with revised material) is now available on Amazon as well as other major book retailers. Instructors may request an examination copy from Cambridge University Press.

We have now seen three approaches, logistic regression, the perceptron, and Support Vector Machines, each of brings a valuable and unique perspective to bear on two class classification. Which of these approaches 'works best' in practice? As we have seen, in the end all three approaches are very tightly related and result in the minimization of the same (e.g., the Softmax cost) - or very similar - cost functions. Therefore it should come as little surprise that while they can differ from dataset to dataset in terms of their performance, in practice all three approaches typically give compariable results. Thus one should feel comfortable using any one of the three approaches or, if resources allow, apply multiple of them and keep whichever produces the best performance on a case by case basis.