Try MIXUP ! (With implementation)

We know the power that lies in deep neural networks with the
excessive parameterization but with deep nets , we also have to
solve the problem of overfitting mainly due to the networks
memorizing the entire image dataset and not learning much.

Solution

We have plenty of ways to alleviate this issue , one could be the use of
augmentations like randomly cropping the images, resizing them,
changing hue, intensity etc , I mean there’s a whole library to it !

But you do need to keep in mind that we cannot blindly use any
augmentations on all datasets and you have to be mindful of what
you use which requires domain knowledge and plenty of experimentations.

The rise of Mixup

Can you see where I’m headed yet? Mixup overcomes the prior
shortcomings by combining different samples of images and labels
to promote a linear relationship among the images , which can act as
a regularizer. Linear relationship is the most simplest relationship that can exist which also ticks Occam’s Razor. This means that you can simply plug and play into any datasets without having to worry about model performance and
most probably will give you much better results.

Formula

mixed_image = λ * image_i1 + (1 − λ) * image_i2
mixed_labels = λ * label_j1 + (1 − λ) * label_j2
where λ must be sampled between 0 and 1

Looks so simple yet so effective, let’s try it first hand for ourselves.

Implementation

Sheeesh, two G.O.A.T.S in one image, I hope this was informative and do let me know how your experiments go!