My Most Viewed Stories

Audio is an entire field on its own when it comes to machine learning, with a wide range of applications including speech recognition, music categorization, and sound event detection. …

As machine learning models continue to increase in complexity and capability, so too does the challenge of optimizing their performance. One effective technique for improving the performance of large, complex models is knowledge distillation, which involves training a smaller, more efficient model to mimic the behavior of a larger, “teacher”…

There’s often the misconception that bigger and bigger models are always necessarily better. For instance, GPT-3 is trained on 570 GB of text and consists of 175 billion parameters. However, whilst training large models helps improve state-of-the-art performance, deploying such cumbersome models (especially on edge devices) is not straightforward nor…

In Machine Learning, we typically aim to optimize for a single metric, for a single task. Multi-task learning (MTL) has led to successes in many applications of machine learning, from natural language processing and speech recognition to computer vision and drug discovery. The most famous case of MTL is probably…

The Attention is all you need’s paper revolutionized the world of Natural Language Processing and Transformer-based architecture became the de-facto standard for natural language processing tasks. It was only a matter of time before someone would actually try to reach the state of the art in Computer Vision, with attention…

Dropout is a regularization technique for neural network models proposed by Srivastava, et al. in their 2014 paper Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Dropout is a technique where randomly selected neurons are ignored during training. They are “dropped-out” randomly. This means that their contribution to…

In Deep Learning, fine-tuning hyperparameters is vital in order to ensure the model’s performance. In this tutorial, we’ll go over dropout regularization for neural networks. From the background and motivation for adopting dropout, to a description of how dropout works theoretically and how to implement it using Pytorch. Setting up For our…

In this article, we’ll briefly go over the definition and use cases of OCR (Optical Character Recognition) and learn how to leverage state of the art solutions in order to extract basically any text from an image file. Introduction OCR is a popular technology that converts any kind of text or…

“In May 2022 over 70,000 developers told us how they learn and level up, which tools they’re using, and what they want.” Here’s the survey if you want to explore it for yourself. Overview Generally speaking, learning to code online increased from 60% to 70% year over year. There’s a tendency…