What NOT to do with data when working with deep neural networks
Introduction
In these days of deep neural networks aplenty, it has become extremely common for teams to encounter the term deep learning and AI much more often than classical machine learning models names. It is indeed exciting to know that one can simply
On the one hand, deep learning pundits claim that such models do extremely well with “large” datasets. On the other hand, users are often confused with such claims and are often frustrated with the poor performance when dealing with applications leveraging deep neural networks when they were promised AI inside.
Motivation
The target for this series of blog posts is to highlight that there is a mismatch in the communication that colleagues working in the deep learning field share and that the consuming application counterparts understand when using these algorithms exposed either as APIs or through other applications processing messages as requests.
Topics covered in this blog series
In this blog series, we will look at some of the common phrases that a user would encounter, how they interpret these and why this often leads to below-par results.
The following are some of the aspects that we would cover in this blog series:
- Effects of copy-pasting training data
- Effects of copy-pasting training and validation data
- Effects of time-based split of test data
- Effects of highly imbalanced categorical targets
- Effects of large variance and anomalies in numeric targets
As each and every blog post in the above list are completed, their links will be updated to make it easier for walking through each topic.
Further topics
The topics listed above are definitely not exhaustive of all the issues that users of deep learning would encounter. Are there any topics that you would add to this list?
Drop me a comment if you feel any topics have been missed out here.
