Yet another NLP example, big law firm carrying out economic investigations and risk management. As a part of their operations they conduct candidate screening on executive level in search of frauds or misconducts that could greatly damage a company if such candidate is hired. Our task was challenging on multiple levels. We had to build a machine learning model to assist analyst's assessments. It was supposed to work fast on noisy sensitive data and generate a ranking of significant articles. At that point it was all coming down to choosing a right metric. We chose ndcg score to pick interesting articles but another problem appeared. It was imperative for the client not to miss any detail and our model was ranking most of top spots with one story if a given person was a involved in a big scandal. The analysts were becoming less focused while scrolling and sustaining their attention was also essential objective to fulfill. So we started grouping similar articles into clusters trying out dozens of embeddings but the client also didn't find it facilitating. At that time we had realized that perhaps our solution is not in tune with client's problem. We sent him a sample batch of articles to see how the client groups them himself. Indeed, we got it all wrong. Instead of adjustable groups the client wanted fixed labels such as 'legal cases' or 'PR scandals', that's it. The real problem was much easier to solve than we assumed.

If you're learning data science imagine that you're a client. Try to solve a problem for yourself instead of just picking a data set. This a good way to go through the entire process of collecting data, posing questions, searching for answers and proper machine learning algorithms, verifying solutions and proving performance of a model that you've built. Build a recommendation system for yourself basing on your interests.