Silver Medal Solution on Kaggle H&M Personalized Fashion Recommendations

The Competition

The competition was hosted by H&M Group on Kaggle. I joined the competition with my colleague from work for fun. We wanted to learn about the recommendation system and agreed that the best way for it is by working on an exciting project like this competition.

our final score

The Task

H&M’s online store offers shoppers an extensive selection of products to browse through. But with too many choices, customers might not quickly find what interests them or what they are looking for, and ultimately, they might not make a purchase. To enhance the shopping experience, product recommendations are essential.

Evaluation Metric

The top 12 predictions will be compared to products that customers actually bought in the next 7 days. The MAP@12 metrics is one of evaluation metric in ranking problem. One way to describe it is as follows “MAP is a metric that tells you how much of the relevant documents (products, in this case) are concentrated in the highest-ranked predictions.” In simple words, if we can place products that were actually bought in the highest ranking, the better the score.

The Data

Dataset

The competition was rather different than previous Kaggle competitions in which we were given not-ready-to-train datasets. We need to come up with pipeline design and preprocess the data before we call model.fit() method. It made the competition setup closer to real-world data science project setup.

Pattern

The datasets consist of 106k products and 1.37m customers. If we included all products as candidates per customer, we’ll have 106k X 1.37mn =145 billion rows of data, not to mention the size of the features. We need to find a better way to reduce the data or we can’t fit the data into RAM.

The Solution

Our solution was inspired by famous two steps architecture: (1) Candidate Retrieval and (2) Ranking. Candidate Retrieval focuses more on recall means that it aims to filter 106k products to 30 that are relevant for customers. Ranking focuses more on precision means that it aims to correct the order of 30 candidates and select only top 12 products.

Recommendation system architecture demonstrating the “funnel” where candidate videos are retrieved and ranked before presenting only a few to the user. Taken from YouTube Recommendation paper.

Candidate Retrieval (Recall)

Cold-start problem
Cold-start problem is a problem where new customer coming and we don’t have any information about them. We used 12 popular products from last 1 week to address this problem.

Ranking (Precision)

From start, we decided to develop ranking models separately. We aimed to have diverse models so that later we could ensemble the predictions. Our approach was different even from how we prepared the training data.

Feature engineering
Overall architecture of our solution

Validation

It’s very important to set up robust validation strategy for Kaggle competition. We used last 7 days in the training data as our validation set. It would be better if we have > 1 fold validation set, but with computation and time constraints we decided to only have 1 fold validation set. We observed with only 1 fold validation set is still giving us good correlation with leaderboard score.

validation strategy

Closing

The competition taught us a lot. Special thanks to my teammate, Hervind Philipe, we earned a silver medal and earned learnings implementing machine learning solutions that revolve around a recommendation system. We hope this post could be a useful reference for anyone who wants to implement similar projects in the future. Thanks for reading all along!

Reference

  1. Covington, Paul. et al, Deep Neural Networks for YouTube Recommendations. link
  2. Eugene Yan, System Design for Recommendations and Search. link
  3. Product launched, relevant articles & customers link

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store