Pair: An Image-based
Product Recommender

Mike Lo

Online shopping is hard!

We have so many choices. How can we find matching products in a personalized and timely manner?

That's why I built Pair:

It's a scalable, AI-powered recommender system for suggesting stylistically similar furniture.

Live at bit.ly/pair-app

Inference
Pipeline

## Model evaluation <br>with <u>Hit Rate @ N</u> -- Use <u>interior design scenes</u> with different furniture to evaluate recommendation performance <img class="plain" style="background: rgba(255,255,255,0.1);" width="700" height="100%" data-src="static/imgs/ikea-display-all-your-memorabilia-and-more__1364319313408-s4.jpg" alt="Down arrow"> Note: ideally would perform A/B testing. Assume furniture that are in the same scene are considered "good" matches. -- <img class="plain" style="background: rgba(255,255,255,0.1);" width="1200" height="100%" data-src="static/imgs/pair-medium-img7.png" alt="Down arrow"> Note: I used a metric called HR@n, which measures how closely Pair matches different furniture in the same room scene. So hit rate +1 every time a top-n recommendation matches an item in a room. For example, if the brown chair (green) returns the coffee table in the same room as one of the top-5 choices (red), then that’s a hit at 5 -- In <u>30%</u> of the cases, Pair achieved HR@5 when choosing from 100+ furniture choices -- <h3>Sample result</h3> <img class="plain" style="background: rgba(255,255,255,0.1);" width="1200" height="100%" data-src="static/imgs/pair-medium-img6.png" alt="Down arrow"> -- <h3>Sample result</h3> <img class="plain" style="background: rgba(255,255,255,0.1);" width="1200" height="100%" data-src="static/imgs/pair-medium-img8.png" alt="Down arrow"> --- <h2>Deployment<br>Architecture</h2> -- <h3>Workflow</h3> <ol> <li class="fragment fade-in-then-semi-out">Built as a <u>Tensorflow/Keras</u> model</li> <li class="fragment fade-in-then-semi-out">Packaged as a <u>Docker container</u></li> <li class="fragment fade-in-then-semi-out">Deployed as a <u>Kubernetes cluster</u> via GKE</li> <li class="fragment fade-in-then-semi-out">Served as a <u>Streamlit</u> application</li> </ol> -- <img class="plain" style="background: rgba(255,255,255,0.1);" width="600" height="100%" data-src="static/imgs/pair-medium-img3.png" alt="Down arrow"> --- ## Future directions <ol> <li class="fragment fade-in-then-semi-out">Solve cold start problem with user metadata</li> <li class="fragment fade-in-then-semi-out">Transfer learning to improve model performance</li> <ol> --- ## Thank you! Note: This is just a short snippet of what I can do in a short amount of time >> I am a quick learner >> Passionate about Zoom's product and what you guys are working on

Other Projects

COVID-19 Case Tracker

Created an interactive dashboard to visualize the latest COVID-19 numbers, built using Python, Kubernetes, and Streamlit.

Live at bit.ly/cov19-tracker

A data tour of Los Angeles

Performed K-means clustering of LA neighborhoods using features engineered from public data on crime rates, rental prices, and venues.

Read more at bit.ly/la-data-tour

Image processing with denoising CNN

Trained a denoising CNN to process noisy experimental projection images in electron tomography.

Read more at bit.ly/dncnn-em

Appendix

Gram Matrix