RecSys’14: Beyond Clicks: Dwell Time for Personalization
Recommendation system paper challenge (6/50)
Why this paper?
RecSys’14 and top popular recommendation system paper.
What problem do they solve?
Recommendation system with dwell time and figure out dwell time is the proxy of long term user satisfaction.
What will I solve this problem?
If I have dwell time information, the naive idea it predict dwell time instead of predict click. Similar with this Youtube paper, they predict watch time.
Another idea is treating dwell time as a feature.
What others consider long-term user value?
Most researchers consider rating and click. There are few rating and there are many noise click.
What is the baseline models?
Click as Target: (GBDT) Gradient Boosted Decision Tree as model and click as target.
What model do they propose?
Dwell Time as Target: With same model setting, their target is dwell time instead of click.
Dwell Time as Weight: With same model setting, they put dwell time as instance weight. For more dwell time click, they give more weights.
Benefits:
- It can improve CTR even not optimize the click in loss function
- It gains higher user engagement.
Experiments they run
Offline experiment
Dwell Time as Wight perform best.
Online experiment
A: linear model
B: GBDT model (Y : click)
C: GBDT model (Y : dwell time)
Not only CTR but also user engagement, C model performs best.
how dwell time can be used in CF frameworks to better improve the performance?
They consider pair-wise objective function.
Baseline model: treat r is click [0, 1]
Their model: treat r is normalized dwell time [0, 6]
Metric
MAP: mean of average precision over all user session
NDCG:
Here is an example:
If the recommendations order is [2, 3, 3, 1, 2]
while the ideal order is [3,3,2,2,1]
for more detail: you can follow this blog.
CONCLUSIONS
This paper does not talk about new model. They address the importance of dwell time and how to simply enable dwell time on existing modeling.
The top popular and simple models are learning to rank and collaborative filtering. Dwell time based model performs better than traditional model setting on both model.
Dwell time can not only improve CTR (making money for business) but also improve user engagement (making user happy).
P.S.
This paper also talks about how to measure dwell time for different device and how to normalize. Client-Side Dwell Time is usually more precise than Server-Side.
Other related blogs:
Trust-aware recommender systems
Performance of recommender algorithms on top-n recommendation tasks
A Matrix Factorization Technique with Trust Propagation for Recommendation in Social Networks
Hidden Factors and Hidden Topics: Understanding Rating Dimensions with Review Text
Best paper in RecSys:
https://recsys.acm.org/best-papers/
My Website: