We are pleased to invite you to contribute to the 1st Workshop on Learning and Evaluating Recommendations with Impressions held in conjunction with the ACM International Conference on Recommender Systems (RecSys 2023) in Singapore, on September 19th 2023.
Recommender systems typically rely on past user interactions as the primary source of information for making predictions. However, observed interactions are a sparse and strongly biased signal, which has significant implications for both learning from user actions and evaluating the quality of recommendations offline
Recently, a source of information that was used in industry but was almost unavailable to the wider research community has emerged with the potential to impact the field in numerous ways: impressions. Impressions refer to the items displayed on the screen when a user interacts with some system — the end product of a recommendation engine is therefore a set of impressions (the recommendations displayed to end users), and the input data for recommendation is likewise obtained from impressions from other (and/or the same) systems. Early research on impressions was constrained by the limited availability of public datasets, but this is rapidly changing and, as a consequence, interest in leveraging impressions has increased. Impressions present new research questions and opportunities, but also bring new challenges.
The Workshop on Learning and Evaluating Recommendations with Impressions (LERI) will focus on all aspects related to leveraging impression data to build and evaluate a recommendation engine. The goal of LERI 2023 is to both help to coalesce researchers and practitioners exploring the use of impressions from different perspectives, and foster increased interest from the community for this important and underexplored topic that has the potential of impacting the field in several ways.
The workshop aims to provide a venue for researchers and practitioners to come together in order to: (i) share experience and lessons learned; (ii) identify key challenges in the area; (iii) build a common mental model and conceptual framework for thinking and researching on the use of impressions; (iv) identify emerging topics and new opportunities. The workshop also aims to lay bridges between practitioners and academics, encourage a wider availability of impression data sources and leverage industry’s experience to guide and inform academic research.
Topics
Topics of interests include, but are not limited to:
- Conceptual framework: definition of “impression”, role of impressions in the recommendation task definition, impressions in multi-stage recommendation, user action attribution to impressions, correct recommendations and causation between impressions and future user actions, bias introduced by training on the user reaction to the system’s impressions (closed vs. open feedback loops).
- Recommendation models: new learning approaches taking advantage of impression data, impression-based input features, sampling from impressions in labelled data, loss functions, model topologies, impression-based reranking.
- Model training: impressions in data pre-processing, sampling, partitioning, hyperparameter tuning with impressions.
- Evaluation: evaluation methodology and metrics, impact on offline evaluation bias (challenges and opportunities).
- User modelling: new models considering user behaviour in face of impressed items.
- Reinforcement learning: new recommendation models based of reinforcement learning leveraging impressions.
- Off-policy estimation: design of OPE techniques leveraging impressions.
- Datasets: collection of new datasets with impressions from different domains, user interfaces, applications.
- Theory: theoretical aspects in the use of impressions for recommender systems, both in the development of new and improved recommender systems and in their evaluation.
- User studies: how the user behaviour is impacted by the composition of impressions, impact of user fatigue, etc.
- Perspectives: new perspectives on existing problems that could benefit or just change by adding impressions as a new variable, as well as old challenges that can be now tackled from new angles, and new challenges that derive from the use of impressions.
Submissions
Submissions of full research papers must be in English, in PDF format in the CEUR-WS two-column conference format available at http://ceur-ws.org/Vol-XXX/CEURART.zip or as an Overleaf template.
Submission will be peer reviewed and accepted papers will appear in the CEUR workshop series (at authors’ discretion). We also welcome contributions from the industry and papers describing ongoing projects. At least one author of the accepted papers must be registered to the workshop.
Authors are invited to submit one of the following types of contributions:
- Long Papers should report on substantial contributions of lasting value (10 pages plus additional pages for references if needed).
- Short Papers typically discuss exciting new work that is not yet mature enough for a long paper (5 pages plus additional pages for references if needed).
The review process is double-blind. Submitted papers will be evaluated according to their originality, technical content, style, clarity, and relevance to the workshop. We are also planning to invite the authors of published articles to submit an extended version of their work to a journal special issue (TBD).
Submission will be through CMT at this link https://cmt3.research.microsoft.com/LERI2023.
Important Dates
- Paper submission deadline:
August 3rd, 2023extended to August 10th, 2023 - Author notification: August 28th, 2023
- Camera-ready version deadline: September 10th, 2023
Deadlines refer to 23:59 (11:59pm) in the AoE (Anywhere on Earth) time zone.
Contacts
If you have questions you can contact us at maurizio [dot] ferrari [at] polimi [dot] it
