Living Labs - An Ethical Challenge for Researchers and Platform Providers

The infamous Facebook emotion contagion experiment is one of the most prominent and best-known online experiments based on the concept of what we here call "living labs". In these kinds of experiments, real-world applications such as social web platforms trigger experimental switches inside their system to present experimental changes to their users - most of the time without the users being aware of their role as virtual guinea pigs. In the Facebook example the researches changed the way users' personal timeline was compiled to test the influence on the users' moods and feelings. The reactions to these experiments showed the inherent ethical issues such living labs settings bring up, mainly the study's lack of informed consent procedures, as well as a more general critique of the flaws in the experimental design. In this chapter, we describe additional use cases: The so-called living labs that focus on experimentation with information systems such as search engines and wikis and especially on their real-world usage. The living labs paradigm allows researchers to conduct research in real-world environments or systems. In the field of information science and especially information retrieval - which is the scientific discipline that is concerned with the research of search engines, information systems, and search related algorithms and techniques - it is still common practice to perform in vitro or offline evaluations using static test collections. Living labs are widely unknown or unavailable to academic researchers in these fields. A main benefit of living labs is their potential to offer new ways and possibilities to experiment with information systems and especially their users, but on the other hand they introduce a whole set of ethical issues that we would like to address in this chapter.