The interest in in-the-wild studies comes from the growing demand for the development of technologies that aim to improve or facilitate certain aspects of everyday life. For that, researchers explore these technologies situated in real and messy environments with several inhabitants and visitors 48. These studies investigate technological possibilities that can modify or create specific behaviors.
In this context, the in-the-wild approach alters the traditional mental design model, focused primarily on designing a technological solution that adheres to existing practices 47.Previous ethnographic and ethnomethodological approaches applied to Interaction Design primarily focus on obtaining insights from analyzing and describing current situations, lacking the ability to offer probes and suggestions for a possible future 28. On the other hand, approaches such as probes and breaching experiments have been used to explore beyond current practices, enabling recommendations for future investigations 5. Cultural Probes, for example, were originally used to research the potential gaps in a technology intervention in communities that are unfamiliar with these technologies, without constraining the designers’ creativity or imposing designs by focusing on users’ needs 7.
By disrupting current practices, researchers can examine behavior changes over periods of time, obtain insights and infer hypotheses and future design ideas for that context 45. However, these insights, hypotheses, and ideas are often unexplored because the prototype is not robust and flexible enough to allow for rapid modifications. Thus, these studies may leave open questions and have a brief understanding of the impact of the design in the behavior change, which may affect also the reliability of the inferences. Exploring diverse designs with a prototype can reveal more about the impact of the design on its use or adoption. Furthermore, it is difficult to predict people’s reactions when first interacting with novel technologies 47.
In this context, evaluating them in the wild is valuable and allows researchers to comprehend how people understand and use the technologies, and appropriate them on their own terms and for their own situated purposes 20. The in-the-wild approach is also highly adaptable to different research contexts, methods, technologies, or theories. Thus, depending on the context and goals of the research the approach will vary wildly 15. Nonetheless, this agnostic view attached to the in-the-wild research has influenced the increasing number of studies without an explicit methodological foundation, which is very problematic. This assumption that methodology matters very little clearly affect the results and consequently the reliability of these studies 23. In addition, researchers often forget to document and describe their reasoning, which makes it difficult to understand the choices made 41, 36. This fact also contradicts the formal scientific paradigm, where there is a protocol to follow, describing the procedures and the reasoning behind the experiment and hypothesis generation. 15.
To mitigate that, we present in this paper a methodology, called on-the-fly prototyping, that allows to the researchers to document their procedures and reasoning, and evaluate as many insights or design variations as possible during the investigation and improve the prototype as the researchers learn from the evaluation in situ.Evolving a Prototype in the WildMany empirical studies incorporate the strategy of evolving situated prototypes without formalizing it 3, 35, 44, 39, 25, 16. These studies generally involve deploying evolutionary prototypes in the wild extended by participatory design or co-design approaches. In this section, we describe some of these studies, demonstrating the main differences with the proposal of the on-the-fly prototyping approach. As technologies progress steadily and user requirements change, the evolution of systems is inevitable. Thus, according to Dittrich et al. 44, the development of applications should encompass the use, maintenance, tailoring, adaptation, and further development. For that, they propose a model, called “design-during-use”, in which a system is updated according to the explicit requests of end users or changes in the system (i.
e., adaptation to new platforms and corrections). This practice of improving the design of a solution according to direct requests from users is related to participatory design. In contrast, on-the-fly prototyping focuses on learning from the experience with prototype and its design variations. This practice has a provocative nature 15 by not limiting the designers’ creativity, which can be disruptive to previously established practices or mental models. However, a balance between contextual appropriateness and provocation is necessary. In the context of ‘speculative’ design in design fiction, Aurger 21 argues that if a design proposal is easily assimilated into the normative evolution of products will pass unseen, but also if it is too unfamiliar can produce a ‘revulsion or outright shock.
‘ Moreover, researchers also must manage their goals by considering the community’s needs.To make the system evolution more collaborative, Botero and Hyysalo 3 used the concept of co-design to expand the evolutionary design. Thus, they evolved the prototype as they learned from the collected data and engaged the users to collaborate in a long-term process of prototype evolution. Likewise, Churchill et al.
11 describe a longitudinal case study with interactive public displays, called “plasma posters.” For 6 months, they collected quantitative data from system logs and qualitative data from eventual observational studies. Based on the collected data and using participatory design through interviews and email surveys, they iteratively designed the plasma poster interface, aiming at improving the prototype design and encouraging social contact. Unlike these studies, on-the-fly prototyping methodology embraces the concept of Rapid Ethnography 9 concept by narrowing the focus of the study, which shrinks the iterative design cycle of the prototype and reduces the demands on resources compared to long-term studies.Most similar studies propose an iterative development or a refined design of the prototype (i.e.
, spiral lifecycle model), which may discourage to test and evaluate other design variations, leading to a brief understanding about the impact of the design on its use or adoption. Summarizing, the on-the-fly prototyping methodology differs from others 3, 35, 44, 39, 25, 16, 43 by encouraging designers to implement, explore and evaluate different settings and scenarios, shrinking the iteration cycle of the prototype and enabling adjustments and changes on the prototype in real time. This approach also encourages the re-evaluation of designs, improves the documentation of the procedures and design rationale, and provides a model for tracing the results of the changes in the prototype. Thus, the on-the-fly prototyping approach evolves the in-the-wild approach by providing a model for designing, deploying, evaluating, and evolving the prototype to improve its design.
On-the-fly ParadigmAccording to the Cambridge dictionary 8, ‘on the fly’ has two meanings. The first means to do something quickly, often in parallel with something else, without overthinking on how it should be done. The second refers to the computer science context and means “without interrupting a computer program that is already running”, which is related to the concept of ‘On-the-fly Programming’.On-the-fly Programming is a style of programming in which the programmer actively modifies the program at runtime, without stopping or restarting it 14. This concept offers much potential for performers and composers. They can use this approach to transform code into actions that change the active modules in their synthesis or composition programs or modify the mappings to their controllers during live musical performances 14, 38.
This live coding is also used in visual and artistic 30 practices, and has inspired other practices such as live hardware hacking 13, live composing 22, and on-the-fly print 17, supporting improvisation and immersion. Although this programming approach can be explored in myriad ways, it is still very challenging.In this paper, we present a methodology called ‘on-the-fly prototyping’ based on a literature review and reflections on our previous experiments. This methodology focuses on the practice of exploring and evaluating several designs with a prototype by evolving it in situ. For that, the prototype design needs to support anticipated and unforeseen modifications, in an orderly and managed evolution. The variations in the installation characteristics include the software and other externalities related to the prototype. This approach is intended mainly for systems for dynamic and complex environments, such as public spaces. Since interactive systems are becoming a critical component of the urban experience, they should be able to adapt as quickly as the environment and requirements changes.
Public spaces are complex, dynamic, and messy by nature, and changes are imminent. Extending the Research in the Wild FrameworkRecently, Rogers and Marshall 45 introduced a framework for understanding in-the-wild research projects. This framework consists of four interrelated cores: Theory, In-Situ, Design, and Technology. In a top-down perspective, this is useful to scope the project during the planning. However, due to lack of an ordering, this framework was adapted in 15 by adding markers in the edges and refocusing the framework on the design core. Considering this last proposal, we extended this framework by adding a new edge (i.
e., the edge D) on the design core to represent the evolution of the design and the beginning of a new cycle of the framework, as seen in Figure 1. Figure 1. Extended framework from the RITW framework 15In a project level, the researchers can consider the four cores. On the other hand, to have a more granular perspective, researchers can move around the structure, analyzing the nature of the influence of the cores changes through the edges.
The edge A considers that introducing novel technology will impact the community in somehow. The edge B reflects about how the theories that shape the research goals and how they apply to the intervention. The edge C encompasses the field inputs or contextual information from the previous observations in order to solve something in that context. The edge D demonstrates the need of reflecting on the lessons learned and the beginning of a new design cycle.
These edge motivations are not necessarily orthogonal because the researchers should also consider the project trade-offs such as ‘community-led’ concerns and ‘designer-led’ aspirations.