XR-GRACE - eXtended Reality for GRounding beauty on sustainable human and Artificial Creative Experiences

Scientific coordinator: Chiara Lucifora, Claudia Scorolli

SHORT DESCRIPTION OF THE RESEARCH

Many cognitive theories still derive their assumptions from information processing theory: perception corresponds to input, action to output; central functions are referred to cognition. Recent neuroscientific data converge instead to show that our actions are planned in terms of the distal effects they are supposed to produce, and that motor planning and perception share common neural resources (perception-action coupling).

Starting from this evidence, the group's research goes further, testing theories of situated cognition in three relatively unexplored areas:

• human and artificial creativity, which can be deployed in the “coupling” between aesthetic experience and creative practice. The enjoyment and production of art is indeed an embodied practice, firmly based on perception and action. In this perspective, the use of “artificial” VR environments that enable the simulation of both aesthetic experience (by incrementally modulating its multisensoriality) and creative acts, also in cooperation with artificial agents (capable of stimulating and supporting creativity) proves particularly promising;
• sustainable beauty in order to explore whether the aesthetics of everyday objects, conveyed in particular by materials, are capable to change our disposition to actions, individually and collectively. We know that visuo-motor representations of manipulable objects are influenced by their size, shape and orientation; however, affordance research lacks an analysis of the possible effects of materials and object pleasantness;
• ownership, with the aim of addressing an unexplored cognitive problem, namely, how do I know that something is Mine or Yours. Specifically, we hypothesize that the conceptual domain of ownership is grounded in the experience of control over external objects and processes of self/other discrimination, and develops as a byproduct of the restrictions experienced during curiosity-driven exploration in physical and social environments.

The collected data are behavioral and kinematic measures to investigate free exploration of (real and virtual) objects to test the emergence of an abstract form of (nonperceptual) categorization based on the physical controllability of objects. In addition, implicit measures are used to index cognitive and emotional engagement with the object/artwork. Finally, the production of linguistic labels is informative of the reinterpretation of meanings that follows interaction with the object and/or work.

The integration of cognitive science methods with AI's own methods of analysis makes it possible to study perception in relation to associated action, and vice versa (common code), highlighting shared mechanisms, as well as proposing explicit computational mechanisms underlying curiosity-driven exploration and intrinsically-motivated reinforcement learning.

PROJECTS AND FUNDING

Dottorato PON green MASS - The Museum as lab of reinvention: for a new Art, Shared and Sustainable

AlmaIdea BONSAI - Grounding Beauty ON Sustainable ActIons

PRIN 2022 Understanding WHat Is Mine (WHIM): theory, models, experiments on the sense of ownership over objects

PNRR FAIR - Spoke 8 PERVASIVE AI  PACE - Perception-action coupling in the domain of human and Artificial Creativity: from aesthetic Experience to artistic practice

PRIN 2022 SACre-D. Schizophrenia, Autism and the Myth of Creativity. An Interdisciplinary Perspective on Psychopathological Expression and its Digitalization

H2020 SPICE Social cohesion, Participation, and Inclusion through Cultural Engagement

ERC Sector

• PE6_7 Artificial intelligence, intelligent systems, natural language processing
• PE6_9 Human computer interaction and interface, visualization
• SH4_2 Personality and social cognition; emotion
• SH4_4 Neurocognitive psychology
• SH4_5 Attention, perception, action, consciousness
• SH5_9 Metaphysics, philosophical anthropology; aesthetics
• SH7_5 Sustainability sciences, environment and resources, ecosystem services
• SH8_8 Visual and performing arts, screen, arts-based research

Other members of the group

• Sabrina Fagioli
• Max Garagnani
• Stefano Mastandrea
• Alessia Tessari
• Luca Tummolini
• Carmelo Mario Vicario

Publications

• Benvenuti, M., Scorolli, C., Di Iorio, A., Pelowski, M. & Chirico, A. (2024) Emerging technologies in varied domains: Perspectives and future research directions, Editorial, Computers in Human Behavior.
• Lucifora, C., Gangemi, A., Scorolli, C. (in press). Attribution of Creative Skills to Human and Artificial Artists in Alethic and Epistemic Possible Worlds. Psychology of Aesthetics, Creativity, and the Arts. 
• Lucifora, C., Schembri, M., Poggi, F., Grasso, G.M., & Gangemi, A. (2023) Virtual reality supports perspective taking in cultural heritage interpretation. Computers in Human Behavior.
• Lucifora, C., Scorolli, C., & Gangemi, A. (2023). CREON: a Creative Ontology based on Psychological and Neuroscientific Studies. In CREAI@ AI* IA (pp. 60-70).
• Masari, G. & Scorolli, C., Homo loquens, homo ridens. Interrelazioni tra risata e linguaggio, Sistemi Intelligenti, 2024, 36, 179 – 206.
• Scorolli, C. & Mazzuca, C. (2024) The Grounding of Concrete and Abstract Language: Consolidated Evidence, New Issues, and New Challenges, in: The Routledge Handbook of Embodied Cognition 2nd Edition, Taylor & Francis Group, pp. 17 – 47.
• Scorolli C., Naddei Grasso E., Stacchio L., Armandi V., Matteucci G., & Marfia G. (2023). Would you rather come to a tango concert in theater or in VR? Aesthetic emotions & social presence in musical experiences, either live, 2D or 3D, Computers in Human Behavior,
• Stacchio, L., Perlino, M., Vagnoni, U., Sasso, F., Scorolli, C., & Marfia, G. (2022) Who will Trust my Digital Twin? Maybe a Clerk in a Brick-and-Mortar Fashion Shop. Proceedings - 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2022, 814-815.
• Strappini, F., Fagioli, S., Mastandrea, S. & Scorolli, C. (2024) Sustainable materials: a linking bridge between material perception, affordance, and aesthetics, Frontiers in Psychology.