Overview of Interactive

Interactivity refers to the various telecommunications signal conversion technologies that output to human sensory organs, such as head-mounted displays and haptic feedback. It includes input technologies represented by position/force/velocity sensors and composite interaction technologies represented by brain-computer interfaces. This includes AR, VR, MR, and other technologies.

Although there is no universal consensus on the exact meaning of interactivity, the most popular definition comes from computer graphics literature. Interactivity is a way of using physical input/output devices to accomplish general tasks in human-computer dialogues. A recent change is that interactivity is the fusion of input and output, composed of all software and hardware elements, providing users with a way to accomplish tasks. Various branches of interactivity, through scanning, modeling, animation design, and other techniques, bring highly realistic virtual humans into the metaverse and provide surreal interaction methods in the symbiotic social environment between humans and machines. At the same time, through XR (VR/AR/MR) and holographic projection technologies, virtual humans can step out of virtual space and seamlessly integrate with real-world scenes, serving as the connector between virtual and physical worlds.

An interaction task is the unit of user input information, such as inputting a piece of text, issuing a command, or specifying a two-dimensional position. A similar concept is a domain object, which is an application data that can be manipulated by the user. Interactivity serves as the adhesive between physical I/O devices and interaction tasks or domain objects. Different types of interactivity can be used to map a specific device to a specific domain object. For example, pen-based text input has different gesture letters. Generally, the poorer the compatibility between the device and the domain object, the more complex the interactivity. For example, using a mouse to specify a two-dimensional point involves a trivial interactivity, while using a mouse to rotate a three-dimensional object requires more creativity to design the interaction and more code to implement it.

VR simulates a virtual world by using computer-generated environments and immerses users into that environment using devices, creating a sense of being present in the virtual world. It emphasizes real-time interaction between the user and the virtual world, resulting in a closed and immersive virtual experience. The virtual world is not directly visible to us but can be seen through devices such as VR glasses, hence the term "virtual reality."

AR is a new technology that seamlessly integrates real-world information and virtual world information. It takes difficult-to-experience physical information (visual, auditory, taste, tactile, olfactory, etc.) within a certain space in the real world and simulates and overlays it onto the real world using computer and other scientific technologies, perceived by human senses, thereby achieving sensory experiences beyond reality. The real environment and virtual objects are simultaneously superimposed in the same image or space, enhancing the real world, hence the term "augmented reality."

Although VR and AR have not reached their full potential yet, there are signs of their fusion, which is known as MR. MR blends the real and virtual worlds together to create a new visual environment. In this environment, there are both real objects and virtual information, and it is crucial that it is "real-time" to enhance the user's sense of reality. This is referred to as "mixed reality."

Currently, there are five main ways in which users interact with AR, VR, XR, and MR: eye-tracking-based eye-controlled interaction, natural language-based voice interaction, body-based gesture interaction, and neuron-based brain-machine (interface) interaction.