Increased Reality

By system of increased reality one understands a system (with the direction Informatique) which makes possible the superposition of a model virtual 3D or 2D to the perception which we naturally have of the Réalité and this in Real-time. This system can as well apply to visual perception (superposition of virtual image to the real images) that with proprioceptive perceptions like tactile or auditive perceptions.

The concept of increased reality thus aims at supplementing our perception of the world reality, while adding to it of the fictitious, nonperceptible elements naturally. Reality increased thus indicates the various methods which make it possible to encrust with realistic way of the virtual objects in a sequence of images. Its applications are multiple and touch fields more and more: video games and " edutainment" , virtual cinema and television (post-production, studios, sporting retransmissions,…), industries (design, design, maintenance, assembly, piloting, telerobotic robotics and, establishment, study of impact,…), medical, etc

Principles

To decorate fictitious objects a sequence Vidéo resulting from a fixed plan hardly poses problems. The applications concerned asking often enormously realism, it is essential that the addition of objects in a scene does not disturb pas la coherence of the filmed contents. The fact of moving the Caméra however implies a movement in the image of the filmed scene. To ensure coherence between two real and virtual flows, a rigid bond must be maintained between the two worlds. In order to give the illusion that these fictitious objects belong to the same world, it is necessary to place them well, to direct them well and to respect scale factors compared to the really filmed objects. To place well the virtual objects compared to the objects of the scene requires to know the position of the camera compared to scene.

The problem of the localization of the camera is thus important and can be solved by various approaches. One can use a system of Capteur S, as of the magnetic sensors which measure the distortion of the Magnetic field to calculate their position, of the optical sensors, the Encodeur S on the engines of the foot of the cameras or, obviously, the Video stream.

In the case of the use of external sensors to the system of catch of sight, information of this system (angle, position, focal distance) is recovered using sensors and the Incrustation is directly reproduced on the good scale on the image to increase.

However, if one considers only the information acquired by the camera, the problem of increased reality brings back to a problem of vision by computer. In certain contexts applicatifs like the Cinema, the whole of the video sequence is available before the treatment. Accordingly of Post-production, of the heavy treatments in term of computing time are possible. Techniques allowing at the same time the rebuilding 3D of a certain number of points of the scene and the localization 3D of the camera are implemented by techniques of Autocalibration or adjustment of beams. Commercial Logiciel S resting on this principle are available right now (one can quote Boujou of the company 2d3 -- exit of the university of Oxford -- and MatchMover of the Realviz company -- exit of the Robotvis project of the INRIA Sophia Antipolis --). These methods are however very dependant on the quality of the mapping of the primitives 2D (noise of extraction, spatial distribution, error count of pairing…) and the user is sometimes put at contribution.

Within the framework of interactive applications (Audio-visual in the " conditions of the direct" , Industry, interactive Video games, medical, military) the recourse to techniques of Autocalibration is not possible. Techniques allowing the localization of the camera starting from the current image (and possibly of the preceding ones) are necessary. If a model of the scene (or part of this one) is available, the calculation from point of view is obviously an ideal solution with this problem. If the structure 3D of the scene is (partially) unknown other approaches resting, for example, on the calculation of the displacement of the camera are possible. The advantages of these interactive approaches are multiple:

They allow a real-virtual integration in real-time (IE, at video rate) because subjacent calculations are relatively can expensive
It is not either necessary to make a calibration " lourd" system as it is the case if other types of sensors are used, nor to lay out a priori of the complete sequence.
It can function on standard platforms PC what implies a relatively weak cost.

In all the cases, once the localized Camera compared to the scene, this information is used to adapt the data-processing model of the scene, so as to be able to calculate the characteristics of the virtual objects which one wants to insert there. These virtual objects then undergo geometrical transformations which will allow of the " plaquer" in the image source, so as to integrate them into it in the most natural possible way.

Applications

Publicity

Increased reality is used for example to insert advertizing inserts in images vidéos turned on sports grounds: the logos of the companies partners of events can thus be always visible whatever the visual angle chosen by the Réalisateur, it is of more possible to post various messages on the same site.

Leisure

Increased reality also makes it possible to plunge the spectator in the middle of a partially real world (of the real decorations) and partially virtual (of the objects, animals…). The spectator becomes actor while interacting with the virtual objects by means of Capteur S. a first application accessible to the general public is envisaged with the Futuroscope as from April 2008, in the attraction the Animals of the Future , carried out by the French company Total Immersion

Industrial

Increased reality makes it possible to insert Automobile S existing only in a Ordinateur in filmed real decorations (cities for example) and to make them interact with the real elements in order to visualize the integration of the Prototype in the real landscape.

See too

Research laboratories

Singapore

Mixed reality lab - Singapore

France

Project ISA
Project Lagadic IRISA-INRIA Rennes (see the demonstrations)
Laboratory of the Photonic Systems
Total Immersion

Germany

Fraunhofer Institute fuer Graphische Datenverarbeitung

Switzerland

MIRAlab

External bonds

Increased Reality: page of Gilles Simon on the applications and the problems of RA
Some examples of application of increased reality realized by Microsoft
Pong actually increased
Blog on increased reality and its application to Futuroscope

Random links: