Establishment of the research infrastructure (WP4)

The objectives set for the infrastructure was:

  1. to allow achievement of a generic multimodal interface for product engineering applications, including 3 output modalities (3D visual, haptics and 3D audio) and 3 input modalities (tracking, gesture and speech), such as to be able to replace totally the conventional desktop interfaces based on 2D display and mouse.
  2. To allow development of applications across the product life cycle, including manufacturing and recycling. Therefore, a full flexible manufacturing cell was also purchased and installed at the department for VEGA purposes.

Some little pieces of equipment were already available in the Department (a HMD 5DT, and 2 sensors Flock of Bird), but most of the other pieces needed to achieve the proposed structure have been newly installed. For the large 3D visualisation a CAVE system as well as the large haptic devices (SPIDAR) have been developed by the VEGA team, in collaboration with the partners from Fraunhofer and Siemens, buying only the components from the producers. This allowed to develop these equipments in only one year with a cost of about 130 KEuro, while the direct purchase from the market would have been costed around 400 KEuro.

Output modalities: 3D visual and haptic feedback

Some visualization systems like HMD 5DT and shuttered glasses StereoGraphics were already available in the laboratory since 2002. Within project VEGA, large scale visualization systems have been envisaged like holobench and CAVE. An innovative multipurpose architecture was developed within VEGA project, able to provide both possibilities for the 3D visualization: CAVE and holobench functionality. Therefore the system is called “holo-CAVE” (fig.1).

Fig. 1. Holo-CAVE: an innovative concept developed by the VEGA team

This solution allows making experiments related to the study of technical tasks that are performed by a human operator in the posture “seated” or alternatively, could be configured as a CAVE system, in which the user is performing tasks that require a “standing” posture.
A second visualization system that was acquired within VEGA mainly for augmented reality applications is the TRIVISIO HMD with see through option (fig.2.).

Fig.2. HMD with see through
purchased for augmented reality

Haptic devices

In order to achieve a multi-modal interface, the second output modality that was chosen is haptic. Out if the commercial haptic device Phantom with 3 degrees of freedom (dof), force-feedback and 6 dof positional sensing that was purchased within the project VEGA, the research team of the laboratory developed a state of the art wired based system (fig.3), following a solution previously developed at University of Tokyo. The system, includes 8 wires and could be used either as a 7 degree of freedom device that manipulates and grasps a 3D virtual object or, alternatively, as a 3+3 dof bi-fingered system that is able to manipulate two touching points in the virtual environment. The advantage of this system is that it provides a large workspace and is scalable, being possible to be integrated into the Holo-CAVE.

Fig.3. Wired haptic system

Fig.4. Data gloves

Input modalities: tracking, gesture and speech

Tracking is an important input technologie that has been included in the targeted VR systems. The precision of tracking is the key for the realistic interaction between the real and virtual worlds, which is achieved by haptic systems integrated into the VR interface.
However, tracking is needed also as part of the Holo-CAVE visualization system in order to enable the user to change its position in the virtual space. This is achieved through a Flock of Bird magnetic tracking system with the range radius of 1.5 meters.
For the gestures needed to emulate various commands in the virtual environment, data gloves are used (fig.4), while for the speech ordinary audio system is used together with the Microsoft voice recognition classes. Together with the dataglove, a Flock of bird sensor is used, in order to track the position of the hand while performing various gestures that have been implemented.

Flexible manufacturing

For the third VEGA research workgroup (WG3) “Virtual manufacturing and recycling”, the infrastructure of the laboratory was enriched with a Flexible manufacturing unit including two articulated robots ABB IRR, two EMCO CNC machines (lathe and milling) and other perirobotic devices included VISION system. A picture of the real flexible cell installed at Brasov is given in fig. 5.
The system allows the achievement of full complex manufacturing cycles from machining to assembly without human intervention. It is provided with a vision system and is in process to be linked with the virtual environment described above in order to allow simulation of complex machining and assembly processes in virtual environment prior to the real execution.

Fig.5. The Flexible manufacturing cell installed in the VEGA lab