Accession Number:



Helicopter Aircrew Training Using Fused Reality

Descriptive Note:

Conference paper

Corporate Author:


Personal Author(s):

Report Date:


Pagination or Media Count:



This paper describes a novel Mixed Reality technique for robust, real-time chromakey processing for training applications using software and off-the-shelf video hardware. This technique has been coined and patented by the author as Fused Reality. Until now chromakeying has been conducted using dedicated hardware, imposing substantial restrictions on the visual environments that will support chromakey. Variations on the traditional chromakey setup, such as using retroreflective screens and light-emitting cameras, can overcome some of the techniques original drawbacks such as lighting difficulties, but they can introduce new problems as well i.e., a users hand obstructing the light projected from the head-mounted source can create an ill-defined silhouette. The novel chromakey method introduced in this paper is applied to training helicopter aircrew personnel using a prototype simulator, the Prototype Aircrew Virtual Environment Training PAVET System. The ultimate goal of the PAVET will be to provide training to Navy aircrewmen in all operational aspects of the MH-60S, including aerial gunnery, search and rescue, and vertical replenishment. A key requirement for these types of tasks is for trainees to see and manipulate physical objects e.g., a jammed gun at close range while viewing an interactive flight and shipboard environment. In order to satisfy space and cost constraints, some physical objects that the aircrewmen physically interact with such as a rescue litter must be capable of being sent out into the virtual environment and later retrieved. Fused Reality accomplishes this critical feature through contraction and expansion of specific real-world objects as they move away and toward the trainee. Fused Realitys adaptive color recognition allows for realistic set lighting, colors, and user movement and positioning.

Subject Categories:

  • Helicopters
  • Military Forces and Organizations
  • Human Factors Engineering and Man Machine Systems

Distribution Statement: