Nimbus is a Hybrid Mixed Reality + Virtual Reality HMD (Head Mounted Display) concept designed from 2010-2012. Long before Oculus and Palmer Luckey, I knew XR (Extended Reality) was the future and I wanted to try my hand at creating it before others inevitably would.
Extensive research was conducted in relation to things such as technical, historical, cultural, philosophical, ergonomic and sensorial considerations and implications of virtually modified reality systems. The end result was the design of a physical device and it’s many states of UI/UX, as well as various modes of control (including eyetracking, gestural expressions and voice activated commands).
S T A T E S O F
I M M E R S I O N
Nimbus offers three different states of immersion to suit different environments and use cases:
– Human-Reality: This is the stand-by mode - similar to having a smartphone's screen turned off and facing down. The user may remain wearing the device without the presence of virtual stimuli.
– Mixed-Reality: This is the standard mode. The user experiences a mixture of real and virtual audio and visual stimuli.
– Virtual-reality: This is the exclusion mode. The user experiences full visual immersion made possible by the electrochromic display that turns the viewfinder opaque, thereby blocking off all external visual stimuli.
The Nimbus project envisions a future where people use HMDs as much as commonly as we presently use smartphones. Thus, the product was designed for highly functional, comfortable and flexible all-day-wear and use. To achieve this, the Nimbus's overall physical design prioritized a balance between:
– Ample Hardware Capabilities by maximizing the volume whilst preserving all-day-wear comfort and style.
– Durability and Comfort through robust structural efficiency and careful anatomical weight distribution.
– Discrete and Modern Styling for unisex all-day casual and professional wear.
– Maximum Field of View for the most immersive experience.
– Minimal Facial Obstruction so as to not hinder face-to-face communication by being mindful of key areas of facial expression
Blueprint – Principal Dimensions and Materials.
After many studies, the project reached a physical design that would achieve full coverage of the horizontal axis of the binocular field of view. The vertical axis is limited by the upper and lower bridges of viewfinder frame, however this is not believed to be a point of concern. Due to the binocular ability of human vision, the horizontal axis of the field of view is a lot more predominant and important than the vertical axis.
The viewfinder's height of 45mm looks to maximize the Field of View while minimizing facial obstruction.
Early studies of physical models allowed us to determine the maximum facial area that the device could without impairing face-to-face communication between individuals by preventing the reading of facial expressions. Thus, it was determined that the profile area height of the viewfinder should not exceed 45 mm, with a weight distribution & support area located at the lowermost edge of the nasal bone (approximately halfway through the nasal dorsum).
The slight gradient polarization at the horizontal edge of the viewfinder serves to improve contrast, thereby improving focus and immersion. However, it still allows the user's eyes to remain visible to others, thereby not impairing eye expression in face-to-face communication.
It is natural for people to support their head with their hands, particularly when siting down a table. To make wearing the device more natural and less evident, special care was taken to contour the area around the zygomatic bone located directly in front of the ear. This allows the user to rest his head in two very common poses.
The slight recessing angle on the viewfinder was designed to avoid uncomfortable and distracting interference caused in particular by indoor aerial light sources when worn at a vertically level resting head position.
The temporal length of an adult can vary by up to 35 mm. To enhance optical immersion and guarantee an adequate light seal, the Nimbus' telescopic arms can extend or retract up to 38 mm, thereby covering all but the most exceptional facial anatomies. The excess 3 mm from 35-38 mm was added in order to facilitate putting the device on or taking it off in the cases of users with the largest temporal lengths.
The Nimbus was designed for unisex, casual & professional, all-day & anytime use.
C O N T R O L S
Nimbus can be controlled through 4 different methods. They have some overlap, but were designed to individually cover the fundamental features of specific types of interfacing. Thus, each control method is not very potent by itself, but becomes very powerful when used in conjunction with each other. Below is a brief summary of each control method, but a more in depth individual breakdown is offered in the following sections.
– Physical Controls: Allows for full navigation as well as configuration of the device, except for text input. As a counter-balance to this, all other modes feature text input.
– Gestural controls: Allows for basic navigation commands. Enhances immersion due to interaction with environment aware virtual elements. Can cause fatigue of the arms when operated without a resting surface.
– Eye tracking. Offers privacy and hands-free operation since only subtle eye movements are required, though it is the least autonomous form of control. It relies on timed delays to confirm actions and is thus relatively not the quickest control method.
– Voice Commands: Foregoes privacy for powerful hands-free automation. This is the most powerful and time efficient way of controlling the device, though its specific abilities depend on the development and refinement of machine-learning based processing. This control method can only be used in environments and situations with moderate noise levels.
C O N T R O L S:
P H Y S I C A L
There are a total of 6 buttons and 1 touch-sensitive that are distributed between both sides of the device. The right side houses the navigation commands and the left side the configuration commands. As such, the device can be (in principle) physically operated with just one hand.
The left side of the device houses 4 buttons for configuration commands.
The right side of the device houses houses 2 buttons and 1 touch strip for navigation commands.
Physical Command Map
The precise size, positioning and distribution of physical commands was determined based on multiple ergonomic studies which looked to find comfortable, natural and logical hand and finger positioning.
As a result, the positioning of these commands were determined by a relaxed "semi-closed" hand raised to the temples. In this position it is noticeable that the three middle fingers - index, middle and annular - align both horizontally and vertically and are therefore the most suitable for operating physical controls.
Relaxed hand positioning viewed from the user's perspective reveals optimal physical button siez, positioning and distribution. The three central fingers align naturally both vertically and horizontally and are thereby the most suitable for operating physical controls.
These ergonomic studies also showed a slight backwards tilting angle from the hand at the temple. This served as a key design consideration that influenced the device's overall shape, with the physical controls being placed at a 10° angle.
When raised to the temples, the hand tilts backwards slightly (for illustrative purposes, the image above exaggerates this effect). This served as a key design consideration for the device's overall shape, with the physical controls being placed at a 10° angle (seen below).
Physical controls were placed at a 10° angle to accommodate the natural angle of a raised hand.
C O N T R O L S:
G E S T U R E S
Three basic one-hand gestures are used to control the device. The fact these require only one hand allows the user to be using his other hand for other purposes. When Nimbus identifies any one of these three gestures, it begins watching for further gestural commands: Neutral, Options and Scroll.
The three basic gestures used to control Nimbus.
– Neutral: This is the standard navigation+selection position that interacts with the currently visible elements. This is equivalent to the tracking + left mouse button on a standard computer mice.
– Options: This opens the options menu for the element currently selected. This is equivalent to the tracking + right mouse button on a standard computer mice.
– Scroll: This scrolls through documents or elements. This is equivalent to the scroll wheel on a standard computer mice.
N E U T R A L
To select an element, simply move your thumb inwards and outwards. You may click an element by realizing the full inwards / outwards thumb movement, or you may drag and move elements by moving the thumb inwards and them moving the hand as a whole. Release the element by moving the thumb outwards again.
O P T I O N S
The Options gestures position is the same as the neutral position, but with the hand turned around (facing inwards toward the user). This gesture provides a few different functionalities.
Options: Back/Forward – A flick of the wrist inwards or outwards serves as Back / Forward commands. The controls are inverted from the illustrations above if the left hand is used.
Options: Quit – By moving the thumb inwards and then holding it there for 5 seconds, the foremost application will be closed.
S C R O L L
C O N T R O L S:
E Y E T R A C K I N G
Controlling Nimbus through eye-tracking offers the best privacy out of all the modes, whilst also being hands-free. It is equivalent to your eyes acting as a mouse cursor. It is the least autonomous out of all control methods. As it relies on timed delays to confirm actions, it is thus not the quickest control method relative to others.
However, advanced users who are familiarized with the interface and their own workflow may adjust the timed-delay to better suit their navigation dexterity.
This mode can result in fatigue if always left active, but can be quickly toggled on/off through physical or voice controls.
C O N T R O L S:
V O I C E C O M M A N D S
This control method foregoes privacy for the power of hands-free automation. Can only be used in environments and situations with moderate noise levels.
With the power of machine learning processing, voice commands can batch actions and skip over multiple navigation steps. Voice commands offer the quickest way to find and open archived files and to input text (though this may not hold true when it comes to editing existing text).
In concept, the specific abilities and features of voice commands are virtually limitless, but in reality this depends on the skill-set of the machine learning algorithms and raw processing power.
I N T E R F A C E
The device's three-dimensional display brings unprecedented challenges and opportunities for designing the UI. Given the device is intended for all-day use, a major concern when designing the operating system interface was to make it so that it would not be overtly invasive or excessively stimulating. Thus, the interface was conceived on the idea of being "light / airy", with a strong focus on maximizing cognitive clarity and operational simplicity. To this effect, the following design concepts were implemented:
– Useful Field of View: While peripheral vision is great for motion detection, it is not good for conducting precise tasks and does not allow for color distinction. It is for this this reason that the workable area used by the interface is limited to the binocular overlap region.
– Dynamic Contrast: By analyzing the real world scene using front facing cameras and light sensors, interface backgrounds and elements shift between light and dark color schemes to enhance visibility. As an example, on a daylight scene the UI adopts darker backgrounds with superimposed lighter elements; while on a nighttime scene the UI adopts lighter backgrounds with superimposed darker elements.
Dynamic Contrast showcased on the Dialer, Phonebook and Contact Details windows.
– Background Transparency: In order to allow the user to remain visually aware of the environment around him, all windows posses semi-transparent backgrounds.
– Multi-dimensional Object Layering Framework: In looking to take advantage of the device's three-dimensional capabilities, a layered multi- object placement framework was designed. When combined with subtle parallax effects, this innovation breaths life into the interface and enhances its cognitive clarity.
Window background transparency on a daylight Dynamic Contrast scheme.
The right side shows element parallax from the Multi-dimensional Object Layering Framework being forcefully evidenced.
– Manual Spatial Placement: In looking to expand use case suitability, Nimbus provides various modes of viewing, organizing, and interacting with content. One such feature allows the user to toggle between interface elements being fixated to the current viewing area or to manually lock instances or groups of interface windows and elements in fixed points in space.
Left: File browser window, document preview and selection information panel.
Right: Keyboard. The rightmost vertical bar opens the numeric keypad.
– Floating Workspace Framework: Without the material restrictions of screen size, position, ratio or even planular projection, a new framework had to be designed to tap into the expansive potential of 3D UIs. The Nimbus Floating Space Framework allocates interface elements to working surfaces, areas and panels to display different kinds of content and virtual tools. The framework looks to conceptually determine optimal spatial allocation, distancing and scale between major UI elements. Though the Nimbus Floating Space Framework was designed in 2011, Google presented a very similar framework for desigining VR interfaces in its Google I/O 2017 conference.
Floating Workspace Framework demonstrates optimal spatial allocation, distancing and scale between major UI elements.
The left side shows the top view and the right side the perspective view.
T E C H N O L O G Y
During its inception in 2010, the Nimbus project based its functional concept on a series of highly advanced (albeit proven) technologies that were restricted to laboratory and military use. Since then these technologies have matured and a few of them have in fact been implemented in today's enterprise and consumer Mixed Reality HMDs, such as the Microsoft Hololens and MagicLeap V1.
Nimbus original inner workings were based with the following technologies:
Hardware Systems Allocation
– VRD (Virtual Retinal Display / aka Retina Screen Projection): This is the same system that is used present-day by the Microsoft Hololens and MagicLeap V1. With the inclusion of motor-assisted mirrors and lenses, the system can calibrate itself to a user's specific inter-pupilary distance (which varies according to age, race and gender, and is essential for the image to projected effectively).
– Electrochromic Glass: This is a type of electrically activated glass that changes its light passing properties when a voltage is applied to it. By applying the electrical load, the glass is able to change from a state of transparency to a translucent state. This allows for switching between the Mixed Reality and Virtual Reality states of immersion.
– Osteophonic Conduction System: is the conduction of sound waves to the inner ear through the use of the skull bone structure. This method of providing auditory stimuli offers two huge advantages as it relates to immersion:
Firstly, the sound effect achieved by osteophonics is functionally equivalent to performing auditory stimuli directly in the ear canal, which in turn allows for natural mixing of real and virtual stimuli and hence a more immersive experience as real a virtual elements fuse together in Mixed Reality mode.
Secondly, this method saves the user from having to insert electronic equipment into their ears, which works in favor of both safety (by allowing, for example, the user to hear the sound of a car horn) as well as for preserving comfort.
The big drawback to osteophonics is its inability to produce low-frequency audio waves (bass).
– Other Technologies: Nimbus also makes use of Touch and Pressure Sensitive Sensors, Infrared Cameras (for Eye Tracking), Parallax front facing cameras for (Gesture Recognition), Multiple Microphones (for Speech Recognition), Bluetooth Antenna, Wi-Fi Antenna, Portable Transceiver (for Celular Networks), Photosensitive Sensor, Accelerometer Sensor, Gyroscopic Sensor, and a Bionic Iris Identifier.
G A L L E R Y
