Contents

Contents
1 Introduction to Augmented Reality 1
Definition and Scope 3
A Brief History of Augmented Reality 4
Examples 13
Industry and Construction 13
Maintenance and Training 17
Medical 18
Personal Information Display 20
Navigation 21
Television 22
Advertising and Commerce 25
Games 27
Related Fields 28
Mixed Reality Continuum 28
Virtual Reality 29
Ubiquitous Computing 30
Summary 31
2 Displays 33
Multimodal Displays 34
Audio Displays 34
Haptic, Tactile, and Tangible Displays 35
Olfactory and Gustatory Displays 37
Visual Perception 39
Requirements and Characteristics 40
Method of Augmentation 40
Ocularity and Stereoscopy 42
Focus 45
Occlusion 47
Resolution and Refresh Rate 48
Field of View 50
Viewpoint Offset 51
Brightness and Contrast 53
Distortions and Aberrations 55
Latency 55
Ergonomics 55
Social Acceptance 55
Spatial Display Model 56
Visual Displays 58
Near-Eye Displays 59
Handheld Displays 69
Stationary Displays 72
Projected Displays 78
Summary 84
3 Tracking 85
Tracking, Calibration, and Registration 86
Coordinate Systems 87
Model Transformation 88
View Transformation 88
Projective Transformation 89
Frames of Reference 89
Characteristics of Tracking Technology 90
Physical Phenomena 90
Measurement Principle 91
Measured Geometric Property 91
Sensor Arrangement 91
Signal Sources 92
Degrees of Freedom 92
Measurement Coordinates 92
Spatial Sensor Arrangement 93
Workspace Coverage 94
Measurement Error 94
Temporal Characteristics 95
Stationary Tracking Systems 96
Mechanical Tracking 96
Electromagnetic Tracking 97
Ultrasonic Tracking 98
Mobile Sensors 99
Global Positioning System 99
Wireless Networks 101
Magnetometer 102
Gyroscope 102
Linear Accelerometer 103
Odometer 104
Optical Tracking 105
Model-Based versus Model-Free Tracking 106
Illumination 106
Markers versus Natural Features 109
Target Identification 113
Sensor Fusion 117
Complementary Sensor Fusion 117
Competitive Sensor Fusion 117
Cooperative Sensor Fusion 118
Summary 120
4 Computer Vision for Augmented Reality 121
Marker Tracking 123
Camera Representation 124
Marker Detection 126
Pose Estimation from Homography 128
Pose Refinement 132
Multiple-Camera Infrared Tracking 132
Blob Detection 133
Establishing Point Correspondences 133
Triangulation from Two Cameras 135
Triangulation from More Than Two Cameras 137
Matching Targets Consisting of Spherical Markers 137
Absolute Orientation 137
Natural Feature Tracking by Detection 138
Interest Point Detection 140
Descriptor Creation 144
Descriptor Matching 145
Perspective-n-Point Pose 146
Robust Pose Estimation 148
Incremental Tracking 149
Active Search 150
Kanade-Lucas-Tomasi Tracking 151
Zero-Normalized Cross-Correlation 152
Hierarchical Search 154
Combined Detection and Tracking 155
Simultaneous Localization and Mapping 156
Five-Point Algorithm for Essential Matrix 157
Bundle Adjustment 158
Parallel Tracking and Mapping 159
Relocalization and Loop Closure 160
Dense Mapping 161
Outdoor Tracking 164
Scalable Visual Matching 165
Prior Information from Sensors 167
Prior Information from Geometry 169
Simultaneous Tracking, Mapping, and Localization 170
Summary 176
5 Calibration and Registration 179
Camera Calibration 180
Internal Camera Parameters 180
Correcting Lens Distortion 182
Display Calibration 183
Single Point Active Alignment Method 185
Head-Mounted Display Calibration Using a Pointing Device 186
Hand–Eye Calibration 188
Registration 190
Geometric Measurement Distortions 190
Error Propagation 191
Latency 192
Filtering and Prediction 192
Summary 194
6 Visual Coherence 195
Registration 196
Occlusion 199
Occlusion Refinement 201
Probabilistic Occlusion 202
Model-Free Occlusion 202
Photometric Registration 205
Image-Based Lighting 207
Light Probes 208
Offline Light Capturing 210
Photometric Registration from Static Images 210
Photometric Registration from Specular Reflections 211
Photometric Registration from Diffuse Reflections 212
Photometric Registration f