Lidar

History and etymology

Lidar originated in the early 1960s, shortly after the invention of the laser, and combined laser-focused imaging with the ability to calculate distances by measuring the time for a signal to return using appropriate sensors and data acquisition electronics. Its first applications came in meteorology, where the National Center for Atmospheric Research used it to measure clouds. The general public became aware of the accuracy and usefulness of lidar systems in 1971 during the Apollo 15 mission, when astronauts used a laser altimeter to map the surface of the moon.

Although some sources treat the word "lidar" as an acronym, the term originated as a portmanteau of "light" and "radar". The first published mention of lidar, in 1963, makes this clear: "Eventually the laser may provide an extremely sensitive detector of particular wavelengths from distant objects. Meanwhile, it is being used to study the moon by 'lidar' (light radar) ..." The Oxford English Dictionary supports this etymology.

The interpretation of "lidar" as an acronym ("LIDAR" or "LiDAR") came later, beginning in 1970, based on the assumption that since the base term "radar" originally started as an acronym for "RAdio Detection And Ranging", "LIDAR" must stand for "LIght Detection And Ranging", or for "Laser Imaging, Detection and Ranging". Although the English language no longer treats "radar" as an acronym and printed texts universally present the word uncapitalized, the word "lidar" became capitalized as "LIDAR" or "LiDAR" in some publications beginning in the 1980s. Currently no consensus exists on capitalization, reflecting uncertainty about whether or not "lidar" is an acronym, and if it is an acronym, whether it should appear in lower case, like "radar". Various publications refer to lidar as "LIDAR", "LiDAR", "LIDaR", or "Lidar". The USGS uses both "LIDAR" and "lidar", sometimes in the same document; the New York Times uses both "lidar" and "Lidar".

General description

Lidar uses ultraviolet, visible, or near infrared light to image objects. It can target a wide range of materials, including non-metallic objects, rocks, rain, chemical compounds, aerosols, clouds and even single molecules. A narrow laser-beam can map physical features with very high resolutions; for example, an aircraft can map terrain at 30-centimetre (12 in) resolution or better.

Lidar has been used extensively for atmospheric research and meteorology. Lidar instruments fitted to aircraft and satellites carry out surveying and mapping – a recent example being the U.S. Geological Survey Experimental Advanced Airborne Research Lidar. NASA has identified lidar as a key technology for enabling autonomous precision safe landing of future robotic and crewed lunar-landing vehicles.

Wavelengths vary to suit the target: from about 10 micrometers to the UV (approximately 250 nm). Typically light is reflected via backscattering. Different types of scattering are used for different lidar applications: most commonly Rayleigh scattering, Mie scattering, Raman scattering, and fluorescence. Based on different kinds of backscattering, the lidar can be accordingly called Rayleigh Lidar, Mie Lidar, Raman Lidar, Na/Fe/K Fluorescence Lidar, and so on. Suitable combinations of wavelengths can allow for remote mapping of atmospheric contents by identifying wavelength-dependent changes in the intensity of the returned signal.

Design

In general there are two kinds of lidar detection schemes: "incoherent" or direct energy detection (which is principally an amplitude measurement) and coherent detection (which is best for Doppler, or phase sensitive measurements). Coherent systems generally use optical heterodyne detection, which, being more sensitive than direct detection, allows them to operate at a much lower power but at the expense of more complex transceiver requirements.

In both coherent and incoherent lidar, there are two types of pulse models: micropulse lidar systems and high energy systems. Micropulse systems have developed as a result of the ever-increasing amount of computer power available combined with advances in laser technology. They use considerably less energy in the laser, typically on the order of one microjoule, and are often "eye-safe," meaning they can be used without safety precautions. High-power systems are common in atmospheric research, where they are widely used for measuring many atmospheric parameters: the height, layering and densities of clouds, cloud particle properties (extinction coefficient, backscatter coefficient, depolarization), temperature, pressure, wind, humidity, trace gas concentration (ozone, methane, nitrous oxide, etc.).

There are several major components to a lidar system:

1. Laser — 600–1000 nm lasers are most common for non-scientific applications. They are inexpensive, but since they can be focused and easily absorbed by the eye, the maximum power is limited by the need to make them eye-safe. Eye-safety is often a requirement for most applications. A common alternative, 1550 nm lasers, are eye-safe at much higher power levels since this wavelength is not focused by the eye, but the detector technology is less advanced and so these wavelengths are generally used at longer ranges and lower accuracies. They are also used for military applications as 1550 nm is not visible in night vision goggles, unlike the shorter 1000 nm infrared laser. Airborne topographic mapping lidars generally use 1064 nm diode pumped YAG lasers, while bathymetric systems generally use 532 nm frequency doubled diode pumped YAG lasers because 532 nm penetrates water with much less attenuation than does 1064 nm. Laser settings include the laser repetition rate (which controls the data collection speed). Pulse length is generally an attribute of the laser cavity length, the number of passes required through the gain material (YAG, YLF, etc.), and Q-switch speed. Better target resolution is achieved with shorter pulses, provided the lidar receiver detectors and electronics have sufficient bandwidth.
2. Scanner and optics — How fast images can be developed is also affected by the speed at which they are scanned. There are several options to scan the azimuth and elevation, including dual oscillating plane mirrors, a combination with a polygon mirror, a dual axis scanner (see Laser scanning). Optic choices affect the angular resolution and range that can be detected. A hole mirror or a beam splitter are options to collect a return signal.
3. Photodetector and receiver electronics — Two main photodetector technologies are used in lidars: solid state photodetectors, such as silicon avalanche photodiodes, or photomultipliers. The sensitivity of the receiver is another parameter that has to be balanced in a lidar design.
4. Position and navigation systems — Lidar sensors that are mounted on mobile platforms such as airplanes or satellites require instrumentation to determine the absolute position and orientation of the sensor. Such devices generally include a Global Positioning System receiver and an Inertial Measurement Unit (IMU).

3D imaging can be achieved using both scanning and non-scanning systems. "3D gated viewing laser radar" is a non-scanning laser ranging system that applies a pulsed laser and a fast gated camera. Research has begun for virtual beam steering using DLP technology.

Imaging lidar can also be performed using arrays of high speed detectors and modulation sensitive detector arrays typically built on single chips using CMOS and hybrid CMOS/CCD fabrication techniques. In these devices each pixel performs some local processing such as demodulation or gating at high speed, downconverting the signals to video rate so that the array may be read like a camera. Using this technique many thousands of pixels / channels may be acquired simultaneously. High resolution 3D lidar cameras use homodyne detection with an electronic CCD or CMOS shutter.

A coherent imaging lidar uses synthetic array heterodyne detection to enable a staring single element receiver to act as though it were an imaging array.

In 2014 Lincoln Laboratory announced a new imaging chip with more than 16,384 pixels, each able to image a single photon, enabling them to capture a wide area in a single image. An earlier generation of the technology with one-quarter as many pixels was dispatched by the U.S. military after the January 2010 Haiti earthquake; a single pass by a business jet at 3,000 meters (10,000 ft.) over Port-au-Prince was able to capture instantaneous snapshots of 600-meter squares of the city at 30 centimetres (12 in), displaying the precise height of rubble strewn in city streets. The new system is another 10x faster. The chip uses indium gallium arsenide (InGaAs), which operates in the infrared spectrum at a relatively long wavelength that allows for higher power and longer ranges. In many applications, such as self-driving cars, the new system will lower costs by not requiring a mechanical component to aim the chip. InGaAs uses less hazardous wavelengths than conventional silicon detectors, which operate at visual wavelengths.

Types of applications

Lidar has a wide range of applications which can be divided into airborne and terrestrial types. These different types of applications require scanners with varying specifications based on the data's purpose, the size of the area to be captured, the range of measurement desired, the cost of equipment, and more.

Airborne lidar

Airborne lidar (also airborne laser scanning) is when a laser scanner, while attached to a plane during flight, creates a 3D point cloud model of the landscape. This is currently the most detailed and accurate method of creating digital elevation models, replacing photogrammetry. One major advantage in comparison with photogrammetry is the ability to filter out reflections from vegetation from the point cloud model to create a digital surface model which represents ground surfaces such as rivers, paths, cultural heritage sites, etc., which are concealed by trees. Within the category of airborne lidar, there is sometimes a distinction made between high-altitude and low-altitude applications, but the main difference is a reduction in both accuracy and point density of data acquired at higher altitudes. Airborne lidar can also be used to create bathymetric models in shallow water.

The main constituents of airborne lidar include digital elevation models (DEM) and digital survey models (DSM). The points and ground points are the vectors of discrete points while DEM and DSM are interpolated raster grids of discrete points. The process also involves capturing of digital aerial photographs. In order to interpret deep seated landslides for example, under the cover of vegetation, scarps, tension cracks or tipped trees air borne lidar is used. Air borne lidar digital elevation models can see through the canopy of forest cover, perform detailed measurements of scarps, erosion and tilting of electric poles.

Airborne lidar data is processed using a toolbox(TIFFS-Toolbox for Lidar Data Filtering and Forest Studies) for lidar data filtering and terrain study software. The data is interpolated to digital terrain models using the software. The laser is directed at the region to be mapped and each point's height above the ground is calculated by subtracting the original z-coordinate from the corresponding digital terrain model elevation. Based on this height above the ground the non-vegetation data is obtained which may include objects such as buildings, electric power lines, flying birds etc. The rest of the points are treated as vegetation and used for modeling and mapping. Within each of these plots, lidar metrics are calculated by calculating statistics such as mean, standard deviation, skewness, percentiles, quadratic mean etc.

Airborne lidar bathymetric technology

The airborne lidar bathymetric technological system involves the measurement of time of flight of a signal from a source to its return to the sensor. The data acquisition technique involves a sea floor mapping component and a ground truth component that includes video transects and sampling. It works using a green spectrum (532 nm) laser beam. Two beams are projected onto a fast rotating mirror, which creates an array of points. One of the beams penetrates the water and also detects the bottom surface of the water under favorable conditions.

The data obtained shows the full extent of the land surface exposed above the sea floor. This technique is extremely useful as it will play an important role in the major sea floor mapping program. The mapping yields onshore topography as well as under water elevations. Sea floor reflectance imaging is another solution product from this system which can benefit mapping of underwater habitats. This technique has been used for three dimensional image mapping of California's waters using a hydrographic lidar.

Drones are now being used with laser scanners, as well as other remote sensors, as a more economical method to scan smaller areas. The possibility of drone remote sensing also eliminates any danger that crews of a manned aircraft may be subjected to in difficult terrain or remote areas.

Terrestrial lidar

Terrestrial applications of lidar (also terrestrial laser scanning) happen on the Earth's surface and can be both stationary or mobile. Stationary terrestrial scanning is most common as a survey method, for example in conventional topography, monitoring, cultural heritage documentation and forensics. The 3D point clouds acquired from these types of scanners can be matched with digital images taken of the scanned area from the scanner's location to create realistic looking 3D models in a relatively short time when compared to other technologies. Each point in the point cloud is given the colour of the pixel from the image taken located at the same angle as the laser beam that created the point.

Mobile lidar (also mobile laser scanning) is when two or more scanners are attached to a moving vehicle to collect data along a path. These scanners are almost always paired with other kinds of equipment, including GNSS receivers and IMUs. One example application is surveying streets, where power lines, exact bridge heights, bordering trees, etc. all need to be taken into account. Instead of collecting each of these measurements individually in the field with a tachymeter, a 3D model from a point cloud can be created where all of the measurements needed can be made, depending on the quality of the data collected. This eliminates the problem of forgetting to take a measurement, so long as the model is available, reliable and has an appropriate level of accuracy.

Terrestrial lidar mapping involves a process of occupancy grid map generation. The process involves an array of cells divided into grids which employs a process to store the height values when lidar data falls into the respective grid cell. A binary map is then created by applying a particular threshold to the cell values for further processing. The next step is to process the radial distance and z-coordinates from each scan to identify which 3D points correspond to each of the specified grid cell leading to the process of data formation.

Data Formation

For the autonomous applications a lidar with a 3D scanner is used for obstacle detection and collision avoidance.The scanner measures radial distance at different angular resolutions. The lidar emits a single laser ray and uses an interior rotating mirror to distribute the laser ray covering a large field of view.

Object Detection

The object detection procedure consists of three steps: Laser point feature calculation and prior filtering, 3D segmentation, object classification and 2D position calculation. The features for segmentation are calculated in the vicinity of a fixed radial distance in 2D and 3D.These include:

Amplitude density

This involves the percentage of points in the vicinity with amplitude values below a particular threshold. Points with a density lower than the threshold are removed from segmentation and further processing.

Height above local minimum in 2D search radius

In this case the points which have a height lower than a certain threshold are removed. On segmentation of the point cloud, all the segments satisfying the object based criteria are included in individual objects.

Advantages

The main advantages of object detection and data formation are that high point density and high spatial resolution is obtained. Also the accuracy of the acquired data sets is increased to a considerable extent.

Disadvantages

Although significant amount of research has been done for object classification from 3D point clouds, the direct extraction of individual points from Lidar inputs has not been achieved.

Object detection for transportation systems

In transportation systems, to ensure vehicle and passenger safety and to develop electronic systems that deliver driver assistance, understanding vehicle and surrounding environment is essential. LIDAR systems plays an important role in safety of transportation systems. Lots of electronic systems which add to the driver assistance and vehicle safety such as Adaptive Cruise Control (ACC), Emergency Brake Assist, Anti-lock Braking System (ABS) depends on the detection of vehicle environment to act autonomously or semi-autonomously. LIDAR mapping and estimation achieve this.

Basics overview: Current LIDAR systems use rotating hexagonal mirrors which split the laser beam. The upper three beams are used for vehicle and obstacles ahead and the lower beams are used to detect lane markings and road features. The major advantage of using LIDAR is that the spatial structure is obtained and this data can be fused with other sensors like RADAR etc. to get a better picture of the vehicle environment in terms of static and dynamic properties of the objects present in the environment.

Applications

There are a wide variety of applications for lidar, in addition to the applications listed below, as it is often mentioned in National lidar dataset programs.

Agriculture

Lidar also can be used to help farmers determine which areas of their fields to apply costly fertilizer. Lidar can create a topographical map of the fields and reveals the slopes and sun exposure of the farm land. Researchers at the Agricultural Research Service blended this topographical information with the farmland yield results from previous years. From this information, researchers categorized the farm land into high-, medium-, or low-yield zones. This technology is valuable to farmers because it indicates which areas to apply the expensive fertilizers to achieve the highest crop yield.

Another application of lidar beyond crop health and terrain mapping is crop mapping in orchards and vineyards. Vehicles equipped with lidar sensors can detect foliage growth to determine if pruning or other maintenance needs to take place, detect variations in fruit production, or perform automated tree counts.

Lidar is useful in GPS-denied situations, such as in nut and fruit orchards where GPS signals to farm equipment featuring precision agriculture technology or a driverless tractor may be partially or completely blocked by overhanging foliage. Lidar sensors can detect the edges of rows so that farming equipment can continue moving until GPS signal can be reestablished.

Plant species classification using a 3D Lidar sensor and machine learning.

This is a methodology for differentiating plant species using a 3D lidar sensor and machine learning. The procedure involves the application of logistic regression functions, support vector machines and neural networks.

Problem Statement

Autonomous robots have been used for a variety of purposes in agriculture ranging from seed and fertilizer dispersions, sensing techniques as well as crop scouting for the task of weed control. In order to execute weed control one of the major factors is the detection and classification of the plant and species. In addition it is difficult to differentiate the characteristics and appearances of the plant and convert the data into computer understandable form. Also, it is not possible to accurately detect structures of the plant in 3D data. The problem is solved by classifying the plant species by using a set of example plants and machine learning methods.

Algorithm Overview

The input to the system are 3D point clouds with range and reflectance values from a 3D laser sensor. The species of the plant is known by applying preprocessing techniques to provide the point cloud for further processing. If the species is already known then the extracted features are added as new data. The particular plant species is labeled and its features are initially stored as an example to identify the particular plant species in the real environment.

Pre-processing

In this process, there are three steps. The first step is to identify the ground plane and the points returning from the ground plane are eliminated. In the second step the remaining points are grouped as one and the individual points are detected. The third step involves the identification of the plant species. An individual coordinate system is developed for each plant. The x and y coordinates are placed on the ground while the z-axis is along the center of the plant. In this way individual plants in row cultivations are separated. The next step is to identify the plant space.

Feature extraction

To understand a set of features to identify the appearance of the plant species the criteria of size and rotation invariance of the 3D laser sensor is used. The set of features are divided into two groups: Reflectance and geometrical features

Reflectance features

Plant species have different variations based on their reflectance values. It is an important tool for plant classification. The reflectance value encodes information about the leaf size and shape. For each of the measuring points a reflectance value between 0 and 1 are determined.

Geometrical Features

The laser sensor also provides a range of data which can be transformed into 3D coordinates. Using the coordinates, the shape, dimension and structure of the plant can be characterized.

Classification

Classification of the plants is based on machine learning. For the learning process a training set is generated with known samples. Machine learning toolbox WEKA is used for this purpose.

Tested plants

The training set is generated with the help of potted plants available in common nurseries. Different plant species is chosen and the corresponding plant cloud is developed. Besides the plant the ground coordinate system is also sensed to determine the plant coordinate system that is essential for feature calculation.

Advantages

This method for distinguishing plant species is highly efficient since it uses a low resolution 3D Lidar sensor and supervised learning. It includes easy to compute feature set with common statistical features which are independent of the plant size.

Archaeology

Lidar has many applications in the field of archaeology including aiding in the planning of field campaigns, mapping features beneath forest canopy, and providing an overview of broad, continuous features that may be indistinguishable on the ground. Lidar can also provide archaeologists with the ability to create high-resolution digital elevation models (DEMs) of archaeological sites that can reveal micro-topography that are otherwise hidden by vegetation. Lidar-derived products can be easily integrated into a Geographic Information System (GIS) for analysis and interpretation. For example, at Fort Beauséjour - Fort Cumberland National Historic Site, Canada, previously undiscovered archaeological features below forest canopy have been mapped that are related to the siege of the Fort in 1755. Features that could not be distinguished on the ground or through aerial photography were identified by overlaying hillshades of the DEM created with artificial illumination from various angles. With lidar, the ability to produce high-resolution datasets quickly and relatively cheaply can be an advantage. Beyond efficiency, its ability to penetrate forest canopy has led to the discovery of features that were not distinguishable through traditional geo-spatial methods and are difficult to reach through field surveys, as in work at Caracol by Arlen Chase and his wife Diane Zaino Chase. The intensity of the returned signal can be used to detect features buried under flat vegetated surfaces such as fields, especially when mapping using the infrared spectrum. The presence of these features affects plant growth and thus the amount of infrared light reflected back. In 2012, lidar was used by a team attempting to find the legendary city of La Ciudad Blanca in the Honduran jungle. During a seven-day mapping period, they found evidence of extensive man-made structures that had eluded ground searches for hundreds of years. In June 2013 the rediscovery of the city of Mahendraparvata was announced. In another study, lidar was used to reveal stone walls, building foundations, abandoned roads, and other features of the landscape in southern New England, USA that had been obscured in aerial photography by the region's dense forest canopy. In May 2012, lidar was used to locate a previously unknown ruined city in the La Mosquitia region of Honduras. In Cambodia Lidar data were used by Demian Evans and Roland Fletcher to reveal anthropogenic changes to Angkor landscape

Autonomous vehicles

Autonomous vehicles use lidar for obstacle detection and avoidance to navigate safely through environments, using rotating laser beams. Cost map or point cloud outputs from the lidar sensor provide the necessary data for robot software to determine where potential obstacles exist in the environment and where the robot is in relation to those potential obstacles. Singapore's Singapore-MIT Alliance for Research and Technology (SMART) is actively developing technologies for autonomous lidar vehicles. Examples of companies that produce lidar sensors commonly used in robotics or vehicle automation are Sick and Hokuyo. Examples of obstacle detection and avoidance products that leverage lidar sensors are the Autonomous Solution, Inc. Forecast 3D Laser System and Velodyne HDL-64E.

The very first generations of automotive adaptive cruise control systems used only lidar sensors.

Approaches of processing LIDAR DATA

Below mentioned are various approaches of processing LIDAR data and utilizing it along with data from other sensors through sensor fusion to detect the vehicle environment conditions.

GRID based processing using 3D LIDAR and fusion with RADAR measurement

In this method, proposed by Philipp Lindner and Gerd Wanielik, laser data is processed using a multidimensional occupancy grid. Data from a 4 layer laser is pre-processed on signal level and then processed on higher level to extract the features of the obstacles. A combination of 2 and 3-dimensional grid structure is utilized and the space in these structures are tessellated into several discrete cells. This way huge amount of raw measurement data can be handled effectively by collecting them in spatial containers, which are the cells of the evidence grid. Each cell is associated with a probability measure that identifies the cell occupation. This probability is calculated by using the range measurement of lidar sensor obtained over time and a new range measurement, that is using Bayes' theorem the latest and previous range measurements of lidar sensor are related. A two dimensional grid can observe an obstacle in front of it, but cannot observe the space behind the obstacle. That's why state of occupancy behind the obstacle is, hence the unknown state is assigned a probability of 0.5. By introducing the third dimension or in other terms using a multi-layer laser, the spatial configuration of an object could be mapped into the grid structure up to a complexity. This is achieved by transferring the measurement points into a 3 dimensional grid. The grid cells which are occupied will possess a probability greater than 0.5 and the mapping would be color coded based on the probability. The cells which are not occupied will possess a probability less than 0.5 and this area will usually be white space. This measurement is then transformed to grid coordinate system by using the sensor position on the vehicle and the vehicle position in the world coordinate system. The coordinates of the sensor depends upon its location on the vehicle and the coordinates of the vehicle is computed using egomotion estimation, which is estimating the vehicle motion relative to rigid scene. But for this method the grid profile must be defined. The grid cells touched by the laser beam when transmitted are calculated by applying Bresenham's line algorithm. To obtain the spatial extended structure a connected component analysis of these cells is performed. This information is then passed on to a rotating caliper algorithm to obtain the spatial characteristics of the object. In addition to the LIDAR detection, RADAR data obtained by using two short range radars is integrated to get the dynamic properties such as velocity of the object. The measurements are assigned to the object using a potential distance function.

Advantages and Disadvantages

The geometric features of the objects are extracted efficiently, from the measurements obtained by the 3D occupancy grid, using rotating caliper algorithm. Fusing the radar data to the lidar measurements give information about the dynamic properties of the obstacle such as velocity and location of the obstacle with respect to the sensor location which helps the vehicle or the driver decide the action to be performed in order to ensure safety. The only concern is the computational requirement to implement this data processing technique. It can be implemented in real time and has been proven efficient if the 3D occupancy grid size is considerable restricted. But this can be improved to an even wider range by using dedicated spatial datastructures that manipulate the spatial data more effectively, for the 3D grid representation.

Fusion of 3D LIDAR and color camera for multiple object detection and tracking

The framework proposed in this method by Soonmin Hwang et al., is split into four steps. First, the data from camera and 3D LIDAR is given as input to the system. Both inputs from LIDAR and camera are parallelly obtained and the color image from the camera is calibrated with the LIDAR. To improve the efficiency, horizontal 3D point sampling is applied as pre-processing. Second, the segmentation stage is where the entire 3D points are divided into several groups per the distance from the sensor and local planes from close plane to far plane are sequentially estimated. The local planes are estimated using statistical analysis. The group of points closer to the sensor are used to compute the initial plane. By using the current local plane, the next local plane is estimated by iterative update. The object proposals in the 2D image are used to separate foreground objects from background. For faster and accurate detection and tracking Binarized Normed Gradients for Objectness Estimation at 300fps is used. BING is a combination of normed gradient and its binarized version which speeds up the feature extraction and testing process, to estimate the objectness of an image window. This way the foreground and background objects are separated. To form objects after estimating the objectness of an image using BING, the 3D points are grouped or clustered. Clustering is done using DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm which could be robust due to its less-parametric characteristic. Using the clustered 3D points, i.e. 3D segment, more accurate region-of-interests (RoIs) are generated by projecting 3D points on the 2D image. The third step is detection, which is broadly divided into two parts. First is object detection in 2D image which is achieved using Fast R-CNN as this method doesn't need training and it also considers an image and several regions of interest. Second is object detection in 3D space which is done by using the spin image method. This method extracts local and global histograms to represent a certain object. To merge the results of 2D image and 3D space object detection, same 3D region is considered and two independent classifiers from 2D image and 3D space are applied to the considered region. Scores calibration is done to get a single confidence score from both detectors. This single score is obtained in the form of probability. The final step is tracking. This is done by associating moving objects in present and past frame. For object tracking, segment matching is adopted. Features such as mean, standard deviation, quantized color histograms, volume size and number of 3D points of a segment are computed. Euclidean distance is used to measure differences between segments. To judge the appearance and disappearance of an object, similar segments (obtained based on the Euclidean distance) from two different frames are taken and the physical distance and dissimilarity scores are calculated. If the scores go beyond a range for every segment in previous frame, the object being tracked is considered to have disappeared.

Advantages and Disadvantages

The advantages of this method are Using 2D image and 3D data together, F l-score (which gives a measure of test's accuracy), average precision (AP) are higher than that when only 3D data from LIDAR is used. These scores are conventional measurements which judge the framework. The drawback of this method is the usage of BING for object proposal estimation as BING predicts a small set of object bounding boxes.

Obstacle detection and road environment recognition using LIDAR

This method proposed by Kun Zhou et al. not only focuses on object detection and tracking but also recognizes lane marking and road features. As mentioned earlier the LIDAR systems use rotating hexagonal mirrors which split the laser beam into 6 beams. The upper 3 layers are used to detect the forward objects such as vehicles and roadside objects. The sensor is made of weather-robust head. The data detected by LIDAR are clustered to several segments and tracked by Kalman filter. Data clustering here is done based on characteristics of each segment based on object model, which distinguish different objects such as vehicles, signboards etc. These characteristics include the dimensions of the object etc. The reflectors on the rear edges of vehicles are used to differentiate vehicles from other objects. Object tracking is done using a 2-stage Kalman filter considering the stability of tracking and the accelerated motion of objects LIDAR reflective intensity data is also used for curb detection by making use of robust regression to deal with occlusions The road marking is detected using a modified Otsu method by distinguishing rough and shiny surfaces.

Advantages

The advantages of this method Roadside reflectors which indicate lane border are sometimes hidden due to various reasons. Therefore, other information is needed to recognize the road border. The LIDAR used in this method can measure the reflectivity from the object. Hence, with this data road border can also be recognized. Also the usage of sensor with weather-robust head helps detecting the objects even in bad weather conditions.

Advantages of using LIDAR measurement

LIDAR measurements help identify the spatial structure of the obstacle. This helps distinguish objects based on size and estimate the impact of driving over it.

LIDAR system provides better range and a large field of view which helps detecting obstacles on the curves. This is one major advantage over RADAR systems which has a narrow field of view. Fusion of LIDAR measurement with different sensors makes the system robust and useful in real-time applications as LIDAR dependent system will not be able to estimate the dynamic information about the detected object.

It has been shown that lidar can be manipulated, such that self-driving cars are tricked into taking evasive action.

Biology and conservation

Lidar has also found many applications in forestry. Canopy heights, biomass measurements, and leaf area can all be studied using airborne lidar systems. Similarly, lidar is also used by many industries, including Energy and Railroad, and the Department of Transportation as a faster way of surveying. Topographic maps can also be generated readily from lidar, including for recreational use such as in the production of orienteering maps.

In addition, the Save-the-Redwoods League is undertaking a project to map the tall redwoods on the Northern California coast. Lidar allows research scientists to not only measure the height of previously unmapped trees but to determine the biodiversity of the redwood forest. Stephen Sillett, who is working with the League on the North Coast lidar project, claims this technology will be useful in directing future efforts to preserve and protect ancient redwood trees.

Geology and soil science

High-resolution digital elevation maps generated by airborne and stationary lidar have led to significant advances in geomorphology (the branch of geoscience concerned with the origin and evolution of the Earth surface topography). The lidar abilities to detect subtle topographic features such as river terraces and river channel banks, to measure the land-surface elevation beneath the vegetation canopy, to better resolve spatial derivatives of elevation, and to detect elevation changes between repeat surveys have enabled many novel studies of the physical and chemical processes that shape landscapes. In 2005 the Tour Ronde in the Mont Blanc massif became the first high alpine mountain on which lidar was employed to monitor the increasing occurrence of severe rock-fall over large rock faces allegedly caused by climate change and degradation of permafrost at high altitude.

In geophysics and tectonics, a combination of aircraft-based lidar and GPS has evolved into an important tool for detecting faults and for measuring uplift. The output of the two technologies can produce extremely accurate elevation models for terrain - models that can even measure ground elevation through trees. This combination was used most famously to find the location of the Seattle Fault in Washington, United States. This combination also measures uplift at Mt. St. Helens by using data from before and after the 2004 uplift. Airborne lidar systems monitor glaciers and have the ability to detect subtle amounts of growth or decline. A satellite-based system, the NASA ICESat, includes a lidar sub-system for this purpose. The NASA Airborne Topographic Mapper is also used extensively to monitor glaciers and perform coastal change analysis. The combination is also used by soil scientists while creating a soil survey. The detailed terrain modeling allows soil scientists to see slope changes and landform breaks which indicate patterns in soil spatial relationships.

Atmospheric remote sensing and meteorology

Initially based on ruby lasers, lidar for meteorological applications was constructed shortly after the invention of the laser and represent one of the first applications of laser technology. Lidar technology has since expanded vastly in capability and lidar systems are used to perform a range of measurements that include profiling clouds, measuring winds, studying aerosols and quantifying various atmospheric components. Atmospheric components can in turn provide useful information including surface pressure (by measuring the absorption of oxygen or nitrogen), greenhouse gas emissions (carbon dioxide and methane), photosynthesis (carbon dioxide), fires (carbon monoxide) and humidity (water vapor). Atmospheric lidars can be either ground-based, airborne or satellite depending on the type of measurement.

Atmospheric lidar remote sensing works in two ways -

1. by measuring backscatter from the atmosphere, and
2. by measuring the scattered reflection off the ground (when the lidar is airborne) or other hard surface.

Backscatter from the atmosphere directly gives a measure of clouds and aerosols. Other derived measurements from backscatter such as winds or cirrus ice crystals require careful selecting of the wavelength and/or polarization detected. Doppler Lidar and Rayleigh Doppler Lidar are used to measure temperature and/or wind speed along the beam by measuring the frequency of the backscattered light. The Doppler broadening of gases in motion allows the determination of properties via the resulting frequency shift. Scanning lidars, such as the conical-scanning NASA HARLIE LIDAR, have been used to measure atmospheric wind velocity. The ESA wind mission ADM-Aeolus will be equipped with a Doppler lidar system in order to provide global measurements of vertical wind profiles. A doppler lidar system was used in the 2008 Summer Olympics to measure wind fields during the yacht competition.

Doppler lidar systems are also now beginning to be successfully applied in the renewable energy sector to acquire wind speed, turbulence, wind veer and wind shear data. Both pulsed and continuous wave systems are being used. Pulsed systems use signal timing to obtain vertical distance resolution, whereas continuous wave systems rely on detector focusing.

The term eolics has been proposed to describe the collaborative and interdisciplinary study of wind using computational fluid mechanics simulations and Doppler lidar measurements.

The ground reflection of an airborne lidar gives a measure of surface reflectivity (assuming the atmospheric transmittance is well known) at the lidar wavelength. However, the ground reflection is typically used for making absorption measurements of the atmosphere. "Differential absorption lidar" (DIAL) measurements utilize two or more closely spaced (<1 nm) wavelengths to factor out surface reflectivity as well as other transmission losses, since these factors are relatively insensitive to wavelength. When tuned to the appropriate absorption lines of a particular gas, DIAL measurements can be used to determine the concentration (mixing ratio) of that particular gas in the atmosphere. This is referred to as an Integrated Path Differential Absorption (IPDA) approach, since it is a measure of the integrated absorption along the entire lidar path. IPDA lidars can be either pulsed or CW and typically use two or more wavelengths. IPDA lidars have been used for remote sensing of carbon dioxide and methane.

Synthetic array lidar allows imaging lidar without the need for an array detector. It can be used for imaging Doppler velocimetry, ultra-fast frame rate (MHz) imaging, as well as for speckle reduction in coherent lidar. An extensive lidar bibliography for atmospheric and hydrospheric applications is given by Grant.

Scheimpflug principles

Another lidar technique for atmospheric remote sensing has emerged. It is based on Scheimpflug principles referred to as scheimpflug lidar (slidar).

"The implication of the Scheimpflug principle is that when a laser beam is transmitted into the atmosphere, the backscattering echo of the entire illuminating probe volume is still in focus simultaneously without diminishing the aperture as long as the object plane, image plane and the lens plane intersect with each other". A two dimensional CCD/CMOS camera is used to resolve the backscattering echo of the transmitted laser beam.

Thus as in the case of conventional lidar technologies continuous wave light sources such as diode lasers can be employed for remote sensing instead of using complicated nano second pulse light sources. The SLidar system is also a robust and inexpensive system based on compact laser diodes and array detectors.

Law enforcement

Lidar speed guns are used by the police to measure the speed of vehicles for speed limit enforcement purposes.

Military

Few military applications are known to be in place and are classified (like the lidar-based speed measurement of the AGM-129 ACM stealth nuclear cruise missile), but a considerable amount of research is underway in their use for imaging. Higher resolution systems collect enough detail to identify targets, such as tanks. Examples of military applications of lidar include the Airborne Laser Mine Detection System (ALMDS) for counter-mine warfare by Areté Associates.

A NATO report (RTO-TR-SET-098) evaluated the potential technologies to do stand-off detection for the discrimination of biological warfare agents. The potential technologies evaluated were Long-Wave Infrared (LWIR), Differential Scattering (DISC), and Ultraviolet Laser Induced Fluorescence (UV-LIF). The report concluded that : Based upon the results of the lidar systems tested and discussed above, the Task Group recommends that the best option for the near-term (2008–2010) application of stand-off detection systems is UV LIF . However, in the long-term, other techniques such as stand-off Raman spectroscopy may prove to be useful for identification of biological warfare agents.

Short-range compact spectrometric lidar based on Laser-Induced Fluorescence (LIF) would address the presence of bio-threats in aerosol form over critical indoor, semi-enclosed and outdoor venues like stadiums, subways, and airports. This near real-time capability would enable rapid detection of a bioaerosol release and allow for timely implementation of measures to protect occupants and minimize the extent of contamination.

The Long-Range Biological Standoff Detection System (LR-BSDS) was developed for the US Army to provide the earliest possible standoff warning of a biological attack. It is an airborne system carried by a helicopter to detect man-made aerosol clouds containing biological and chemical agents at long range. The LR-BSDS, with a detection range of 30 km or more, was fielded in June 1997. Five lidar units produced by the German company Sick AG were used for short range detection on Stanley, the autonomous car that won the 2005 DARPA Grand Challenge.

A robotic Boeing AH-6 performed a fully autonomous flight in June 2010, including avoiding obstacles using lidar.

Mining

Lidar is used in the mining industry for various tasks. The calculation of ore volumes is accomplished by periodic (monthly) scanning in areas of ore removal, then comparing surface data to the previous scan.

Lidar sensors may also be used for obstacle detection and avoidance for robotic mining vehicles such as in the Komatsu Autonomous Haulage System (AHS) used in Rio Tinto's Mine of the Future.

Physics and astronomy

A worldwide network of observatories uses lidars to measure the distance to reflectors placed on the moon, allowing the position of the moon to be measured with mm precision and tests of general relativity to be done. MOLA, the Mars Orbiting Laser Altimeter, used a lidar instrument in a Mars-orbiting satellite (the NASA Mars Global Surveyor) to produce a spectacularly precise global topographic survey of the red planet.

In September, 2008, the NASA Phoenix Lander used lidar to detect snow in the atmosphere of Mars.

In atmospheric physics, lidar is used as a remote detection instrument to measure densities of certain constituents of the middle and upper atmosphere, such as potassium, sodium, or molecular nitrogen and oxygen. These measurements can be used to calculate temperatures. Lidar can also be used to measure wind speed and to provide information about vertical distribution of the aerosol particles.

At the JET nuclear fusion research facility, in the UK near Abingdon, Oxfordshire, lidar Thomson Scattering is used to determine Electron Density and Temperature profiles of the plasma.

Rock mechanics

LiDAR has been widely used in rock mechanics for rock mass characterization and slope change detection. Some important geomechanical properties from the rock mass can be extracted from the 3D point clouds obtained by means of the LiDAR. Some of these properties are:

Some of these properties have been used to assess the geomechanical quality of the rock mass through the RMR index. Moreover, as the orientations of discontinuities can be extracted using the existing methodologies, it is possible to assess the geomechanical quality of a rock slope through the SMR index. In addition to this, the comparison of different 3D point clouds from a slope acquired at different times allows to study the changes produced on the scene during this time interval as a result of rockfalls or any other landsliding processes.

Robotics

Lidar technology is being used in robotics for the perception of the environment as well as object classification. The ability of lidar technology to provide three-dimensional elevation maps of the terrain, high precision distance to the ground, and approach velocity can enable safe landing of robotic and manned vehicles with a high degree of precision. Refer to the Military section above for further examples.

Spaceflight

Lidar is increasingly being utilized for rangefinding and orbital element calculation of relative velocity in proximity operations and stationkeeping of spacecraft. Lidar has also been used for atmospheric studies from space. Short pulses of laser light beamed from a spacecraft can reflect off of tiny particles in the atmosphere and back to a telescope aligned with the spacecraft laser. By precisely timing the lidar 'echo,' and by measuring how much laser light is received by the telescope, scientists can accurately determine the location, distribution and nature of the particles. The result is a revolutionary new tool for studying constituents in the atmosphere, from cloud droplets to industrial pollutants, that are difficult to detect by other means."

Surveying

Airborne lidar sensors are used by companies in the remote sensing field. They can be used to create a DTM (Digital Terrain Model) or DEM (Digital Elevation Model); this is quite a common practice for larger areas as a plane can acquire 3–4 km wide swaths in a single flyover. Greater vertical accuracy of below 50 mm can be achieved with a lower flyover, even in forests, where it is able to give the height of the canopy as well as the ground elevation. Typically, a GNSS receiver configured over a georeferenced control point is needed to link the data in with the WGS (World Geodetic System).

Transport

LiDAR has been used in the railroad industry to generate asset health reports for asset management and by departments of transportation to assess their road conditions. CivilMaps.com is a leading company in the field. Lidar has been used in adaptive cruise control (ACC) systems for automobiles. Systems such as those by Siemens and Hella use a lidar device mounted on the front of the vehicle, such as the bumper, to monitor the distance between the vehicle and any vehicle in front of it. In the event the vehicle in front slows down or is too close, the ACC applies the brakes to slow the vehicle. When the road ahead is clear, the ACC allows the vehicle to accelerate to a speed preset by the driver. Refer to the Military section above for further examples.

Wind farm optimization

Lidar can be used to increase the energy output from wind farms by accurately measuring wind speeds and wind turbulence. Experimental lidar systems can be mounted on the nacelle of a wind turbine or integrated into the rotating spinner to measure oncoming horizontal winds, winds in the wake of the wind turbine, and proactively adjust blades to protect components and increase power. Lidar is also used to characterise the incident wind resource for comparison with wind turbine power production to verify the performance of the wind turbine by measuring the wind turbine's power curve. Wind farm optimization can be considered a topic in applied eolics.

Solar photovoltaic deployment optimization

Lidar can also be used to assist planners and developers in optimizing solar photovoltaic systems at the city level by determining appropriate roof tops and for determining shading losses. Recent works focus on buildings' facades solar potential estimation, or by incorporating more detailed shading losses by considering the influence from vegetation and larger surrounding terrain.

Video games

Racing game iRacing features scanned tracks, resulting in bumps with millimeter precision in the in-game 3D mapping environment.

Other uses

The video for the song "House of Cards" by Radiohead was believed to be the first use of real-time 3D laser scanning to record a music video. The range data in the video is not completely from a lidar, as structured light scanning is also used.

Alternative technologies

Recent development of Structure From Motion (SFM) technologies allows delivering 3D images and maps based on data extracted from visual and IR photography. The elevation or 3D data is extracted using multiple parallel passes over mapped area, yielding both visual light images and 3D structure from the same sensor, which is often a specially chosen and calibrated digital camera.