Automatic classification of lidar point clouds in a railway environment

AUTOMATIC CLASSIFICATION OF LIDAR POINT CLOUDS IN A

RAILWAY ENVIRONMENT

MOSTAFA ARASTOUNIA March, 2012

SUPERVISORS:

Dr. Ir. S.J. Oude Elberink

Dr. K. Khoshelham

Thesis submitted to the Faculty of Geo-Information Science and Earth Observation of the University of Twente in partial fulfilment of the

requirements for the degree of Master of Science in Geo-information Science and Earth Observation.

Specialization: GEOINFORMATICS

SUPERVISORS:

Dr. Ir. S.J. Oude Elberink Dr. K. Khoshelham

THESIS ASSESSMENT BOARD:

Prof. Dr. Ir. M.G. Vosselman (Chair)

Dr. Ir. D. Lutters (External Examiner, University of Twente)

AUTOMATIC CLASSIFICATION OF LIDAR POINT CLOUDS IN A

RAILWAY ENVIRONMENT

MOSTAFA ARASTOUNIA

Enschede, The Netherlands, March, 2012

DISCLAIMER

This document describes work undertaken as part of a programme of study at the Faculty of Geo-Information Science and

Earth Observation of the University of Twente. All views and opinions expressed therein remain the sole responsibility of the

author, and do not necessarily represent those of the Faculty.

The purpose of this research is to classify points on rails, contact and catenary wires from 3D lidar point clouds in a railway environment. Two dataset were used for this research i.e. a terrestrial dataset and an airborne dataset. The classification of objects in a railway environment is important as railway companies tend to have a database of their infrastructures due to maintenance and design purposes. In addition, by automating this process companies reduce the cost of maintenance substantially and also by using 3D lidar data they obtain more precise data of their infrastructures.

The idea is to do a knowledge based classification which takes advantage of our knowledge of railway environment to classify points on the objects of interest. First characteristics of objects of interest are inspected and based on them constraints are defined. To do the classification, two different methods are developed. First Template Matching (TM) is developed based on the idea of defining a pattern of rails and wires and classifying points by seeking this pattern in data. Next, Region Growing (RG) method is

developed based on the idea of finding small parts of rails and wires as seed points and then looking for the rest of them in the neighbourhood of already classified ones based on their characteristics.

Results of both methods were evaluated for completeness, correctness and speed by visual inspection and statistical analysis. Both TM and RG methods were applied for terrestrial dataset and they both managed to classify majority of rail and wire points (completeness). Results of RG contained less wrong

classifications than that of TM (correctness) and RG did the classification 3 times as fast as TM (speed).

As the result of RG in terrestrial dataset was better than that of TM, only RG method was applied for airborne dataset to check its performance to classify points on curved rails. RG method managed to classify majority of rail and wire points with no considerable wrong classifications.

To conclude, the idea of knowledge based classification of points on the objects in 3D lidar point clouds is feasible as both methods of TM and RG succeeded to do so. However, RG was more promising and more efficient than TM due to its more accurate results and higher speed algorithm. Furthermore, both methods still can be improved to be more efficient i.e. getting more accurate results in a shorter period of time. Finally as the classification is done just based on characteristics of objects, it can be done for other objects in a railway environment as well as objects in other environments e.g. urban and roads.

Keywords

Mobile laser scanning (MLS), lidar, Knowledge based classification, 3D point clouds, Region Growing,

Template Matching

I owe my deepest gratitude to my supervisors Dr. Ir. S.J. Oude Elberink and Dr. K. Khoshelham for guiding me through my MSc thesis. My special thanks for their patience, knowledge and support.

I would like to thank all staff in GFM Domain, ITC Faculty, University of Twente, The Netherlands for their support which made this thesis possible. I offer my sincere gratitude to Prof. Dr. Ir. M.G. Vosselman for his help, advice and time.

I also would like to thank my dear friends. It was a great experience to study with people from different countries with different language, culture, interest, attitude, music and food.

I do not have a special word to show my thankfulness to my family. This thesis would not have been possible without help, support, guidance and encouragement of my dear family who always supported me in hard circumstances.

Finally, I dedicate this thesis to my dear parents Mohammad Ali Arastounia and Sakineh Bahrami Hidaji

and my dear brother Mehrdad Arastounia.

1. Introduction ... 1

1.1. Motivation and problem statement ...1

1.2. Research identification ...2

1.3. Thesis structure ...3

2. Literature review ... 5

2.1. Principles of laser scanning ...5

2.2. Road extraction from 3D laser scanned data ...5

2.3. Building extraction from 3D laser scanned data ...5

2.4. Power line extraction from 3D laser scanned data ...6

2.5. Object extraction in an urban area from 3D laser scanned data ...6

2.6. Object extraction in a railway environment from 3D laser scanned data ...6

3. Materials ... 9

3.1. First dataset: terrestrial laser scanned data ...9

3.2. Second dataset: airborne laser scanned data ... 10

3.3. Softwares used for programming and visualization ... 11

4. Methodology and implementation of 3D lidar data classification ... 13

4.1. General approach ... 13

4.2. Characteristics of objects of interest ... 14

4.3. Applied constraints ... 14

4.4. Template Matching method ... 15

4.5. Region Growing method ... 25

5. Results of 3D lidar data classification ... 29

5.1. Results of Template Matching method on terrestrial laser scanned data ... 29

5.2. Results of Region Growing method on terrestrial laser scanned data ... 36

5.3. Results of Region Growing method on airborne laser scanned data ... 42

6. Discussion about 3D lidar classification ... 45

6.1. Discussion about common steps of Template Matching and Region Growing methods ... 45

6.2. Discussion about Template Matching ... 47

6.3. Discussion about Region Growing ... 49

6.4. Comparion between Template Matching and Region Growing ... 51

7. Conclusions and recommendations for future classification of 3D lidar data... 53

7.1. Answers to research questions ... 53

7.2. Conclusions ... 54

7.3. Recommendations ... 55

List of references ... 57

Appendix 1: Description of wires in a typical railway environment ... 59

Appendix 2: The railway environment of the terrestrial dataset in Google maps ... 61

Appendix 3: The railway environment of the airborne dataset in Google maps ... 65

Appendix 4: A closer view of parts of terrestrial dataset where different methods were applied ... 67

Appendix 5: Other rail patterns and their results in Template Matching method ... 69

Figure 3-1. An oblique view of the provided dataset of a railway environment in The Netherlands ... 9

Figure 3-2. Two gap areas caused by shadow effect of the outer rails ... 10

Figure 3-3. Front View of the airborne laser scanned data ... 11

Figure 4-1. Flowchart of Template Matching method ... 15

Figure 4-2. Grey arrows point to height jumps caused by rails while the white arrow points to other parts of track bed without any height jumps ... 17

Figure 4-3. Using 10, 80 and 90 percentiles to remove noises for implementation of height jump detection method ... 18

Figure 4-4. Big height jump caused by railway station platform V.S. small height jumps caused by rails .... 19

Figure 4-5. Relative height of wires to rails ... 20

Figure 4-6. In top view, wires shown by yellow and brown colour lie between rails shown by green colour ... 20

Figure 4-7. Rail and wire pattern in a railway environment ... 22

Figure 4-8. Rail and wire pattern as a greyscale image ... 22

Figure 4-9. Flowchart of Region Growing method ... 25

Figure 5-1. Result of pre classification of points based on their heights ... 29

Figure 5-2. Result of first classification of rails including some wrong classifications and some not- classified points ... 30

Figure 5-3. The extent of railway environment in Google maps ... 30

Figure 5-4. Result of first classification of rails and wires. Wrong classifications are shown by ovals and arrows point to areas where no rail and wire points are classified ... 31

Figure 5-5. The greyscale image generated from 3D point clouds ... 32

Figure 5-6. The Result of Template Matching containing rails and wires ... 32

Figure 5-7. Result of Template Matching containing only rail pattern ... 33

Figure 5-8. Filling gap areas by Hough Transform method ... 33

Figure 5-9. Result of Template Matching with filled gaps. Grey points represent new candidate rail points ... 33

Figure 5-10. Result of converting greyscale image back to 3D point clouds ... 34

Figure 5-11. Result of Template Matching after verifying candidate rail points ... 34

Figure 5-12. Final result of Template Matching method with some points on a tower which are wrongly classified as catenary wires ... 35

Figure 5-13. Result of height jump detection method for the first 5 m of rails with some points wrongly classified as rails and some area where no rail points are classified ... 36

Figure 5-14. Result of Hough Transform method in which seed points of rail growing are recognized and differentiated from points on track bed ... 37

Figure 5-15. Result of classification of the first 5 meters in which rail and wire points are classified separately... 38

Figure 5-16. Result of classification of the first 5 meters in which each pair of rails and their corresponding contact wire are recognized separately ... 38

Figure 5-17. Result of classification of the first 5 meters in which all rails and wires are recognized differently ... 39

Figure 5-18. Growing pattern of rails in top view ... 39

Figure 5-19. Growing pattern of contact wires ... 40

recognized separately ... 41

Figure 5-22. Final result of Region Growing in which all rails and wires are recognized differently ... 41

Figure 5-23. Result of height jump detection method for terrestrial dataset ... 42

Figure 5-24. Result of height jump detection method for airborne dataset ... 42

Figure 5-25. Seed points of different rails and wires in airborne dataset which were extracted manually ... 43

Figure 5-26. Growing pattern of rails in airborne dataset ... 43

Figure 5-27. Growing pattern of wires in airborne dataset ... 44

Figure 5-28. Final result of RG on airborne dataset ... 44

Figure 6-1. Appearance of rails in high resolution terrestrial point clouds ... 46

Figure 6-2. Appearance of rails in low resolution airborne point clouds ... 47

Figure 6-3. Accumulative histogram of geometric distances of classified rail points to 3D rail models for Template Matching (Diaz Benito, To be published in March 2012)... 48

Figure 6-4. Accumulative histogram of geometric distances of classified rail points to 3D rail models by

Region Growing (Diaz Benito, To be published in March 2012) ... 50

1. INTRODUCTION

1.1. Motivation and problem statement

The purpose of this research is to automatically classify points on objects of interest from 3D lidar point clouds in a railway environment. Two datasets were used in this research. First was a terrestrial laser scanned data and the second was an airborne laser scanned data, both acquired by mobile laser scanning (MLS) in the railway environment but in two different areas in The Netherlands. The main aspects of this research are:

1. The need to have geo information about objects in a railway environment 2. Applying an automatic method

3. Using 3D mobile laser scanned data

In the following paragraphs, the above mentioned aspects are discussed.

A lot of railway companies want to have 3D data about their infrastructures including railways, wires, towers, signs, stations and other objects in the railway environment. In general, railway companies tend to have a database of their railway infrastructures. There are several motivations for having a database of railway infrastructures like for maintenance and design purposes.

For the purpose of maintenance, having data about infrastructures helps companies to monitor them from time to time and check their deformation or if the rails are broken or the distance between them, gauge, has changed. Problematic ecosystem is another instance that can jeopardize the safety of the railway environment. “Unchecked vegetation growth (e.g. massive tree roots) along the rail corridor can

undermine the stability of the sub-grade material. On the other hand, some vegetation can also provide a needed mechanism to resist the effects of erosion. However, depending on their location, trees along the rail corridor can also interfere with passing trains by obstructing the free passage of rail vehicles” (Morgan, 2009). This is absolutely crucial as any deformation of rails, change of gauge or problematic ecosystem will result in deadly catastrophes. Thus it highlights the importance and necessity of the maintenance. In addition to maintenance, future designs can also be improved. In other words, having 3D data about existing railway infrastructure provides the opportunity of finding its problems and defects and solving them for future designs.

One of the motivations of this research is to automate the classification of points on the objects of interest in a railway environment. Currently in some countries the task of maintenance like inspection of rails is done manually i.e. by hiring some people to go to the railway environment and make sure that the rails are not broken or deformed. This makes the maintenance very expensive as the railway company has to pay for the hired people. Thus automating this task helps railway companies to decrease the cost and increase the speed of maintenance substantially. As a result, monitoring the railway environment and in general maintenance will be done more frequently which culminates in higher safety of travelling by trains and that is what railway companies are seeking for.

A good starting point to develop an automatic algorithm to recognize objects is to inspect how humans do

that. In fact, human being recognizes an object by considering its characteristics like size, shape, colour,

words, humans have a database of objects they are familiar with and to recognize another object, he tries to find the object in his database which is the most similar with the one he observed. This strategy is the core idea of this research i.e. teaching the computer the characteristics of our objects of interest as well as teaching it how to extract them.

The following is the motivation of using MLS systems to collect data of railway environment. Some advantages of using MLS systems are high speed of data acquisition as well as high accuracy and resolution of data. “The main benefit of terrestrial laser scanning as the imaging component compared with

photogrammetric systems is the direct acquisition of 3D information” (Vosselman et al., 2010) with respect to this issue that in mobile laser scanning “Data Processing and product supply are principally identical to procedures in the terrestrial laser scanning static mode” (Vosselman et al., 2010).

Furthermore, MLS is less dependent on weather conditions than aerial photogrammetry and is cheaper than some other methods of data acquisition like ground surveying. As the output of MLS is lidar point clouds consisting of points with 3D coordinates, Antonarakis et al. (2008) stated “Light Detection and Ranging (LiDAR) provides high resolution horizontal and vertical spatial point cloud data and is increasingly being used in a number of applications and disciplines, which have concentrated on the exploit and manipulation of the data using mainly its three dimensional nature. LiDAR information potential is made even greater though, with its consideration of intensity”.

1.2. Research identification

The purpose of this research is to automatically classify points on objects of interest from lidar data in a railway environment. The objects of interest are rails, contact wires and catenary wires and their

classification is done by developing an algorithm that is able to recognize objects of interest by inspecting how these objects look like in the MLS data and also by inspecting their spatial relation to each other and to other objects. For a detailed description of wires in a typical railway environment see Appendix 1.

The result which is a classified point cloud can be used for further analysis e.g. 3D modelling, making measurements, checking change of gauge and inspecting continuity of rails. For this research particularly, the classified point clouds of the terrestrial laser scanned data is used for 3D modelling of rails by Diaz Benito (To be published in March 2012).

1.2.1. Research objectives

The objective is to classify points on the objects of interest, each in one class and the remained points are classified in another class as other objects. As a result, there will be four classes and each point in the lidar point cloud will be labelled as one of the classes of rails, contact wires, catenary wires or other objects.

1.2.2. Research questions

The following is the list of research questions:

1. What characteristics of the objects of interest can be used for classification?

2. How to define and implement the constraints for classification?

3. How to assess the result of classification?

1.2.3. Innovation aimed at

The novelty of this research is how to use our knowledge of the railway environment to classify the objects of interest. In other words, the innovation is aimed at the way the constraints are defined and implemented for classification of MLS data in a railway environment.

1.3. Thesis structure

This thesis consists of seven chapters. The first chapter is the introduction which explains the motivation, problem statement, research objectives, research questions and innovation. Then literature review is done in Chapter 2 which is followed by Chapter 3 including a description of data and soft wares used for this research. Chapter 4 starts with the description of characteristics of objects of interest and the applied constraints and is pursued by the explanation of the two developed methods i.e. Template Matching and Region Growing as well as their implementations. Chapter 5 presents the results of applying Template Matching and Region Growing methods. In Chapter 6, a discussion about the methodology and results is presented. Finally the last chapter answers to research questions and makes some conclusions and recommendations about potential follow ups and applied methods.

2. LITERATURE REVIEW

A large number of researches were done previously to extract various objects from laser scanned data in different environments like road or urban environment. However, not much research is done to extract objects in a railway environment. The data used for these researches were either airborne or terrestrial and some used the integration of laser scanned data with images.

The following are some previous works of automatic object extraction from 3D lidar data in different environments. In cases that the extracted objects are different from objects in a railway environment, there were some points in developed method or strategy which could be used for extraction of objects in a railway environment.

2.1. Principles of laser scanning

A reference text book is written by Vosselman et al. (2010) in which data acquisition, characteristics and some applications of lidar data are explained. Vosselman et al. (2004) also review several techniques that can be used for recognizing specific structures in 3D lidar point clouds. Wehr et al. (1999) give an introduction about various topics related to laser scanning and also principles of airborne laser scanning.

Then, they explain different scanning mechanisms, integration with GPS/INS and data processing of laser scanned data.

2.2. Road extraction from 3D laser scanned data

Road environment was one of areas which were extracted using lidar data. For instance, automatic road extraction from dense urban area was done by Pagounis et al. (2007) for improving roads safety using terrestrial laser scanned data. Tiwari et al. (2009) pursued a semiautomatic approach to do so with satellite images and airborne laser altimetry. Hu et al. (2004) did the extraction of roads from urban area using lidar data integrated with high resolution images.

In addition, Pu et al. (2011) developed an automated method for recognizing and extracting structures of road inventories. He started by a rough classification to three classes as ground surface, objects on ground and objects off ground. He considered characteristics of features like position, orientation, shape, etc. as well as their topological relations like intersect and angle. Then he applied a hybrid method of data driven and model driven to recognize objects in a road environment. Finally he evaluated his method by visual performance analysis for completeness and correctness and recommended to use data with high point density, reflectance data and integrating lidar data with images.

2.3. Building extraction from 3D laser scanned data

Extraction of building is one of popular topics of previous years. For example, Pu et al. (2009) presented a

knowledge based reconstruction of building facade models from terrestrial laser scanned data. The

detection are other reasons to extract information from buildings which was done by Armesto-González et al. (2010) and Dash et al. (2004) using terrestrial laser scanned data.

2.4. Power line extraction from 3D laser scanned data

Jwa et al. (2009) developed an automatic method to reconstruct 3D power lines using airborne lidar data.

He applies three constraints: height of transmission line above terrain, direction of power line and the location of towers. In this paper, Hough Transform algorithm is also used to detect linear features. This research was followed by Jwa et al. (2010) to improve the results of previous methods which used airborne lidar data as well.

2.5. Object extraction in an urban area from 3D laser scanned data

Charaniya et al. (2004) did a point-based supervised classification by aerial lidar data to classify points to 4 classes of trees, grass, roads and roofs. To do this, he used the integration of aerial lidar data with aerial images and applied constraints of height information, height variation, multiple returns and intensity data of laser data and luminance of images. At the end, he concludes that which constraints are useful to classify which objects e.g. height feature is an important classifier for terrain and height variation is important for classification of high vegetation.

Bretar et al. (2004) modelled terrain by classification of airborne laser data. He classified points as two classes of terrain and non-terrain. He followed a multistep method in which points are classified as terrain and non-terrain iteratively until the last step. He used the height feature of points and also he inspected the data locally i.e. inspecting the points in their neighbourhood. At the end, he concludes that

neighbourhood size and point cloud density are very dependent on each other.

Using terrestrial laser scanning for 3D modelling of tree structures is another example in this field (Rosell et al., 2009; Van der Zande et al., 2006). Nardinocchi et al. (2003) also presented a strategy for

classification of lidar data as terrain, building and vegetation. Antonarakis et al. (2008) proposed a supervised object based approach to classify different land covers using airborne lidar data without the need to manipulate multispectral images. Vosselman et al. (2005) discuss the possibility of using airborne laser scanned data for two purposes: change detection and semi-automated 3D mapping of urban environment.

Furthermore, Sithole et al. (2006) detected bridges in airborne laser scanned data. To do so, they proposed a geometric algorithm based on the idea that bridges can be detected in a landscape based on their cross sections (profiles). They used several dataset and concluded that the method works well for all datasets.

2.6. Object extraction in a railway environment from 3D laser scanned data

Morgan (2009) described the 3D lidar data acquisition in a railway environment. A laser scanner is mounted on a railway speeder and scans the railway environment. Then he shows that the achieved data meets survey grade criteria by providing enough details and accuracy to extract essential spatial

information of engineering, maintenance and construction applications.

Moreover, Jeon (2009) extracted and modelled railroad wires from airborne laser scanned data. He applied two different methods to extract different wires. For contact wires, he applied tracking algorithm and for catenary wires he used RANSAC algorithm. Finally he validated his methods by visual inspection and concluded that his method managed to extract and model wires completely and accurately.

Gedrange et al. (2011) did a knowledge based extraction of catenary wire masts in which he used the airborne laser data integrated with images. First by doing object based image analysis, he automatically extracted railroad track centre lines. Then by applying some spatial constraints like distance, area, height and intersection, he extracted support tubes of catenary wire masts and finally catenary wire masts. He assessed his methods by counting the number of true and wrong extraction of catenary wire masts as well as the number of them which were not extracted.

Haasnoot (2001) explains the advantages of using airborne lidar data for corridor mapping. He focuses specifically on laser scanning systems mounted on a helicopter and integrated with images. He starts with an introduction of airborne laser scanning systems, aerial cameras, helicopters, etc. Then he describes the different steps of the data acquisition process and finally specifications of derived lidar data.

Beger et al. (2011) did a data fusion of extremely high resolution aerial imagery and lidar data to combine the advantages of both datasets for automatic extraction of railroad track centrelines. He developed a three step strategy starting by object based image analysis to derive a railroad track mask from ortho imagery. He followed by classifying rail tracks laser points and then finding the rail tracks centre lines from classified laser points by RANSAC algorithm.

Neubert et al. (2008) extracted some objects in a railway environment using very high resolution helicopter

borne lidar data and ortho image data. He proposed two methods. RANSAC algorithm for linear object

extraction and a knowledge based classification method for the rest of object.

3. MATERIALS

For this research two dataset were used i.e. one terrestrial and one airborne. As mentioned in section 1.2, the result of this research which is classified point clouds was used for 3D modelling by Diaz Benito (To be published in March 2012). The main dataset on which the 3D modelling was done is terrestrial data.

On the other hand, the developed algorithm required to be checked whether they can classify points on both straight and curved rails but the terrestrial dataset only include straight rails. So the airborne dataset was used to check the capability of algorithm to classify points on curved rails. Furthermore, by using the airborne dataset, the performance of the algorithm for another dataset with lower resolution and less precise coordinates than that of terrestrial dataset was tested.

3.1. First dataset: terrestrial laser scanned data

The data for this research is 3D lidar point clouds of a railway environment which is provided by a Dutch railway company called Movares. The point cloud is obtained by registering several laser strips and each point has 3D coordinates in the Dutch national coordinate system while the precision of the coordinates is in millimetres.

The data covers a distance of about 670 meters of the railway between the cities Arnhem and Nijmegen

and near the city Elst in The Netherlands. It includes two pairs of straight rails, two contact wires, two

catenary wires, ten towers with a distance of about 67 meters from each other as well as points on all other

objects in a railway environment e.g. railway stations, trees, people, roads and so forth. Although it is

cleaned off noises like air dust and humidity, there are still small amount of noises which needs to be taken

care of. Figure 3-1 shows the terrestrial dataset from an oblique view. For its views in Google maps and

Google street views see Appendix 2.

The data was acquired by a mobile terrestrial laser scanner and its resolution, point density, is not homogeneous. For instance, the resolution in the middle of the track bed is much higher than that in the outer parts of track bed. That is so due to scanning pattern of data acquisition. As a result, the resolution close to inner rails is higher than the area close to outer rails. The average point density on track bed is 130 points in 1 square meter area on track bed.

In addition, there are some gap areas in the data due to shadow effect of the outer rails. Shadow effect happens once the body of the rail prevents the laser beams to reach the other side of the rail and this leads to a gap of points just next to the outer rails. In Figure 3-2, two white arrows point to two gap areas caused by shadow effect of the outer rails.

3.2. Second dataset: airborne laser scanned data

The airborne dataset like the terrestrial one is 3D lidar point clouds of a railway environment which is provided by Earth Observation Science Department of ITC Faculty, University of Twente, The

Netherlands. It also registered i.e. its points have 3D coordinates in Dutch national coordinate system and the precision of coordinates is in centimetres.

This dataset represents a more complicated configuration of a railway environment than that in terrestrial dataset due to the existence of more number of rails and wires, curved rails and wires which connect other rails and wires, train wagon. It covers a distance of about 80 meters of a railway environment near the central train station of the city Enschede, The Netherlands. In its point clouds, there are points only on objects in a railway environment i.e. two pairs of straight rails, four pairs of curved rails, seven contact wires, seven catenary wires, five towers and a train carriage and there are no points on objects in the surroundings of railway environment like trees, people and train station. For its views in Google maps see Appendix 3.

Figure 3-2. Two gap areas caused by shadow effect of the outer rails

This dataset is acquired by airborne laser scanning and its resolution is much lower than that of terrestrial dataset. Its point density is also not homogeneous and its resolution varies in different part of dataset. The average point density on track bed is 52 points in 1 square meter area on track bed. Due to its low

resolution and existence of train carriages on rails, there are a lot of gap areas in which there are no points on rails or wires.

Figure 3-3 shows the front view of the airborne dataset in which the big white arrow points to a train carriage standing on rails and two small grey arrows point to two towers. Wires are on top of the image and rails are evident by white lines on the track bed.

3.3. Softwares used for programming and visualization

Python and MATLAB were the two programming languages used for this research. MATLAB was mainly used due to its useful tools for image processing. Moreover, the software PCM was extensively used for the purpose of data visualization. PCM stands for Point Cloud Mapper and is developed by Prof. George Vosselman in Earth Observation and Science Department of ITC Faculty, University of Twente, The Netherlands.

Figure 3-3. Front View of the airborne laser scanned data

4. METHODOLOGY AND IMPLEMENTATION OF 3D LIDAR DATA CLASSIFICATION

4.1. General approach

The proposed approach is a knowledge based classification which takes advantage of our knowledge of the railway environment to classify points on desirable objects. Also as data contains only elevation information and no reflectance information, the algorithm requires to utilize the spatial relation between objects.

To do the classification, first the characteristics of the objects of interest are inspected. Then for each object, based on its deemed characteristics, constraints are defined. The defined constraints should be helpful to differentiate the objects of interest from other objects and from each other. In addition, constraints should be applied efficiently as defining a few weak constraints leads to a poor classification and using too many constraints make the process complicated and slow and it does not necessarily improve the results.

The main dataset is the terrestrial one on which 3D modelling was done by Diaz Benito (To be published in March 2012). The terrestrial dataset represents a simple configuration of a railway environment

including only two pairs of straight rails. While the airborne dataset represents a complicated configuration of a railway environment including four pairs of curved rails. So the airborne dataset was used to check the ability of the algorithm for classification of curved rails in a low resolution and low precision dataset.

For this research, two different methods i.e. Template matching (TM) and Region Growing (RG) were developed. Both methods were applied for terrestrial dataset and based on achieved results, it was concluded that RG was more efficient and faster than TM. Therefore, only RG method was applied for airborne dataset to check its performance for more complicated railway environment settings.

In terrestrial dataset, different sections of dataset are almost the same in the sense that they all have same objects. The only difference is the section which includes an intersection of railway and a road. Although both TM and RG methods could be run for the whole dataset, they were run for two different sections of about 100 meters long as two subsets of the whole dataset. For a closer view of the parts of terrestrial dataset in which TM and RG methods were applied see Appendix 4.

TM method could be applied in any section of dataset but it was applied in the hardest section for classification due to the existence of an intersection of railway and a road. If it manages to classify points in that section, then it will succeed to do the classification in other sections as well. On the other hand, RG method first finds a short piece of rails and wires and then finds the rest of rails and wires by following them in one direction. So it required to be applied at the beginning of dataset because if it is applied in the middle of dataset, it should follow rails and wires in two directions which is not able to do it. The result of both methods is a point cloud which its points are classified as four classes of:

1. Rails

2. Contact wires

3. Catenary wires

4. Other objects

4.2. Characteristics of objects of interest 4.2.1. Characteristics of rails

The characteristics of rails that were used for their classification:

1. Rails lie on the track bed.

2. Rails are considered as linear objects that for a short length like 5 meters direction can be defined.

3. Height of rails is 10 cm so that the points on top of rails are 10 cm higher than points on track bed.

4. The width of rails is 7 cm.

5. Gauge, the distance between two tracks of rails, is about 1.5 meters.

6. There is a limitation on slope of rails. The maximum slope was considered 5% so the height of certain length of a piece of rail does not exceed 5% of its length.

7. There is a limitation on maximum curvature of rails. The maximum displacement of the end of a 1 meter piece of rail is considered 5cm so the direction of a certain length of rail does not exceed a limit.

4.2.2. Characteristics of contact wires

The characteristics of contact wires that were used for their classification:

1. Contact wires are in an elevation of 5-5.5 meters above rails.

2. In top view, as if all points on rails and contact wires are projected on a planimetric plane, contact wires lie between two rail tracks.

Two characteristics mentioned above are about the relative position of contact wires to rails which is part of railway environment specifications. These specifications are the same for all other parts of Dutch railway environment.

4.2.3. Characteristics of catenary wires

The characteristics of catenary wires that were used for their classification:

1. Catenary wires are in an elevation of more than 5.5 meters above rails.

2. In top view, as if all points on rails and catenary wires are projected on a planimetric plane, catenary wires lie between two rail tracks.

Two characteristics mentioned above are about the relative position of catenary wires to rails which is part of railway environment specifications. These specifications are the same for all other parts of Dutch railway environment.

4.3. Applied constraints

Based on railway environment specifications and particularly characteristics of rails and wires, the following constraints were applied for classification of rails and wires:

1. Relative position of wires to rails.

1.1. Relative height of wires above rails.

1.1.1. Contact wires: 5-5.5 meters above rails.

1.1.2. Catenary wires: more than 5 meters above rails.

1.2. Planimetric position of wires with respect to rails.

1.2.1. In top view, both contact and catenary wires lie between two rail tracks.

2. Rail height: 10 cm 3. Rail width: 7 cm 4. Gauge: 1.5 m

5. Maximum slope of rails: 5%

6. Maximum curvature of rails: 5 cm curvature in each 10 m of rails.

4.4. Template Matching method

The general approach is to classify the objects of interest in several steps i.e. first the algorithm does a

rough classification of points on the objects of interest. Then it improves the classification results in next

steps by excluding the wrongly classified points and including the desirable points which were not

classified in previous steps. Figure 4-1 indicates the flowchart of TM method.

4.4.1. First classification of rails 4.4.1.1. Pre classification

First a rough classification is done based on the height of points. This leads to the classification of points as 4 classes:

1. Class 1: contains all points on the track bed and rails.

It is assumed that the most common elevation in the dataset represents the elevation of track bed.

This assumption was made due to higher point density of track bed in comparison to other objects. Besides, due to slope of the terrain, the elevation of track bed may change. So to include all points on rails and track bed in class 1, all points with elevation 0.5 m more or less than the most common elevation in the dataset are classified as class1.

Maximum Track bed height= the most common elevation + 0.5 meter Minimum Track bed height= the most common elevation - 0.5 meter

2. Class 2: contains all points with the maximum elevation 5 meters above track bed. Class 2 does not include any points on the objects of interest. So all points with a maximum elevation of 5 meters above maximum track bed height are classified as class 2.

3. Class 3: contains all points with the elevation 5-5.5 meters above track bed including points on contact wires. So all points with an elevation between 5 to 5.5 meters above maximum track bed height are classified as class 3.

4. Class 4: contains all points with the elevation more than 5.5 meters above track bed including the points on catenary wires. So all points with an elevation more than 5.5 meters above maximum track bed height are classified as class 4.

As a result, rail points should be detected out of class 1, contact wire points out of class 3 and catenary wire points out of class 4 and all points in class 2 are undesirable for us.

Relative heights of contact and catenary wires to rails are constant (5-5.5 m and more than 5.5 m above rails respectively) due to railway environment specification of The Netherlands.

4.4.1.2. Grid generation

As mentioned in section 4.4.1.1, class 1 contains all rail points so to classify rail points only points of class 1 are processed. The main idea is to inspect the data locally and this grid facilitates the inspection of data in details and locally.

So to implement, a 2D grid with cell size of 50 by 50 cm

was generated. In this step, cell size is very

influential on results and is highly dependent on data resolution, point density. A smaller cell size would

result in a cell with small number of points which decrease the reliability of results and a bigger cell size

can lead to cells containing two rails or rails with other objects which makes the analysis more

complicated. In fact, it is preferred that each cell contains maximum of one rail as it makes the process more efficient. Direction is a characteristic of rails that can be used if there is just one piece of rail in a cell.

As the grid is 2D, the location of each cell is indicated by two elements i.e. row and column. So to identify a cell in which a point falls, two elements of row and column should be determined by following steps:

1. Finding minimum of Xs and Ys of all points in class 1: MinX and MinY.

2. Determining that each point lies in which cell in a way described below:

a. For a point with coordinates: P(X,Y,Z).

b. For a grid cell size of 0.5 m.

c. Row = ((X-MinX)/0.5)+1 d. Column = ((Y-MinY)/0.5)+1

e. Row and Column indicate the location of the cell in which the point P falls in.

3. Continue the above steps for all points of class 1.

In this way, based on planimetric position of points, their X and Y coordinates, it was determined each point falls in which cell. Then, further analysis was done on all points which lie in one cell.

The idea is to have enough number of points in each cell so that they can represent the characteristics of objects in that cell which leads to reliable statistical analysis results. By an inspection of cells containing different number of points, it was found that the ones having less than 30 points do not represent true information about the characteristics of objects inside. Therefore, just cells with more than 30 points inside were considered for further analysis.

4.4.1.3. Height jump detection

The height jump detection (HJD) algorithm uses the fact that the height difference between the points on top of the rail and the points on the track bed is about 10 cm which is the rail height. The main idea is that the rails cause a height jump on the track bed due to their heights. In Figure 4-2, grey arrows show rails which cause height jumps and the white arrow shows another part of track bed without any height jumps.

Figure 4-2. Grey arrows point to height jumps caused by rails while the white arrow points to other parts of

track bed without any height jumps

The HJD method was applied to detect rail points and as only class 1 contains points on rails, only points of class 1 were processed for this stage and the strategy was:

1. To determine if a cell contains rail points or not by inspecting:

a. Whether there is a height jump in a cell or not.

b. If the size of the height jump is the same as the one caused by rails.

2. To differentiate the points on rail tracks from the points on track bed in a cell containing rails.

To determine if there is a height jump in a cell, the height difference between the highest and lowest points in a cell is inspected. If there is a rail in a cell, the height difference between the highest and lowest points should be at least 10 cm which is the rail height. But as the data contains noise, usually the lowest and highest points in each cell do not represent the rail and track bed respectively. This is evident in Figure 4-3 in which rail points are indicated by grey colour and track bed points by white colour and 10, 80 and 90 percentile elevations represent the elevations of track bed, bottom of rail and top of rail respectively.

Moreover, the size of the height jump needs to be checked. Besides rails, there are some other objects that cause height jumps like signs, electrical devices, bumps on track bed and the short walls where the railway stations and track bed intersect (Figure 4-4). So to differentiate between these objects from rails, the algorithm inspects the size of height jump. In other words, the size of height jump determines if it is a piece of rail or not. For instance, the short walls of intersection of track bed and railway stations and some signs make a bigger height jump while electrical devices and bumps on track bed make a smaller height jumps than that of caused by rails.

Figure 4-3. Using 10, 80 and 90 percentiles to remove noises for

implementation of height jump detection method

Therefore, with respect to the rail height, the algorithm considers only the height jumps in the range of 10- 25 cm as rails and ignores the rest. In Figure 4-4, two arrows on left indicate the height jumps caused by rails and the one on the right point to height jump caused by railway station.

As a result, the constraint is defined as:

If a cell contains a piece of rail, the height difference between the 10 and 90 percentile elevations of its points is more than 10 cm and less than 25 cm:

10 (centimetres) < (90%Elevation) – (10% Elevation) < 25 (centimetres)

Then for the cells which are considered to have rail points, the points on rails needs to be differentiated from the ones on track bed. To do this, the points higher than 80 percentile elevation of all points in that cell were considered as points on rails (Figure 4-3).

One should pay attention that this is the first classification of rail points and some wrong classifications and some classification gaps are expected which will be taken care of in the next steps.

4.4.2. First classification of wires

To classify the points on wires, three characteristics of wires were considered:

1. Contact wires are in elevation of 5-5.5 m above rails (Figure 4-5).

2. Catenary wires are in elevation of more than 5.5 m above rails (Figure 4-5).

Figure 4-4. Big height jump caused by railway station platform V.S. small height jumps caused by rails

3. In top view, as if all rail and wire points are projected on a planimetric plane, both contact and catenary wire points lie between rail points (Figure 4-6).

To this stage, some rail points are classified in previous step and all wire points are in class 3 and 4 of pre classification based on height. So if all classified rail points and all points in class 3 and 4 are projected on a planimetric plane, out of all points in class 3 and 4, only points on wires lie in a close neighbourhood of classified rail points.

To implement, all projected classified rail points were put in one KD tree data structure and all projected points of class 3 and 4 were put in another KD tree data structure. Then by doing the nearest neighbour analysis, only points of class 3 and 4 which were in a close neighbourhood of rail points were considered as contact and catenary wire points respectively.

4.4.3. Excluding wrongly classified rail points

It should be noticed that the first classification of wire points is dependent on the results of first classification of rail points. That is so because the first classification of wire points is done based on nearest neighbour analysis between points of class 3 and 4 and the classified rail points in previous step.

As in the results of first classification of rail points, some wrong classifications and some classification gaps are expected, the results of first classification of wires are expected to have some wrong

classifications and some classification gaps as well.

Figure 4-5. Relative height of wires to rails

Figure 4-6. In top view, wires shown by yellow and

orange colour lie between rails shown by green colour

The idea of TM was to seek the rail pattern in the classified points of previous steps to detect only truly rail points and exclude the wrongly classified ones. This was done by considering the spatial relation of wires to rails.

In fact, there are three points that should be noticed:

1. The purpose of this step is to exclude wrong classification of just rail points and not wire points.

2. TM was aimed to exclude wrong classifications and not aimed to include any more rail or wire points which were not classified in previous steps. Classification of the rest of rail and wire points which were not classified in previous steps is done in next steps described in sections 4.4.4 and 4.4.5.

3. The input of this step is the classified rail and wire points which is obtained in previous steps and the output is only true rail points.

So the point cloud is converted to a greyscale image and a kernel based on the defined rail pattern is generated. The rail pattern is defined based on the characteristics of rails and contact wires, mentioned in section 4.2, and contains:

1. Two straight rails:

a. 1 meter long

b. Distance between the axis of rail tracks:1.4 m c. Width of rail tracks: 20 cm

2. One contact wire:

a. In a certain elevation above rail tracks

b. In top view, its planimetric position lies between rail tracks c. Width of contact wire: 30 cm

3. A black area in surroundings of rails

Then to find rail patterns, the kernel is moved through the greyscale image with the following strategy:

1. Move the kernel through the first 5 meters of rails to get the main direction of rails.

2. Move the kernel through whole greyscale image every 5 rows and in each row every 3 cells to find some of rail patterns as a starting point for the next step.

3. Continue searching for rail pattern from the detected rail patterns in previous step.

a. Continue the search for rail pattern just in the neighbourhood of its two ends and follow them to find the rest of rail patterns.

b. Update the rails direction every 1 meter after each detection of rail pattern.

The strategy explained above is an optimized one as first it commences the rail pattern seeking with 5 rows and 3 columns steps. Then, it continues the search in the neighbourhood of detected patterns and follows them to detect the rest of rails. This algorithm is more efficient and faster than searching for rail pattern in each cell of greyscale image one by one.

The result up to this section would be a greyscale image in which most of the wrongly classified rail points

are excluded. However, some wrongly recognized rail patterns are still expected which are mostly scattered

component analysis is applied and small disconnected components are eliminated. As a result, only big components representing true rails are remained.

To implement, the kernel was generated as a greyscale image containing a pair of white colour lines of 1 meter long as rail tracks and one grey colour line of 1 meter long as contact wire between rail tracks.

Figure 4-7 shows the rail pattern in a railway environment and Figure 4-8 indicates the rail pattern as a greyscale image.

In implementation, to convert the point cloud to a greyscale image, the steps below were done:

1. A greyscale image was generated and the grey value of all its cells was set to zero.

2. It was determined each rail and wire point falls in which cell.

3. The grey value of each cell with at least one rail point inside was set to 255 (shown by white colour)

4. The grey value of each cell with at least one contact wire point inside was set to 128 (shown by grey colour)

The resolution, cell size, of greyscale image and kernel was chosen according to point cloud resolution. In addition, step 2 above was done based on planimetric position of points i.e. all rail and wire points which were classified in previous steps were projected on a planimetric plane. Next, as the location of each cell in

Figure 4-7. Rail and wire pattern in a railway environment

Figure 4-8. Rail and wire

pattern as a greyscale image

greyscale image is identified by two elements of row and column, to determine the cell in which a point P(X,Y,Z) falls, following steps were pursued:

1. The minimum of Xs and Ys of all points were found: MinX and MinY 2. For a grid cell size of 0.05 m

3. Row = ((X-MinX)/0.05)+1 4. Column = ((Y-MinY)/0.05)+1

5. Continue the above steps for all points

For each cell the kernel was rotated by different angles from -90 to +90 degrees and checked if the pattern of greyscale image in that cell matches with the rail pattern represented by the kernel. For each cell as the kernel rotated, the correlation between the image and kernel was calculated for each angle. Out of all correlations calculated for each cell, only the maximum correlation was considered and checked whether it exceeded a threshold or not. If the maximum correlation between the kernel and a part of the greyscale image was more than the threshold, it was concluded that part of the image contained rail pattern. In other words, the high correlation between the greyscale image and the kernel means there are some rail points in that part of the greyscale image. Then the rail points in that part are recognized according to the kernel pattern with respect to the angle by which the correlation exceeded the threshold.

To calculate the correlation threshold, some sampling on correlation between the kernel and different parts of greyscale image was done. Based on achieved results, the correlation between the kernel and parts containing rail patterns exceeded 0.3 in a 0-1 scale in which 0 represents no match and 1 represents complete match between kernel and a cell in greyscale image.

Finally, connected component analysis was applied to remove wrongly recognized rail patterns. In fact, it eliminated components with maximum size of a rail pattern of 1 meter long.

4.4.4. Classifying the rest of rail points

The result of previous step is a greyscale image containing some white and some black cells. White cells represent the rails and the rest of cells are black. To convert back the greyscale image to point clouds, like its first conversion in section 4.4.3, it is determined that each point falls in which cell. If a point fell in a white cell then it is labelled as a rail point and if it fell in a black cell, it is not labelled.

Therefore, the result is point clouds including some areas where no rail points are classified. It is assumed that not-classified pieces of rails are small enough to be considered as straight so to fill this classification gap, Hough Transform algorithm is applied. By applying Hough Transform algorithm, the parameters of corresponding lines of rails are calculated and by following these lines the gap areas are filled. The points in these gaps are labelled differently from true rail points and as potential rail points. To verify the potential rail points, it should be checked if in an elevation more than 5 m above them, there are some wire points or not. If so, they would be classified as rails.

To implement, it should be noticed that the main classification gap is an intersection between railway and

a road as it is a cross levelling and rails in that part do not cause a height jump. The length of this gap

which is the width of road intersecting railway was about 20 meters. For this length, rails were assumed to

be straight. To use the Hough Transform algorithm some parameters needed to be set and they were set

based on railway environment specifications mentioned in section 4.2.

The Hough Transform algorithm parameters were:

1. Maximum number of lines to be detected

a. i.e. maximum number of rails exist next to each other, set to 20 rails

b. Can be increased or decreased for other railway environments with more or less number of rails

2. Neighbourhood suppression

a. Which make the algorithm to detect maximum one rail in defined neighbourhood and suppress the rest of rails in that neighbourhood

b. Set based on gauge, meaning that in a neighbourhood of smaller than gauge maximum one rail exists

3. Maximum length of gap areas to be filled by algorithm a. Maximum width of roads intersecting with railway b. Was set to 50 meters

After converting the greyscale image back to 3D point clouds, it was checked whether in an elevation more than 5 m above potential rail points, there were some wires or not. To do so, again the nearest neighbour analysis was done between projected points of class 3 and 4 and all potential rail points. Only the potential rail points which were in a close neighbourhood of projected points of class 3 and 4 were labelled as true rails and the rest were labelled as other objects.

4.4.5. Classifying the rest of wire points

Up to this stage, all rail points are classified but some classification gap of wire points are expected. That is so because the first classification of wires was dependent on rails and so if rail points of a piece of rail were not classified in the first classification then the wires above were not classified as well. To improve the classification of wire points, the constraint about relative position of wires to rails is used i.e. if all rail points and all points of class 3 and 4 are projected on a planimetric plane, only the wire points lie in a very close neighbourhood of rail points.

To implement this idea, like the first classification of wire points, all projected classified rail points were put in one KD tree data structure and all projected points of class 3 and 4 were put in another KD tree data structure. Then by doing the nearest neighbour analysis, only points of class 3 and 4 which were in a close neighbourhood of rail points were considered as contact and catenary wire points respectively.

One should notice that although in this step used to classify wire points like their first classification in

section 4.4.2, nearest neighbour analysis was used. However, the result of this step is more accurate than

that of first classification of wires. That is so because in this step for nearest neighbour analysis, all

correctly classified rail points were used. While in first classification of wires, the rail points used for

nearest neighbour analysis contained some wrongly classified points as well as not including all rail points.

4.5. Region Growing method

The idea of Region Growing (RG) is first to find a small piece of an object of interest as seed points and

then look for the rest of that object in its neighbourhood according to its characteristics. Here rails and

contact wires are classified by the RG method and catenary wires are classified by the nearest neighbour

analysis. Figure 4-9 shows the flowchart of RG method. As it was mentioned in section 4.1, RG was

applied for two datasets, a terrestrial and an airborne dataset.

4.5.1. First classification of rails

In RG, first classification of rail points are done for the first 5 meters to obtain the rail seed points which are used for growing in section 4.5.3. In fact, for first classification of rail points same steps as in TM i.e.

pre classification based on height, grid generation and HJD are followed. The difference is that here these three steps are done only for the first 5 meters of point clouds. As a result, the first 5 meters of classified rail points are achieved. However, like the result of HJD in TM method, there are some wrong

classifications and some rail points are not classified in the first classification. Then by applying Hough Transform algorithm, the straight lines corresponding straight rails are detected and by following these lines, not-classified rail points are classified and wrong classifications are excluded. Hough Transform algorithm is applied based on the assumption that rails are straight for the first 5 meters due to limitation on maximum rail curvature.

For terrestrial dataset, the implementation of these three steps were the same as in TM as the same dataset was used for TM. The result of this step contained all rail points of the first 5 meters which were used as seed points for rail growing in section 4.5.3.

For airborne dataset, the seed points were detected manually because finding a short piece of rail as seed points is done by HJD method. In addition, as the resolution of airborne dataset is low and data resolution has a great influence on HJD results, it fails to classify a short piece of rail as rail seed points. On the other hand, as RG uses the height and direction of rail seed points to find the rest of rail points, the number of seed points should be enough to give a true value of their height and direction. As a result, in airborne dataset the first 15 meters of rails were considered as rail seed points. While this length was 5 meters for terrestrial dataset as it has a higher resolution.

4.5.2. First Classification of contact wires

First classification of contact wire points are done for the first 5 meters to obtain the contact wire seed points which are used for growing in section 4.5.4. For the first classification of contact wire points same steps as in TM i.e. nearest neighbour analysis between rail points and points in class 3 is done.

For terrestrial dataset, the first classification of wires was done by the same algorithm as in TM i.e. nearest neighbour analysis. So all classified rail points were put in one KD tree data structure and all points in class 3 were put in another KD tree data structure. Then by doing the nearest neighbour analysis, only the points in class 3 which were in a close neighbourhood of rail points were classified as contact wire seed points.

For airborne dataset, like detecting rail seed points, the first 15 meters of contact wires were detected manually as seed points due to the low resolution of the dataset.

4.5.3. Classifying the rest of rail points

The idea of RG is first to find a short piece of rail as rail seed points and then look for the rest of the rail points in its neighbourhood according to rail characteristics. Rail growing is done with respect to the fact that there is a limitation on slope and curvature of rails. First the algorithm extracts the height and direction of rail seed points as the starting height and direction for rail growing. Then points in a close neighbourhood of seed points with almost the same height as them are considered as candidate new rail points. Then it is inspected if each of these candidate points is added to rail points in its close

neighbourhood, the direction would change substantially or not. If not, that point is classified as rail point.

Then, newly classified rail points of each step are considered as new seed points and again their height and direction are calculated for next steps of growing. In other words, after each step of growing, height and direction are updated. The growing for rail points is followed until no more rail points are classified.

Having small growing step means that these parameters are updated more frequently which leads to more accurate results of classification even in steep areas with curved rails. In addition, smaller steps of growing results in higher speed of the algorithm because small step size means the algorithm should look for points on rails in a smaller neighbourhood around seed points. On the other hand, having very small growing step size is risky because a gap in the dataset bigger than the growing step leads to failure of algorithm to classify the rest of points after that gap. Therefore, growing step is preferred to be as small as possible and with respect to resolution of dataset, it cannot be smaller than the biggest gap in the dataset.

To implement, to find points in a close neighbourhood of seed points nearest neighbour analysis between seed points and points in track bed was done. The rail growing step for terrestrial dataset was considered as 1 meter while it was considered 5 meters for airborne dataset. The growing step of airborne dataset was chosen bigger than that of terrestrial dataset because airborne dataset has a lower resolution and bigger gaps than those of terrestrial dataset.

Although the developed algorithm grows each rail track separately, it also recognizes the pairs of rails and their corresponding contact and catenary wires above them. Terrestrial dataset contains straight rails in a flat area without considerable elevation change of rails and track bed. But for areas with more complicated configuration of rails, the ability of rail pair recognition can be very helpful. In such areas, the algorithm can grow each pair of rail tracks together and for each growing step checks if they are parallel and their distance from each other is the same as gauge or not. In other words, the ability of rail pair recognition can provide extra strong constraints for rail classification in case the current algorithm fails.

To implement rail pair recognition, rail seed points of each rail track are put in different KD tree data structures. Then by nearest neighbour analysis, the rails that are in a distance of 1.5 m from each other are labelled as the same pair. Up to this stage, the rail pairs of the first 5 meters are recognized. For growing, newly classified rail points in each growing step are labelled as they belong to the same pair of rails that their seed points do.

4.5.4. Classifying the rest of contact wire points

To classify contact wire points RG algorithm is again applied but the difference is just the constraint of direction is used and the constraint of height is ignored. In rail growing, the constraint of height is used to differentiate between the points on track bed and the points on rails but wires are already separated in pre classification step. In other words, in pre classification step, all contact wire points are separated from catenary wire points and are classified as class 3. Therefore, for contact wire growing, only the constraint of direction is used and the algorithm looks for new contact wire points only in points of class 3 which are close to contact wire seed points.

To implement, the same strategy as in rail growing was applied excluding the constraint of direction i.e.to

find points in a close neighbourhood of seed points nearest neighbour analysis between seed points and

points of class 3 was done. The contact wire growing step for terrestrial dataset was considered as 3

meters while it was considered 5 meters for airborne dataset. The growing step of airborne dataset was

bigger than growing steps for rails because the resolution of terrestrial dataset on rails is higher than that on wires. That is so due to close distance of mobile laser scanner to rails than to wires in terrestrial dataset.

While in airborne dataset, as the mobile laser scanner is on the air, the difference of its distance to rails and wires does not make a big change.

4.5.5. Classifying catenary wire points

The method applied for this part is the nearest neighbour analysis between classified contact wires in previous step and points of class 4. By considering this fact that catenary wires are exactly above contact wires, if all contact wire points which are classified in section 4.5.4 and all points in class 4 are projected on a planimetric plane, only catenary wire points lie in a very close neighbourhood of contact wire points.

To implement, all projected classified contact wire points were put in one KD tree data structure and all

projected points of class 4 were put in another KD tree data structure. Then by doing the nearest

neighbour analysis, only points of class 4 which were in a close neighbourhood of contact wires were

considered as catenary wires respectively. This step was the same for both terrestrial and airborne datsets.