data screenshots, texlab label prefix fix
This commit is contained in:
@@ -169,9 +169,9 @@
|
|||||||
% **************************************************************************************************
|
% **************************************************************************************************
|
||||||
% mainmatter (=content)
|
% mainmatter (=content)
|
||||||
|
|
||||||
\newchapter{Introduction}{chap:introduction}
|
\newchapter{introduction}{Introduction}
|
||||||
%\todo[inline, color=green!40]{its a master thesis where we try to know how trustworthy the sensor data for robot navigation is}
|
%\todo[inline, color=green!40]{its a master thesis where we try to know how trustworthy the sensor data for robot navigation is}
|
||||||
%\newsection{Motivation and Problem Statement}{sec:motivation}
|
%\newsection{Motivation and Problem Statement}{motivation}
|
||||||
%\todo[inline]{lidar and its role in robot navigation. discuss sensor degradation and its effects on navigation.}
|
%\todo[inline]{lidar and its role in robot navigation. discuss sensor degradation and its effects on navigation.}
|
||||||
|
|
||||||
Autonomous robots have gained more and more prevailance in search and rescue missions due to not endangering another human being and still being able to fulfil the difficult tasks of navigating hazardous environments like collapsed structures, identifying and locating victims and assessing the environment's safety for human rescue teams. To understand the environment, robots employ multiple sensor systems such as lidar, radar, ToF, ultrasound, optical cameras or infrared cameras of which lidar is the most prominently used due to its accuracy. The robots use the sensors' data to map their environments, navigate their surroundings and make decisions like which paths to prioritize. Many of the aforementioned algorithms are deep learning-based algorithms which are trained on large amounts of data whose characteristics are learned by the models.
|
Autonomous robots have gained more and more prevailance in search and rescue missions due to not endangering another human being and still being able to fulfil the difficult tasks of navigating hazardous environments like collapsed structures, identifying and locating victims and assessing the environment's safety for human rescue teams. To understand the environment, robots employ multiple sensor systems such as lidar, radar, ToF, ultrasound, optical cameras or infrared cameras of which lidar is the most prominently used due to its accuracy. The robots use the sensors' data to map their environments, navigate their surroundings and make decisions like which paths to prioritize. Many of the aforementioned algorithms are deep learning-based algorithms which are trained on large amounts of data whose characteristics are learned by the models.
|
||||||
@@ -191,22 +191,22 @@
|
|||||||
%\todo[inline, color=green!40]{difficult environments for sensors to produce good data quality (earthquakes, rescue robots), produced data may be unreliable, we don't know how trustworthy that data is (no quantification, confidence), since all navigation and decision making is based on input data, this makes the whole pipeline untrustworthy/problematic}
|
%\todo[inline, color=green!40]{difficult environments for sensors to produce good data quality (earthquakes, rescue robots), produced data may be unreliable, we don't know how trustworthy that data is (no quantification, confidence), since all navigation and decision making is based on input data, this makes the whole pipeline untrustworthy/problematic}
|
||||||
%\todo[inline, color=green!40]{contribution/idea of this thesis is to calculate a confidence score which describes how trustworthy input data is. algorithms further down the pipeline (slam, navigation, decision) can use this to make more informed decisions - examples: collect more data by reducing speed, find alternative routes, signal for help, do not attempt navigation, more heavily weight input from other sensors}
|
%\todo[inline, color=green!40]{contribution/idea of this thesis is to calculate a confidence score which describes how trustworthy input data is. algorithms further down the pipeline (slam, navigation, decision) can use this to make more informed decisions - examples: collect more data by reducing speed, find alternative routes, signal for help, do not attempt navigation, more heavily weight input from other sensors}
|
||||||
|
|
||||||
\newsection{Scope of Research}{chap:scope_research}
|
\newsection{scope_research}{Scope of Research}
|
||||||
\todo[inline]{output is score, thresholding (yes/no), maybe confidence in sensor/data? NOT how this score is used in navigation/other decisions further down the line}
|
\todo[inline]{output is score, thresholding (yes/no), maybe confidence in sensor/data? NOT how this score is used in navigation/other decisions further down the line}
|
||||||
\todo[inline]{Sensor degradation due to dust/smoke not rain/fog/...}
|
\todo[inline]{Sensor degradation due to dust/smoke not rain/fog/...}
|
||||||
\todo[inline, color=green!40]{we look at domain of rescue robots which save buried people after earthquakes, or in dangerous conditions (after fires, collapsed buildings) which means we are mostly working with indoors or subterranean environments which oftentimes are polluted by smoke and a lot of dust, ideally works for any kind of sensor data degradation but we only explore this domain}
|
\todo[inline, color=green!40]{we look at domain of rescue robots which save buried people after earthquakes, or in dangerous conditions (after fires, collapsed buildings) which means we are mostly working with indoors or subterranean environments which oftentimes are polluted by smoke and a lot of dust, ideally works for any kind of sensor data degradation but we only explore this domain}
|
||||||
\todo[inline, color=green!40]{mostly use lidar (state of the art) since they are very accurate in 3d mapping environments, so we focus on quantifying how trustworthy the lidar data is by itself. we do not look at other sensor data (tof, ultrasound, optical)}
|
\todo[inline, color=green!40]{mostly use lidar (state of the art) since they are very accurate in 3d mapping environments, so we focus on quantifying how trustworthy the lidar data is by itself. we do not look at other sensor data (tof, ultrasound, optical)}
|
||||||
\todo[inline, color=green!40]{intended output is confidence score which simply means higher score = worse data quality, lower score = trustworthy data. this score can be interpreted by algorithms in pipeline. we do not look at how this is implemented in the algorithms, no binary classifier but analog value, if this is wished followup algorithm has to decide (example by threshold or other methods)}
|
\todo[inline, color=green!40]{intended output is confidence score which simply means higher score = worse data quality, lower score = trustworthy data. this score can be interpreted by algorithms in pipeline. we do not look at how this is implemented in the algorithms, no binary classifier but analog value, if this is wished followup algorithm has to decide (example by threshold or other methods)}
|
||||||
|
|
||||||
\newsection{Structure of the Thesis}{sec:thesis_structure}
|
\newsection{thesis_structure}{Structure of the Thesis}
|
||||||
\todo[inline]{brief overview of thesis structure}
|
\todo[inline]{brief overview of thesis structure}
|
||||||
\todo[inline, color=green!40]{in section x we discuss anomaly detection, semi-supervised learning since such an algorithm was used as the chosen method, we also discuss how lidar works and the data it produces. then in we discuss in detail the chosen method DeepSAD in section X, in section 4 we discuss the traing and evaluation data, in sec 5 we describe our setup for training and evaluation (whole pipeline). results are presented and discussed in section 6. section 7 contains a conclusion and discusses future work}
|
\todo[inline, color=green!40]{in section x we discuss anomaly detection, semi-supervised learning since such an algorithm was used as the chosen method, we also discuss how lidar works and the data it produces. then in we discuss in detail the chosen method DeepSAD in section X, in section 4 we discuss the traing and evaluation data, in sec 5 we describe our setup for training and evaluation (whole pipeline). results are presented and discussed in section 6. section 7 contains a conclusion and discusses future work}
|
||||||
|
|
||||||
\newchapter{Background and Related Work}{chap:background}
|
\newchapter{background}{Background and Related Work}
|
||||||
|
|
||||||
\todo[inline, color=green!40]{in this section we will discuss necessary background knowledge for our chosen method and the sensor data we work with. related work exists mostly from autonomous driving which does not include subter data and mostly looks at precipitation as source of degradation, we modeled after one such paper and try to adapt the same method for the domain of rescue robots, this method is a semi-supervised deep learning approach to anomaly detection which we describe in more detail in sections 2.1 and 2.2. in the last subsection 2.3 we discuss lidar sensors and the data they produce}
|
\todo[inline, color=green!40]{in this section we will discuss necessary background knowledge for our chosen method and the sensor data we work with. related work exists mostly from autonomous driving which does not include subter data and mostly looks at precipitation as source of degradation, we modeled after one such paper and try to adapt the same method for the domain of rescue robots, this method is a semi-supervised deep learning approach to anomaly detection which we describe in more detail in sections 2.1 and 2.2. in the last subsection 2.3 we discuss lidar sensors and the data they produce}
|
||||||
|
|
||||||
\newsection{Anomaly Detection}{sec:anomaly_detection}
|
\newsection{anomaly_detection}{Anomaly Detection}
|
||||||
|
|
||||||
\todo[inline, color=green!40]{cite exists since X and has been used to find anomalous data in many domains and works with all kinds of data types/structures (visual, audio, numbers). examples healthcare (computer vision diagnostics, early detection), financial anomalies (credit card fraud, maybe other example), security/safety video cameras (public, traffic, factories).}
|
\todo[inline, color=green!40]{cite exists since X and has been used to find anomalous data in many domains and works with all kinds of data types/structures (visual, audio, numbers). examples healthcare (computer vision diagnostics, early detection), financial anomalies (credit card fraud, maybe other example), security/safety video cameras (public, traffic, factories).}
|
||||||
\todo[inline, color=green!40]{the goal of these algorithms is to differentiate between normal and anomalous data by finding statistically relevant information which separates the two, since these methods learn how normal data typically is distributed they do not have to have prior knowledge of the types of all anomalies, therefore can potentially detect unseen, unclassified anomalies as well. main challenges when implementing are that its difficult to cleanly separate normal from anormal data}
|
\todo[inline, color=green!40]{the goal of these algorithms is to differentiate between normal and anomalous data by finding statistically relevant information which separates the two, since these methods learn how normal data typically is distributed they do not have to have prior knowledge of the types of all anomalies, therefore can potentially detect unseen, unclassified anomalies as well. main challenges when implementing are that its difficult to cleanly separate normal from anormal data}
|
||||||
@@ -226,7 +226,7 @@
|
|||||||
\todo[inline]{Figure example normal data boundaries, single outliers o1, o2, cluster of outliers o3. difficult to define boundaries so that all normal data inside and anomalies outside }
|
\todo[inline]{Figure example normal data boundaries, single outliers o1, o2, cluster of outliers o3. difficult to define boundaries so that all normal data inside and anomalies outside }
|
||||||
\fi
|
\fi
|
||||||
|
|
||||||
\newsection{Semi-Supervised Learning Algorithms}{sec:semi_supervised}
|
\newsection{semi_supervised}{Semi-Supervised Learning Algorithms}
|
||||||
\todo[inline]{Quick overview of the DeepSAD metho}
|
\todo[inline]{Quick overview of the DeepSAD metho}
|
||||||
\todo[inline, color=green!40]{deep learning based (neural network with hidden layers), neural networks which get trained using backpropagation, to learn to solve a novel task by defining some target}
|
\todo[inline, color=green!40]{deep learning based (neural network with hidden layers), neural networks which get trained using backpropagation, to learn to solve a novel task by defining some target}
|
||||||
\todo[inline, color=green!40]{data labels decide training setting (supervised, non-supervised, semi-supervised incl explanation), supervised often classification based, but not possible if no labels available, un-supervised has no well-defined target, often used to fined common hidden factors in data (distribution). semi-supervised more like a sub method of unsupervised which additionally uses little (often handlabelled) data to improve method performance}
|
\todo[inline, color=green!40]{data labels decide training setting (supervised, non-supervised, semi-supervised incl explanation), supervised often classification based, but not possible if no labels available, un-supervised has no well-defined target, often used to fined common hidden factors in data (distribution). semi-supervised more like a sub method of unsupervised which additionally uses little (often handlabelled) data to improve method performance}
|
||||||
@@ -234,14 +234,14 @@
|
|||||||
\todo[inline, color=green!40]{find easy illustrative example with figure of semi-supervised learning and include + explain here}
|
\todo[inline, color=green!40]{find easy illustrative example with figure of semi-supervised learning and include + explain here}
|
||||||
\todo[inline, color=green!40]{our chosen method DeepSAD is a semi-supervised deep learning method whose workings will be discussed in more detail in secion X}
|
\todo[inline, color=green!40]{our chosen method DeepSAD is a semi-supervised deep learning method whose workings will be discussed in more detail in secion X}
|
||||||
|
|
||||||
\newsection{Autoencoder}{sec:autoencoder}
|
\newsection{autoencoder}{Autoencoder}
|
||||||
\todo[inline]{autoencoder explanation}
|
\todo[inline]{autoencoder explanation}
|
||||||
\todo[inline, color=green!40]{autoencoders are a neural network architecture archetype (words) whose training target is to reproduce the input data itself - hence the name. the architecture is most commonly a mirrored one consisting of an encoder which transforms input data into a hyperspace represantation in a latent space and a decoder which transforms the latent space into the same data format as the input data (phrasing), this method typically results in the encoder learning to extract the most robust and critical information of the data and the (todo maybe something about the decoder + citation for both). it is used in many domains translations, LLMs, something with images (search example + citations)}
|
\todo[inline, color=green!40]{autoencoders are a neural network architecture archetype (words) whose training target is to reproduce the input data itself - hence the name. the architecture is most commonly a mirrored one consisting of an encoder which transforms input data into a hyperspace represantation in a latent space and a decoder which transforms the latent space into the same data format as the input data (phrasing), this method typically results in the encoder learning to extract the most robust and critical information of the data and the (todo maybe something about the decoder + citation for both). it is used in many domains translations, LLMs, something with images (search example + citations)}
|
||||||
\todo[inline, color=green!40]{typical encoder decoder mirrored figure}
|
\todo[inline, color=green!40]{typical encoder decoder mirrored figure}
|
||||||
\todo[inline, color=green!40]{explain figure}
|
\todo[inline, color=green!40]{explain figure}
|
||||||
\todo[inline, color=green!40]{our chosen method DeepSAD uses an autoencoder to translate input data into a latent space, in which it can more easily differentiate between normal and anomalous data}
|
\todo[inline, color=green!40]{our chosen method DeepSAD uses an autoencoder to translate input data into a latent space, in which it can more easily differentiate between normal and anomalous data}
|
||||||
|
|
||||||
\newsection{Lidar - Light Detection and Ranging}{sec:lidar_related_work}
|
\newsection{lidar_related_work}{Lidar - Light Detection and Ranging}
|
||||||
\todo[inline]{related work in lidar}
|
\todo[inline]{related work in lidar}
|
||||||
\todo[inline, color=green!40]{the older more commonly known radar works by sending out an electromagnetic wave in the radiofrequency and detecting the time it takes to return (if it returns at all) signalling a reflective object in the path of the radiowave. lidar works on the same principle but sends out a lightray produced by a laser (citation needed) and measuring the time it takes for the ray to return to the sensor. since the speed of light is constant in air the system can calculate the distance between the sensor and the measured point. modern lidar systems send out multiple, often millions of measurement rays per second which results in a three dimensional pointcloud, constructed from the information in which direction the ray was cast and the distance that was measured}
|
\todo[inline, color=green!40]{the older more commonly known radar works by sending out an electromagnetic wave in the radiofrequency and detecting the time it takes to return (if it returns at all) signalling a reflective object in the path of the radiowave. lidar works on the same principle but sends out a lightray produced by a laser (citation needed) and measuring the time it takes for the ray to return to the sensor. since the speed of light is constant in air the system can calculate the distance between the sensor and the measured point. modern lidar systems send out multiple, often millions of measurement rays per second which results in a three dimensional pointcloud, constructed from the information in which direction the ray was cast and the distance that was measured}
|
||||||
\todo[inline, color=green!40]{lidar is used in most domains reliant on accurate 3d representations of the world like autonomous driving, robot navigation, (+ maybe quickly look up two other domains), its main advantage is high measurement accuracy, precision (use correct term), and high resolution (possible due to single point measurements instead of cones like radar, ToF, Ultrasonic) which enables more detailed mappings of the environment}
|
\todo[inline, color=green!40]{lidar is used in most domains reliant on accurate 3d representations of the world like autonomous driving, robot navigation, (+ maybe quickly look up two other domains), its main advantage is high measurement accuracy, precision (use correct term), and high resolution (possible due to single point measurements instead of cones like radar, ToF, Ultrasonic) which enables more detailed mappings of the environment}
|
||||||
@@ -249,10 +249,10 @@
|
|||||||
\todo[inline, color=green!40]{because of the given advantages of lidar it is most commonly used nowadays on robot platforms for environment mapping and navigiation - so we chose to demonstrate our method based on degraded data collected by a lidar sensor as discussed in more dtail in section (data section)}
|
\todo[inline, color=green!40]{because of the given advantages of lidar it is most commonly used nowadays on robot platforms for environment mapping and navigiation - so we chose to demonstrate our method based on degraded data collected by a lidar sensor as discussed in more dtail in section (data section)}
|
||||||
|
|
||||||
|
|
||||||
\newchapter{DeepSAD: Semi-Supervised Anomaly Detection}{chap:deepsad}
|
\newchapter{deepsad}{DeepSAD: Semi-Supervised Anomaly Detection}
|
||||||
Deep Semi-Supervised Anomaly Detection~\cite{deepsad} is a deep-learning based anomaly detection method whose performance in regards to sensor degradation quantification we explore in this thesis. It is a semi-supervised method which allows the introduction of manually labeled samples in addition to the unlabeled training data to improve the algorithm's performance over its unsupervised predecessor Deep One-Class Classification~\cite{deepsvdd}.\todo{phrasing} The working principle of the method is to encode the input data onto a latent space and train the network to cluster normal data close together while anomalies get mapped further away in that latent space.
|
Deep Semi-Supervised Anomaly Detection~\cite{deepsad} is a deep-learning based anomaly detection method whose performance in regards to sensor degradation quantification we explore in this thesis. It is a semi-supervised method which allows the introduction of manually labeled samples in addition to the unlabeled training data to improve the algorithm's performance over its unsupervised predecessor Deep One-Class Classification~\cite{deepsvdd}.\todo{phrasing} The working principle of the method is to encode the input data onto a latent space and train the network to cluster normal data close together while anomalies get mapped further away in that latent space.
|
||||||
%\todo[inline, color=green!40]{DeepSAD is a semi-supervised anomaly detection method proposed in cite, which is based on an unsupervised method (DeepSVDD) and additionally allows for providing some labeled data which is used during the training phase to improve the method's performance}
|
%\todo[inline, color=green!40]{DeepSAD is a semi-supervised anomaly detection method proposed in cite, which is based on an unsupervised method (DeepSVDD) and additionally allows for providing some labeled data which is used during the training phase to improve the method's performance}
|
||||||
\newsection{Algorithm Description}{sec:algorithm_description}
|
\newsection{algorithm_description}{Algorithm Description}
|
||||||
%\todo[inline]{explain deepsad in detail}
|
%\todo[inline]{explain deepsad in detail}
|
||||||
|
|
||||||
\todo[inline, color=green!40]{Core idea of the algorithm is to learn a transformation to map input data into a latent space where normal data clusters close together and anomalous data gets mapped further away. to achieve this the methods first includes a pretraining step of an auto-encoder to extract the most relevant information, second it fixes a hypersphere center in the auto-encoders latent space as a target point for normal data and third it traings the network to map normal data closer to that hypersphere center. Fourth The resulting network can map new data into this latent space and interpret its distance from the hypersphere center as an anomaly score which is larger the more anomalous the datapoint is}
|
\todo[inline, color=green!40]{Core idea of the algorithm is to learn a transformation to map input data into a latent space where normal data clusters close together and anomalous data gets mapped further away. to achieve this the methods first includes a pretraining step of an auto-encoder to extract the most relevant information, second it fixes a hypersphere center in the auto-encoders latent space as a target point for normal data and third it traings the network to map normal data closer to that hypersphere center. Fourth The resulting network can map new data into this latent space and interpret its distance from the hypersphere center as an anomaly score which is larger the more anomalous the datapoint is}
|
||||||
@@ -261,7 +261,7 @@
|
|||||||
\todo[inline, color=green!40]{explanation training step: during the main training step the method starts with the pre-trained weights of the encoder but removes the decoder from the architecture since it optimizes the output in the latent space and does not need to reproduce the input data format. it does so by minimizing the geometric distance of each input data's latent space represenation to the previously defined hypersphere center c. Due to normal data being more common in the inputs this results in normal data clustering closely to C and anormal data being pushed away from it. additionally during this step the labeled data is used to more correctly map normal and anormal data}
|
\todo[inline, color=green!40]{explanation training step: during the main training step the method starts with the pre-trained weights of the encoder but removes the decoder from the architecture since it optimizes the output in the latent space and does not need to reproduce the input data format. it does so by minimizing the geometric distance of each input data's latent space represenation to the previously defined hypersphere center c. Due to normal data being more common in the inputs this results in normal data clustering closely to C and anormal data being pushed away from it. additionally during this step the labeled data is used to more correctly map normal and anormal data}
|
||||||
\todo[inline, color=green!40]{explanation inference step: with the trained network we can transform new input data into the latent space and calculate its distance from the hypersphere center which will be smaller the more confident the network is in the data being normal and larger the more likely the data is anomalous. This output score is an analog value dependent on multiple factors like the latent space dimensionality, encoder architecture and ??? and has to be interpreted further to be used (for example thresholding)}
|
\todo[inline, color=green!40]{explanation inference step: with the trained network we can transform new input data into the latent space and calculate its distance from the hypersphere center which will be smaller the more confident the network is in the data being normal and larger the more likely the data is anomalous. This output score is an analog value dependent on multiple factors like the latent space dimensionality, encoder architecture and ??? and has to be interpreted further to be used (for example thresholding)}
|
||||||
|
|
||||||
\newsection{Algorithm Details and Hyperparameters}{sec:algorithm_details}
|
\newsection{algorithm_details}{Algorithm Details and Hyperparameters}
|
||||||
\todo[inline]{backpropagation optimization formula, hyperaparameters explanation}
|
\todo[inline]{backpropagation optimization formula, hyperaparameters explanation}
|
||||||
|
|
||||||
\todo[inline, color=green!40]{in formula X we see the optimization target of the algorithm. explain in one paragraph the variables in the optimization formula}
|
\todo[inline, color=green!40]{in formula X we see the optimization target of the algorithm. explain in one paragraph the variables in the optimization formula}
|
||||||
@@ -274,10 +274,10 @@
|
|||||||
\end{equation}
|
\end{equation}
|
||||||
|
|
||||||
|
|
||||||
\newsection{Advantages and Limitations}{sec:advantages_limitations}
|
\newsection{advantages_limitations}{Advantages and Limitations}
|
||||||
\todo[inline]{semi supervised, learns normality by amount of data (no labeling/ground truth required), very few labels for better training to specific situation}
|
\todo[inline]{semi supervised, learns normality by amount of data (no labeling/ground truth required), very few labels for better training to specific situation}
|
||||||
|
|
||||||
\newchapter{Data and Preprocessing}{chap:data_preprocessing}
|
\newchapter{data_preprocessing}{Data and Preprocessing}
|
||||||
%\todo[inline, color=green!40]{good data important for learning based methods and for evaluation. in this chapter we talk about the requirements we have for our data and the difficulties that come with them and will then give some information about the dataset that was used as well as how the data was preprocessed for the experiments (sec 4.2)}
|
%\todo[inline, color=green!40]{good data important for learning based methods and for evaluation. in this chapter we talk about the requirements we have for our data and the difficulties that come with them and will then give some information about the dataset that was used as well as how the data was preprocessed for the experiments (sec 4.2)}
|
||||||
|
|
||||||
%Fortunately situations like earthquakes, structural failures and other circumstances where rescue robots need to be employed are uncommon occurences. When such an operation is conducted, the main focus lies on the fast and safe rescue of any survivors from the hazardous environment, therefore it makes sense that data collection is not a priority. Paired with the rare occurences this leads to a lack of publicly available data of such situations. To improve any method, a large enough, diversified and high quality dataset is always necessary to provide a comprehensive evaluation. Additionally, in this work we evaluate a training based method, which increases the requirements on the data manifold, which makes it all the more complex to find a suitable dataset. In this chapter we will state the requirements we defined for the data, talk about the dataset that was chosen for this task, including some statistics and points of interest, as well as how it was preprocessed for the training and evaluation of the methods.
|
%Fortunately situations like earthquakes, structural failures and other circumstances where rescue robots need to be employed are uncommon occurences. When such an operation is conducted, the main focus lies on the fast and safe rescue of any survivors from the hazardous environment, therefore it makes sense that data collection is not a priority. Paired with the rare occurences this leads to a lack of publicly available data of such situations. To improve any method, a large enough, diversified and high quality dataset is always necessary to provide a comprehensive evaluation. Additionally, in this work we evaluate a training based method, which increases the requirements on the data manifold, which makes it all the more complex to find a suitable dataset. In this chapter we will state the requirements we defined for the data, talk about the dataset that was chosen for this task, including some statistics and points of interest, as well as how it was preprocessed for the training and evaluation of the methods.
|
||||||
@@ -287,7 +287,7 @@ Situations such as earthquakes, structural failures, and other emergencies that
|
|||||||
In this chapter, we outline the specific requirements we established for the data, describe the dataset selected for this task—including key statistics and notable features—and explain the preprocessing steps applied for training and evaluating the methods.
|
In this chapter, we outline the specific requirements we established for the data, describe the dataset selected for this task—including key statistics and notable features—and explain the preprocessing steps applied for training and evaluating the methods.
|
||||||
|
|
||||||
|
|
||||||
\newsection{Data}{sec:data}
|
\newsection{data}{Data}
|
||||||
|
|
||||||
|
|
||||||
%\todo[inline]{describe data sources, limitations}
|
%\todo[inline]{describe data sources, limitations}
|
||||||
@@ -352,31 +352,30 @@ During the measurement campaign, 14 experiments were conducted—10 without the
|
|||||||
|
|
||||||
In the anomalous experiments, the artificial smoke machine appears to have been running for some time before data collection began, as evidenced by both camera images and LiDAR data showing an even distribution of water vapor around the machine. The stationary experiment is particularly unique: the smoke machine was positioned very close to the sensor platform and was actively generating new, dense smoke, to the extent that the LiDAR registered the surface of the fresh water vapor as if it were a solid object.
|
In the anomalous experiments, the artificial smoke machine appears to have been running for some time before data collection began, as evidenced by both camera images and LiDAR data showing an even distribution of water vapor around the machine. The stationary experiment is particularly unique: the smoke machine was positioned very close to the sensor platform and was actively generating new, dense smoke, to the extent that the LiDAR registered the surface of the fresh water vapor as if it were a solid object.
|
||||||
|
|
||||||
|
%\todo[inline, color=green!40]{shortly mention the differences in conditions for these experiments and why they do not matter for us}
|
||||||
|
|
||||||
\todo[inline, color=green!40]{shortly mention the differences in conditions for these experiments and why they do not matter for us}
|
%The 14 experiments differ regarding the available illumination, the presence of humans-traversing the measurement grounds- or additional static objects as artifcats and of course regarding the presence of the water vapor from the smoke machine. Aside from the artifical smoke which is essential for our use case, the other differences during the individual experiments are of no interestet to us and do not affect it in any way. Regardless of illumination, the lidar sensor produces indistinguishable pointclouds and any static objects do not factor into our quantification of the pointclouds' degradation.
|
||||||
\todo[inline, color=green!40]{include representative image of pointcloud and camera image}
|
|
||||||
|
The 14 experiments varied in illumination conditions, the presence of humans on the measurement grounds, and additional static artifacts, as well as in the presence of water vapor from the smoke machine. For our purposes, only the artificial smoke is relevant; differences in lighting or incidental static objects do not affect our analysis. Regardless of illumination, the LiDAR sensor consistently produces comparable point clouds, and the presence of static objects does not influence our quantification of point cloud degradation.
|
||||||
|
|
||||||
|
%\todo[inline, color=green!40]{include representative image of pointcloud and camera image}
|
||||||
|
|
||||||
|
The figures~\ref{fig:data_screenshot_pointcloud}~and~\ref{fig:data_screenshot_camera} show an representative depiction of the environment of the experiments as a camera image of the IR camera and the pointcloud created by the OS1 lidar sensor at practically the same time.
|
||||||
|
|
||||||
|
\fig{data_screenshot_pointcloud}{figures/data_screenshot_pointcloud.png}{Screenshot of 3D rendering of an experiment without smoke and with illumination (same frame and roughly same alignment as figure~\ref{fig:data_screenshot_camera}). Point color corresponds to measurement range and axis in center of figure is the lidar's position.}
|
||||||
|
\fig{data_screenshot_camera}{figures/data_screenshot_camera.png}{Screenshot of IR camera output of an experiment without smoke and with illumination (same frame and roughly same alignment as figure~\ref{fig:data_screenshot_pointcloud})}
|
||||||
|
|
||||||
%\todo[inline, color=green!40]{talk about how much data is available (maybe a plot about data?), number of experiments with/without degradation, other factors in these experiments which do not concern our use-case of them}
|
%\todo[inline, color=green!40]{talk about how much data is available (maybe a plot about data?), number of experiments with/without degradation, other factors in these experiments which do not concern our use-case of them}
|
||||||
%Regarding the amount of data, of the 10 normal experiments the shortest was 88.7 seconds and the longest 363.1 seconds with a mean of 157.65 seconds between all 10 experiments, which results in 15765 non-degraded pointclouds. Of the 4 anomalous experiments, the shortest was the stationary one with 11.7 seconds and the longest was 62.1 seconds, having a mean of 47.325 seconds, resulting in 1893 degraded pointclouds. This gives us 17658 pointclouds alltogether with 89.28\% of them being non-degraded/normal samples and the other 10.72\% of them begin degraded/anomalous samples.
|
%Regarding the amount of data, of the 10 normal experiments the shortest was 88.7 seconds and the longest 363.1 seconds with a mean of 157.65 seconds between all 10 experiments, which results in 15765 non-degraded pointclouds. Of the 4 anomalous experiments, the shortest was the stationary one with 11.7 seconds and the longest was 62.1 seconds, having a mean of 47.325 seconds, resulting in 1893 degraded pointclouds. This gives us 17658 pointclouds alltogether with 89.28\% of them being non-degraded/normal samples and the other 10.72\% of them begin degraded/anomalous samples.
|
||||||
|
|
||||||
Regarding the dataset volume, the 10 normal experiments ranged from 88.7 to 363.1 seconds, with an average duration of 157.65 seconds. At a capture rate of 10 frames per second, these experiments yield 15,765 non-degraded point clouds. In contrast, the 4 anomalous experiments, including one stationary experiment lasting 11.7 seconds and another extending to 62.1 seconds, averaged 47.33 seconds, resulting in 1,893 degraded point clouds. In total, the dataset comprises 17,658 point clouds, with approximately 89.28\% classified as non-degraded (normal) and 10.72\% as degraded (anomalous). The distribution of experimental data is visualized in figure~\ref{fig:data_points_pie}.
|
Regarding the dataset volume, the 10 normal experiments ranged from 88.7 to 363.1 seconds, with an average duration of 157.65 seconds. At a capture rate of 10 frames per second, these experiments yield 15,765 non-degraded point clouds. In contrast, the 4 anomalous experiments, including one stationary experiment lasting 11.7 seconds and another extending to 62.1 seconds, averaged 47.33 seconds, resulting in 1,893 degraded point clouds. In total, the dataset comprises 17,658 point clouds, with approximately 89.28\% classified as non-degraded (normal) and 10.72\% as degraded (anomalous). The distribution of experimental data is visualized in figure~\ref{fig:data_points_pie}.
|
||||||
|
|
||||||
\begin{figure}
|
\fig{data_points_pie}{figures/data_points_pie.png}{Pie chart visualizing the amount and distribution of normal and anomalous pointclouds in \cite{subter}}
|
||||||
\begin{center}
|
|
||||||
\includegraphics[width=0.9\textwidth]{figures/data_points_pie.png}
|
|
||||||
\end{center}
|
|
||||||
\caption{Pie chart visualizing the amount and distribution of normal and anomalous pointclouds in \cite{subter}}\label{fig:data_points_pie}
|
|
||||||
\end{figure}
|
|
||||||
|
|
||||||
%BEGIN missing points
|
%BEGIN missing points
|
||||||
As we can see in figure~\ref{fig:data_missing_points}, the artifical smoke introduced as explicit degradation during some experiments results in more missing measurements during scans, which can be explained by measurement rays hitting airborne particles but not being reflected back to the sensor in a way it can measure.
|
As we can see in figure~\ref{fig:data_missing_points}, the artifical smoke introduced as explicit degradation during some experiments results in more missing measurements during scans, which can be explained by measurement rays hitting airborne particles but not being reflected back to the sensor in a way it can measure.
|
||||||
|
|
||||||
\begin{figure}
|
\fig{data_missing_points}{figures/data_missing_points.png}{Density histogram showing the percentage of missing measurements per scan for normal experiments without degradation and anomalous experiments with artifical smoke introduced as degradation.}
|
||||||
\begin{center}
|
|
||||||
\includegraphics[width=0.9\textwidth]{figures/data_missing_points.png}
|
|
||||||
\end{center}
|
|
||||||
\caption{Density histogram showing the percentage of missing measurements per scan for normal experiments without degradation and anomalous experiments with artifical smoke introduced as degradation.}\label{fig:data_missing_points}
|
|
||||||
\end{figure}
|
|
||||||
%END missing points
|
%END missing points
|
||||||
|
|
||||||
%BEGIN early returns
|
%BEGIN early returns
|
||||||
@@ -387,49 +386,47 @@ Regarding the dataset volume, the 10 normal experiments ranged from 88.7 to 363.
|
|||||||
|
|
||||||
While the density of these near-sensor returns might be used to estimate data quality when the sensor is already in an environment with airborne particles, this method cannot anticipate data degradation before the sensor enters such an area. Since LiDAR is intended to capture visible geometry from a distance, it is preferable to quantify potential degradation of an area in advance. For these reasons, we did not incorporate this phenomenon into our subsequent analysis.
|
While the density of these near-sensor returns might be used to estimate data quality when the sensor is already in an environment with airborne particles, this method cannot anticipate data degradation before the sensor enters such an area. Since LiDAR is intended to capture visible geometry from a distance, it is preferable to quantify potential degradation of an area in advance. For these reasons, we did not incorporate this phenomenon into our subsequent analysis.
|
||||||
|
|
||||||
\begin{figure}
|
\fig{particles_near_sensor}{figures/particles_near_sensor_boxplot_zoomed_500.png}{Box diagram depicting the percentage of measurements closer than 50 centimeters to the sensor for normal and anomalous experiments}
|
||||||
\begin{center}
|
|
||||||
\includegraphics[width=0.9\textwidth]{figures/particles_near_sensor_boxplot_zoomed_500.png}
|
|
||||||
\end{center}
|
|
||||||
\caption{Box diagram depicting the percentage of measurements closer than 50 centimeters to the sensor for normal and anomalous experiments}\label{fig:particles_near_sensor}
|
|
||||||
\end{figure}
|
|
||||||
%END early returns
|
%END early returns
|
||||||
|
|
||||||
\newsection{Preprocessing Steps and Labeling}{sec:preprocessing}
|
\newsection{preprocessing}{Preprocessing Steps and Labeling}
|
||||||
\todo[inline]{describe how 3d lidar data was preprocessed (2d projection), labeling}
|
%\todo{describe how 3d lidar data was preprocessed (2d projection), labeling}
|
||||||
\todo[inline]{screenshots of 2d projections?}
|
%\todo[inline]{screenshots of 2d projections?}
|
||||||
|
|
||||||
|
%\todo[inline, color=green!40]{while as described in sec X the method DeepSAD is not dependend on any specific type/structure of data it requires to train an auto encoder in the pretraining step. such autoencoders are better understood in the image domain since there are many uses cases for this such as X (TODO citation needed), there are also 3d data auto encoders such as X (todo find example). same as the reference paper (rain cite) we chose to transform the 3d data to 2d by using a spherical spherical projection to map each of the 3d points onto a 2d plane where the range of each measurement can be expressed as the brightness of a single pixel. this leaves us with a 2d image of resolution 32x2048 (channels by horizontal measurements), which is helpful for visualization as well as for choosing a simpler architecture for the autoencoder of deepsad, the data in the rosbag is sparse meaning that measurements of the lidar which did not produce any value (no return ray detected before sensor specific timeout) are simply not present in the lidar scan. meaning we have at most 65xxx measurements per scan but mostly fewer than this, (maybe statistic about this? could aslo be interesting to show smoke experiment stuff)}
|
||||||
|
|
||||||
|
Test Section~\ref{sec:algorithm_description} more test.
|
||||||
|
|
||||||
\todo[inline, color=green!40]{while as described in sec X the method DeepSAD is not dependend on any specific type/structure of data it requires to train an auto encoder in the pretraining step. such autoencoders are better understood in the image domain since there are many uses cases for this such as X (TODO citation needed), there are also 3d data auto encoders such as X (todo find example). same as the reference paper (rain cite) we chose to transform the 3d data to 2d by using a spherical spherical projection to map each of the 3d points onto a 2d plane where the range of each measurement can be expressed as the brightness of a single pixel. this leaves us with a 2d image of resolution 32x2048 (channels by horizontal measurements), which is helpful for visualization as well as for choosing a simpler architecture for the autoencoder of deepsad, the data in the rosbag is sparse meaning that measurements of the lidar which did not produce any value (no return ray detected before sensor specific timeout) are simply not present in the lidar scan. meaning we have at most 65xxx measurements per scan but mostly fewer than this, (maybe statistic about this? could aslo be interesting to show smoke experiment stuff)}
|
|
||||||
\todo[inline, color=green!40]{to achieve this transformation we used the helpful measurement index and channel present in each measurement point of the dataset which allowed a perfect reconstruction of the 2d projection without calculating the pixel position in the projection of each measurement via angles which in our experience typically leads to some ambiguity in the projection (multiple measurements mapping to the same pixel due to precision loss/other errors) the measurement index increases even for unavailable measurements (no ray return) so we can simply create the 2d projection by mapping the normalized range (FIXME really normalized) value to the pixel position y = channel, x = measurement index. by initalizing the array to NaN values originally we have a 2d data structure with the range values and NaN on pixel positions where originally no measurement took place (missing measurements in scans due to no ray return)}
|
\todo[inline, color=green!40]{to achieve this transformation we used the helpful measurement index and channel present in each measurement point of the dataset which allowed a perfect reconstruction of the 2d projection without calculating the pixel position in the projection of each measurement via angles which in our experience typically leads to some ambiguity in the projection (multiple measurements mapping to the same pixel due to precision loss/other errors) the measurement index increases even for unavailable measurements (no ray return) so we can simply create the 2d projection by mapping the normalized range (FIXME really normalized) value to the pixel position y = channel, x = measurement index. by initalizing the array to NaN values originally we have a 2d data structure with the range values and NaN on pixel positions where originally no measurement took place (missing measurements in scans due to no ray return)}
|
||||||
\todo[inline, color=green!40]{another important preprocessing step is labeling of the lidar frames as normal/anormal. this is one hand used during training (experiments with zero labeled up to most of the data being labeled) and on the other hand is important for evaluation of the method performance. originally we do not have any labels on the data regarding degradation and no analog values from another sensor which measures current smoke particles in the air. our simple approach was to label all frames from experiments which included artifical degradation by fog machine smoke as anomalous and all frames from experiments without artifical degradation as normal.}
|
\todo[inline, color=green!40]{another important preprocessing step is labeling of the lidar frames as normal/anormal. this is one hand used during training (experiments with zero labeled up to most of the data being labeled) and on the other hand is important for evaluation of the method performance. originally we do not have any labels on the data regarding degradation and no analog values from another sensor which measures current smoke particles in the air. our simple approach was to label all frames from experiments which included artifical degradation by fog machine smoke as anomalous and all frames from experiments without artifical degradation as normal.}
|
||||||
\todo[inline, color=green!40]{this simple labeling method is quite flawed since we do not label based on the actual degradation of the scan (not by some kind of threshold of analog measurement threshold, statistical info about scan) since (TODO FIXME) this would result in training which only learns this given metric (example missing measurement points) which would make this methodology useless since we could simply use that same measurement as an more simple way to quantify the scan's degradation. }
|
\todo[inline, color=green!40]{this simple labeling method is quite flawed since we do not label based on the actual degradation of the scan (not by some kind of threshold of analog measurement threshold, statistical info about scan) since (TODO FIXME) this would result in training which only learns this given metric (example missing measurement points) which would make this methodology useless since we could simply use that same measurement as an more simple way to quantify the scan's degradation. }
|
||||||
\todo[inline]{TODO maybe evaluate based on different thresholds? missing datapoints, number of detected outliers, number of particles in phantom circle around sensor?}
|
\todo[inline]{TODO maybe evaluate based on different thresholds? missing datapoints, number of detected outliers, number of particles in phantom circle around sensor?}
|
||||||
|
|
||||||
|
|
||||||
\newchapter{Experimental Setup}{chap:experimental_setup}
|
\newchapter{experimental_setup}{Experimental Setup}
|
||||||
\newsection{DeepSAD Autoencoder Architecture}{sec:autoencoder_architecture}
|
\newsection{autoencoder_architecture}{DeepSAD Autoencoder Architecture}
|
||||||
\newsection{Training/Evaluation Data Distribution}{sec:data_setup}
|
\newsection{data_setup}{Training/Evaluation Data Distribution}
|
||||||
\todo[inline]{which data was used how in training/evaluation}
|
\todo[inline]{which data was used how in training/evaluation}
|
||||||
\todo[inline]{explain concept of global/local application for global-/window quantifiction}
|
\todo[inline]{explain concept of global/local application for global-/window quantifiction}
|
||||||
|
|
||||||
\newsection{Evaluation Metrics}{sec:evaluation_metrics}
|
\newsection{evaluation_metrics}{Evaluation Metrics}
|
||||||
\todo[inline]{k-fold evaluation, ROC, generalization (evaluation on other datasets?)}
|
\todo[inline]{k-fold evaluation, ROC, generalization (evaluation on other datasets?)}
|
||||||
|
|
||||||
\newsection{Hyperparameters}{sec:hyperparameters}
|
\newsection{hyperparameters}{Hyperparameters}
|
||||||
\todo[inline]{vary hyperparameters (no labeled anomalies vs some), specific training on local windows (only z-axis difference?), window size?}
|
\todo[inline]{vary hyperparameters (no labeled anomalies vs some), specific training on local windows (only z-axis difference?), window size?}
|
||||||
|
|
||||||
\newchapter{Results and Discussion}{chap:results_discussion}
|
\newchapter{results_discussion}{Results and Discussion}
|
||||||
\newsection{Results}{sec:results}
|
\newsection{results}{Results}
|
||||||
\todo[inline]{some results, ROC curves, for both global and local}
|
\todo[inline]{some results, ROC curves, for both global and local}
|
||||||
|
|
||||||
\newsection{Hyperparameter Analysis}{sec:hyperparameter_analysis}
|
\newsection{hyperparameter_analysis}{Hyperparameter Analysis}
|
||||||
\todo[inline]{result for different amounts of labeled data}
|
\todo[inline]{result for different amounts of labeled data}
|
||||||
|
|
||||||
\newchapter{Conclusion and Future Work}{chap:conclusion_future_work}
|
\newchapter{conclusion_future_work}{Conclusion and Future Work}
|
||||||
\newsection{Conclusion}{sec:conclusion}
|
\newsection{conclusion}{Conclusion}
|
||||||
\todo[inline]{summarize what has been achieved}
|
\todo[inline]{summarize what has been achieved}
|
||||||
|
|
||||||
\newsection{Future Work}{sec:future_work}
|
\newsection{future_work}{Future Work}
|
||||||
\todo[inline]{confirm results with real smoke data}
|
\todo[inline]{confirm results with real smoke data}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -40,18 +40,18 @@
|
|||||||
|
|
||||||
% new chapter, section, ... plus a few addons
|
% new chapter, section, ... plus a few addons
|
||||||
% part
|
% part
|
||||||
\newcommand{\newpart}[2]{\FloatBarrier\cleardoublepage\part{#1}\label{part:#2}}%
|
\newcommand{\newpart}[2]{\FloatBarrier\cleardoublepage\part{#2}\label{part:#1}}%
|
||||||
% chapter
|
% chapter
|
||||||
\newcommand{\newchapter}[2]{\FloatBarrier\chapter{#1}\label{chp:#2}\renewcommand{\pwd}{#2}}
|
\newcommand{\newchapter}[2]{\FloatBarrier\chapter{#2}\label{chp:#1}\renewcommand{\pwd}{#1}}
|
||||||
\newcommand{\newchapterNoTOC}[1]{\FloatBarrier\stepcounter{chapter}\chapter*{#1}}%
|
\newcommand{\newchapterNoTOC}[1]{\FloatBarrier\stepcounter{chapter}\chapter*{#1}}%
|
||||||
% section
|
% section
|
||||||
\newcommand{\newsection}[2]{\FloatBarrier\vspace{5mm}\section{#1}\label{sec:#2}}%
|
\newcommand{\newsection}[2]{\FloatBarrier\vspace{5mm}\section{#2}\label{sec:#1}}%
|
||||||
\newcommand{\newsectionNoTOC}[1]{\FloatBarrier\vspace{5mm}\stepcounter{section}\section*{#1}}%
|
\newcommand{\newsectionNoTOC}[1]{\FloatBarrier\vspace{5mm}\stepcounter{section}\section*{#1}}%
|
||||||
% subsection
|
% subsection
|
||||||
\newcommand{\newsubsection}[2]{\FloatBarrier\vspace{3mm}\subsection{#1}\label{sec:#2}}%
|
\newcommand{\newsubsection}[2]{\FloatBarrier\vspace{3mm}\subsection{#2}\label{sec:#1}}%
|
||||||
\newcommand{\newsubsectionNoTOC}[1]{\FloatBarrier\vspace{3mm}\stepcounter{subsection}\subsection*{#1}}%
|
\newcommand{\newsubsectionNoTOC}[1]{\FloatBarrier\vspace{3mm}\stepcounter{subsection}\subsection*{#1}}%
|
||||||
% subsubsection
|
% subsubsection
|
||||||
\newcommand{\newsubsubsection}[2]{\vspace{2mm}\subsubsection{#1}\label{sec:#2}}%
|
\newcommand{\newsubsubsection}[2]{\vspace{2mm}\subsubsection{#2}\label{sec:#1}}%
|
||||||
\newcommand{\newsubsubsectionNoTOC}[1]{\vspace{2mm}\stepcounter{subsubsection}\subsubsection*{#1}}%
|
\newcommand{\newsubsubsectionNoTOC}[1]{\vspace{2mm}\stepcounter{subsubsection}\subsubsection*{#1}}%
|
||||||
|
|
||||||
|
|
||||||
@@ -145,7 +145,7 @@
|
|||||||
|
|
||||||
% figures without frames
|
% figures without frames
|
||||||
% standard
|
% standard
|
||||||
\newcommand{\fig}[3]{\begin{figure}\centering\includegraphics[width=\textwidth]{#1}\caption{#2}\label{fig:#3}\end{figure}}%
|
\newcommand{\fig}[3]{\begin{figure}\centering\includegraphics[width=\textwidth]{#2}\caption{#3}\label{fig:#1}\end{figure}}%
|
||||||
% with controllable parameters
|
% with controllable parameters
|
||||||
\newcommand{\figc}[4]{\begin{figure}\centering\includegraphics[#1]{#2}\caption{#3}\label{fig:#4}\end{figure}}%
|
\newcommand{\figc}[4]{\begin{figure}\centering\includegraphics[#1]{#2}\caption{#3}\label{fig:#4}\end{figure}}%
|
||||||
% two subfigures
|
% two subfigures
|
||||||
@@ -199,4 +199,4 @@
|
|||||||
% slighly darkened colors for text
|
% slighly darkened colors for text
|
||||||
\definecolor{bkred}{rgb}{0.9,0,0}
|
\definecolor{bkred}{rgb}{0.9,0,0}
|
||||||
\definecolor{bkgreen}{rgb}{0,0.67,0}
|
\definecolor{bkgreen}{rgb}{0,0.67,0}
|
||||||
\definecolor{bkblue}{rgb}{0,0,0.75}
|
\definecolor{bkblue}{rgb}{0,0,0.75}
|
||||||
|
|||||||
BIN
thesis/figures/data_screenshot_camera.png
Normal file
BIN
thesis/figures/data_screenshot_camera.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 783 KiB |
BIN
thesis/figures/data_screenshot_pointcloud.png
Normal file
BIN
thesis/figures/data_screenshot_pointcloud.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 221 KiB |
Reference in New Issue
Block a user