Point Cloud Drawbacks

As you may know, I am a big point cloud enthusiast. It is that long ago I discovered point clouds (8 years), but I had the chance to begin with the explosion in Reality Capture tools. I still remember though my first hands on laser scanners, getting these amazing (and still amazing) point clouds in minutes, but then... processing (read manual overloaded repetitive digitization)... Filtering, registration, cleaning, segmenting, classifying, meshing, digitizing ... All these were heavily manual. It evolved for some parts (mainly registration, filtering and meshing) but the main bottleneck that I had at the time is still unresolved : why do we bother changing the nature of the data (point cloud to vector) per application ? Is there not a more efficient workflow ?

Analogous to our visual and cognitive system, 3 steps will condition the completeness of the surveyed object. First, the perception, i.e., how the visual system processes the visual information to construct a structured description of the shape of the object/scene. Second, the shape recognition or how the product of the perceptual treatment will contact stored representations in the form of known objects (it will construct a perceptual depiction that will be a representation of the same nature stored in memory). Finally, the identification (labelling), i.e., when a stored structural representation is activated, it will in turn activate the unit of meaning (concept) that corresponds to it, located in the semantic system.

Sensors are the analogue to our perception, and aim at extracting the visual stimuli it is sensitive to (spatial information, colour, luminance, movement, etc.). At this stage, neither the information on the shape of the object nor the label is extracted. In the case of 3D remote sensing, the quality of observation is therefore critical to enable high quality and relevant information extraction about the application. As such, the sensory perceptive processing capable of extracting visual primitives must be as objective and complete as possible, making sensors for point cloud generation favourable. Constituted of a multitude of points, they are a great way to reconstruct environments tangibly, and enabling further primitive’s extractions (discontinuity, corners, edges, contour„ etc.) as in our perceptive visual system. However, their lack of semantics makes them a bona-fide spatial representation, thus of limited value if not enhanced.

And this is were I want to get to : we need intelligence within our virtual datasets ! This, in order to avoid brain work and manual processes, but also for the sake of interoperability. There are many applications that will use point clouds differently, but extracting deliverables per application doesn't seems to be the most efficient (and eco-friendly in terms of storage footprint). However, if experts knowledge is formalized and integrated within point clouds, you can only guess how centralized and efficient an infrastructure becomes ?

The following research article "Model for reasoning from semantically rich point cloud data" gives a conceptual model to integrate these varying informations within point clouds and to make them available to reasoners. This is a big step toward higher interoperability and virtual environments. The next step, is to automate the recognition of feature and object of interest for one specific application. This is ongoing developments at the moment I write this post, and first results based on Neural Networks are promising (but I need more data to train models, so please, if you want to contribute I will be happy to receive any 3D point cloud data :)).

In the example above extracted from the exhaustive MDPI article "3D Point Clouds in Archaeology: Advances in Acquisition, Processing and Knowledge Integration Applied to Quasi-Planar Objects", we can see how formal knowledge in ontologies can directly help classification routines and be useful for experts and non-experts.

So yes, point clouds are huge; yes we need specific "tricks" to store them and process them, but so were videos back some decades ago ! So what does this imply in your specific industry ? That you will soon be able to work with a "brain representation" of the 3D captured environment for your application for you to query as you deem. But of course, the big landscape in 3D sensors make the recognition process not reliable enough (at least to my eyes). I have many datasets from Leica P30, some from Faro, but if you are interested, specific dataset that comprises all you want to detect for your specific application will greatly enhance the success ratio (and help me in my public research) !