ARO-Net encodes contextual information from anchors, which is different against most existing methods (such as Points2Surf and ConvONet) that encode neighboring information around query point, yielding better reconstruction (see the holes) on sparse input and generalizability on unseen categories.
We introduce anchored radial observations (ARO), a novel shape encoding for learning implicit field representation of 3D shapes that is category-agnostic and generalizable amid significant shape variations.
The main idea behind our work is to reason about shapes through partial observations from a set of viewpoints, called anchors. We develop a general and unified shape representation by employing a fixed set of anchors, via Fibonacci sampling, and designing a coordinate-based deep neural network to predict the occupancy value of a query point in space. Differently from prior neural implicit models that use global shape feature, our shape encoder operates on contextual, query-specific features. To predict point occupancy, locally observed shape information from the perspective of the anchors surrounding the input query point are encoded and aggregated through an attention module, before implicit decoding is performed.
We demonstrate the quality and generality of our network, coined ARO-Net, on surface reconstruction from sparse point clouds, with tests on novel and unseen object categories, "one-shape" training, and comparisons to state-of-the-art neural and classical methods for reconstruction and tessellation.
If we have a set of anchors that together observe the entire surface of the shape, whether a query point is inside/outside can be determined by its relationship to the radial observations of anchors. For example, in subfigure (b) when anchors are all interior,
a query point (red star) is inside the shape ⇔ it is covered by at least one radial observation (the red anchor)
ARO learns an implicit function by modeling the observations from different anchors to query point, which is contextual and query-specific, making it quite different from existing methods.
In practice, shapes can only be partially observed by a set of anchors. We introduce ARO-Net to predict the occupancy from less/incomplete Information after training with data. ARO-Net learns to predict the occupancy of a query point given only partial observations.
We use Fibonacci Sampling to construct the anchors. Layered Fibonacci sampling makes the anchors have generally good visibility about the shapes.
A somewhat extreme toy example comparing ARO-Net to prior occupancy prediction networks on 3D reconstruction from a sparse point cloud of a cube (a), with training on a single sphere.
Visually, ARO-Net produces the most complete, artifact-free results with faithful reconstruction of fine details. In this airplane case, ARO-Net managed to reconstruct the left-side turbine from extremely sparse points, while other methods all completely missed it.
ARO-Net exhibits superior robustness to sparsity of input point clouds compared to its close competitors.
When all methods are trained on 2048-point input and tested on 2048-point input, ARO achieves the best reconstruction quality while results from others are also okay.
However, when testing on 1024 and 512 points as input, ARO-Net performs much better than other SOTA methods.
@article{wang2023aro,
author = {Wang, Yizhi and Huang, Zeyu and Shamir, Ariel and Huang, Hui and Zhang, Hao and Hu, Ruizhen},
title = {ARO-Net: Learning Implicit Fields from Anchored Radial Observations},
journal = {CVPR},
year = {2023},
}