Nerf training generated 0 samples, Aborting training

[Learningm]

Hi, I want to try this excellent reconstruction method, and I convert my own dataset from nerfstudio format to neus2 format.

I wonder what the value 'offset' & 'scale' should be, so I use the default value [0.5, 0.5, 0.5] & 0.5 as in the data_convention.md. I check ngp tutorial and it seems the 'offset' need to use visualization to determinate.

After I set the default value, and copy the corresponding intrinsic_matrix and transform_matrix from nerfstudio format, I run neus2 training and got this error : Nerf training generated 0 samples, Aborting training.

How to fix it? My own data can be successfully trained using nerfstudio. Thanks !

[Anirudh-Kakarlapudi]

Hi @Learningm, Please try using base.json as config file.

[Learningm]

Hi @Learningm, Please try using base.json as config file.

Hi @Anirudh-Kakarlapudi ，same error using base.json.
The error seems occurs here:

NeuS2/src/testbed_nerf.cu

Line 3540 in 61f9cbf

tlog::warning() << "Nerf training generated 0 samples. Aborting training.";

I also tried several options, including 'from_na' : true or false, 'scale' : 0.5 or 1.0, 'offset' : [0.5, 0.5, 0.5] or [0, 0, 0], it remains same error, I guess it maybe dataset problem. Could you please figure me out which function to do the dataset rescale and recenter operation? It's a little bit hard for understanding the cuda code.

[brianneoberson]

Hi @Learningm , I ran into the same problem and was able to fix it by making sure my object was within the bounding box. I used IDR's camera preprocessing to normalize my scene with respect to the visual hull. Hope this helps! :)

[FeiiYin]

@brianneoberson
Hi, I want to ask about the dataset preprocessing. I use a synthesize dataset from sapien, which can run on threestudio. It cannot run here, raise Nerf training generated 0 samples, Aborting training.

And I follow #26 (comment)
to preprocess the dataset, it raise the bellow error. It seems that my input data (camera or mask) is not right.
My input camera is proj_mtx_new, mask is extracted from the alpha of a png.
Thanks in advance!

c2w = transform[:3, :4] # transform is a synthesized parameters
mv = torch.linalg.inv(transform)
fovy =  np.random.uniform(self.fovy_range_min, self.fovy_range_max)
self.resolution = [512, 512]
K = np.array([
                [fovy, 0, self.resolution[0] / 2, 0],
                [0, fovy, self.resolution[1] / 2, 0],
                [0, 0, 1, 0],
                [0, 0, 0, 1]
            ])
K = torch.from_numpy(K).float()
proj_mtx_new = K @ mv

Number of points:0
/mnt/petrelfs/wangtengfei/code/NeuS2/tools/preprocess_cameras.py:220: RuntimeWarning: Mean of empty slice.
  centroid = np.array(all_Xs).mean(axis=0)
/opt/conda/lib/python3.9/site-packages/numpy/core/_methods.py:192: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
/opt/conda/lib/python3.9/site-packages/numpy/core/_methods.py:269: RuntimeWarning: Degrees of freedom <= 0 for slice
  ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof,
/opt/conda/lib/python3.9/site-packages/numpy/core/_methods.py:226: RuntimeWarning: invalid value encountered in divide
  arrmean = um.true_divide(arrmean, div, out=arrmean,
/opt/conda/lib/python3.9/site-packages/numpy/core/_methods.py:261: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
  File "/mnt/petrelfs/wangtengfei/code/NeuS2/tools/preprocess_cameras.py", line 312, in <module>
    get_normalization(opt.source_dir, opt.use_linear_init, dataset)
  File "/mnt/petrelfs/wangtengfei/code/NeuS2/tools/preprocess_cameras.py", line 266, in get_normalization
    normalization, all_Xs = get_normalization_function(Ps, mask_points_all, number_of_normalization_points, number_of_cameras,masks_all)
  File "/mnt/petrelfs/wangtengfei/code/NeuS2/tools/preprocess_cameras.py", line 225, in get_normalization_function
    centroid,scale,all_Xs = refine_visual_hull(masks_all, Ps, scale, centroid)
  File "/mnt/petrelfs/wangtengfei/code/NeuS2/tools/preprocess_cameras.py", line 151, in refine_visual_hull
    points = points + center[:, np.newaxis]
IndexError: invalid index to scalar variable.