Error Internal at: 346 - Empty obj.file

[SeskoDk]

Dear author,

thank you very much for your great work.

Issue:

I am not able to generate a mesh (obj.file) based on my own custom dataset.
I used the provided branch neuspp, colmap and the colmap2nerf.py to generate a transform.json.
This generated transform.json is compatible with the instant-ngp pipeline, but unfortunately not with neus2pp.
Based on my understanding, the instant-ngp trasnform.json should also work with neus2pp.
Perhaps I missed a specific step or did not consider a required format, I am not sure.

My Approach

1. I took the pictures with my phone
2. Used colmap to generate the follwoing files:

DATA_PATH
 ├─ images (Phone)
 ├─ sparse   (Colmap)
 ├─ dense     (Colmap)
 ├─ cameras.text (Colmap)
 ├─ database.db  (Colmap)
 ├─ images.txt  (Colmap)
 ├─ points3D.txt (Colmap)
 ├─ project.ini (Colmap)

3. Generated the transform.json file via colmap2nerf.py from instant-npg, using the follwoing command:

instant-ngp$ python scripts/colmap2nerf.py --images 'DATA_PATH/images' --text 'DATA_PATH' --aabb_scale 32

The resulted transform.json is organized as follows:

{
  "camera_angle_x": 0.8743924098673197,
  "camera_angle_y": 1.114530598807743,
  "fl_x": 3183.842367574047,
  "fl_y": 3183.842367574047,
  "k1": 0.0015741206657167866,
  "k2": 0,
  "k3": 0,
  "k4": 0,
  "p1": 0,
  "p2": 0,
  "is_fisheye": false,
  "cx": 1488.0,
  "cy": 1984.0,
  "w": 2976.0,
  "h": 3968.0,
  "aabb_scale": 32,
  "frames": [
    {
      "file_path": ".data/images/00001.jpg",
      "sharpness": 8.348386299231485,
      "transform_matrix": [
        [
          -0.006901961827569161,
          -0.38868945495233237,
          0.9213429712032302,
          3.875667772007341
        ],
        [
          0.9938922933833255,
          -0.10414655703566049,
          -0.03649114702771986,
          -0.018699091474430635
        ],
        [
          0.11013842234861199,
          0.9154638181379564,
          0.3870342693576213,
          1.4236670216607075
        ],
        [
          0.0,
          0.0,
          0.0,
          1.0
        ]
      ]
    },
    ...
  ]
}

4. Run the following command, to generate a mesh as obj.file:

neuspp$  python scripts/run.py --scene transform.json --name custom_mesh --network base.json --n_steps 15000

5. Resulted in an empty obj.file with the Info:

INFO     #vertices=0 #triangles=0
Got cutlass error: Error Internal at: 346

[19reborn]

Hi @SeskoDk, for the neuspp branch, you need to change the aabb_scale in the transform.json to 1, and adjust the scale and offset parameters to make your interested object in the aabb.

[19reborn]

Hi @SeskoDk , I'm sorry I didn't notice you have used the base.json. You should use the womask.json and the result should be fine then.

[SeskoDk]

Hi @yiming Wang ***@***.***>, does this mean, I can use the colmap2nerf.py to generate the transform.json and afterwards pass the transform.json to the neus.run.py file with --network config/nerf/womask.json? Am Mo., 18. Sept. 2023 um 11:20 Uhr schrieb Yiming Wang < ***@***.***>:

…

Hi @SeskoDk <https://github.com/SeskoDk> , I'm sorry I didn't notice you have used the base.json. You should use the womask.json and the result should be fine then. — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHB2F3XKVLDRU7LYTPLX3AG5LANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[19reborn]

Yes, you can use --network womask.json instead of --network base.json.

[SeskoDk]

I have now started the run.py with the womask.json. Unfortunately, the mesh of the dwarf is not representative. Attached, you will find the resulting mesh. Were you able to create a mesh for the dwarf? [image: dwarf_mesh.png] [image: dwarf_2.png] [image: dwarf_3.png] Am Mo., 18. Sept. 2023 um 11:51 Uhr schrieb Yiming Wang < ***@***.***>:

…

Yes, you can use --network womask.json instead of --network base.json. — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHG7SVEPL4ORUGS4JJLX3AKR5ANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[19reborn]

Hi @SeskoDk , I can get the correct mesh using your provided data.

[SeskoDk]

This is quite interesting and at the same time confusing to me. Can you tell me which steps you did to accomplish this beautiful mesh? Thank you in advance for everything BR Sesko Yiming Wang ***@***.***> schrieb am Mo., 18. Sep. 2023, 18:35:

…

Hi @SeskoDk <https://github.com/SeskoDk> , I can get the correct mesh using your provided data. [image: image] <https://user-images.githubusercontent.com/55176925/268714028-da0badac-882f-4a0d-ac14-6067214ddb94.png> — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHGDHTXQDIBIA5TWMJTX3BZ5VANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[19reborn]

I trained it for 5000 steps. I also encountered the Cutlass Error: Error Internal at: 346, but the code output the mesh before it broke down. Could you please re-check the codes you use? It should be in the neuspp branch and run with womask.json.

[SeskoDk]

I finally achieved the same results with the testbed by rebuilding the repo. Perhaps I messed up my conda environment. Thank you very much for your patience and support. I only have one problem. Unfortunately to launch the run.py I have to solve that Issue (see below). According to the Isse #43 NVlabs/instant-ngp#43 I have to install python 3.10 but that approach did not work for me. Can you help me in this regard? [image: pyngp.png] Am Mo., 18. Sept. 2023 um 19:34 Uhr schrieb Yiming Wang < ***@***.***>:

…

I trained it for 5000 steps. I also encountered the Cutlass Error: Error Internal at: 346, but the code output the mesh before it broke down. Could you please re-check the codes you use? It should be in the neuspp branch and run with womask.json. — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHG3XM2MJ54RNWSPI3TX3CA3HANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[19reborn]

Hi @SeskoDk , it's because the cmake didn't find the Python path in your environment. You can try adjusting your system path or set it manually for your cmake compiling.

[SeskoDk]

Hi Yiming, thanks I managed to solve that problem. But what is interesting to me, is that even when I use the neuspp branch with the colmap the run.py crashes like the mesh disappears. I hope I am not to disturbing but, can you maybe manage to generate a suitable mesh with this custom dataset? https://drive.google.com/file/d/1gkrBu-HNSMDSSmi18vrqp_SEswR2B5IR/view?usp=drive_link That above link contains the images and the corresponding transform.json file. THank you very much for you patience and understanding. BR Sesko Am Do., 21. Sept. 2023 um 15:37 Uhr schrieb Yiming Wang < ***@***.***>:

…

Hi @SeskoDk <https://github.com/SeskoDk> , it's because the cmake didn't find the Python path in your environment. You can try adjusting your system path or set it manually for your cmake compiling. — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHAU7JKRNJ6HYZLHWSLX3Q7H5ANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[19reborn]

Hi @SeskoDk, I generate a mesh here. https://drive.google.com/file/d/1xRfDVpAjBes9kzm4aTlTEn4bGkGsVwUP/view?usp=drive_link

[SeskoDk]

Wow, I dont get it. May I kindly request you to explain once again what exactly you did? In essence, I did it exactly as we discussed, but I didn't get these results. Did you make any changes in the Transform.json file, or what exactly are you doing differently? Thank you in advance, and warm regards. Am Mo., 25. Sept. 2023 um 12:52 Uhr schrieb Yiming Wang < ***@***.***>:

…

Hi @SeskoDk <https://github.com/SeskoDk>, I generate a mesh here. https://drive.google.com/file/d/1xRfDVpAjBes9kzm4aTlTEn4bGkGsVwUP/view?usp=drive_link — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHD766LG2B7MGIRB3ITX4FPAPANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[19reborn]

Hi @SeskoDk, I set the SDF_GRID in line 39 of testbed_nerf.cu to 1 (rember to cmake again after the change), the scale in the transform.json to 0.25, and the n_frequencies in the bg_network/encoding of the womask.json to 10.

[SeskoDk]

Thank you very much for you help, now I also was able to do it. Can you tell me how to interpret these changes? like is why do you set these values? I would like to be able to understand this change to better create future scenes. Am Mo., 25. Sept. 2023 um 13:18 Uhr schrieb Yiming Wang < ***@***.***>:

…

Hi @SeskoDk <https://github.com/SeskoDk>, I set the SDF_GRID in line 39 of testbed_nerf.cu to 1 (rember to cmake again after the change), the scale in the transform.json to 0.25, and the n_frequencies in the bg_network/encoding of the womask.json to 10. — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APK2UHHT4M5KORWUJLNT7KLX4FR6TANCNFSM6AAAAAA4UW5NLI> . You are receiving this because you were mentioned.Message ID: ***@***.***>