Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[question] How to save Meshroom files of Google Colab, so you can run it afterwards (the 12 hour time limit) #767

Open
ironman26820 opened this issue Jan 21, 2020 · 7 comments

Comments

@ironman26820
Copy link

ironman26820 commented Jan 21, 2020

Describe the problem
I have taken 177 images of an object (I know its a lot of photos) and tried to make a model about 6 times on my PC. It would always crash on either Feature Extraction or DepthMap. I then saw about Google Colab and so I uploaded all the images on Google Colab and started the meshing. I am now at DepthMap 24/59, and around 6 hours after which Colab will terminate it. Taking average times, I can determine that the DepthMap will take 4-5 hours to complete. I can download all the data (as soon as Depth is completed) on my Computer and then continue it afterwards. I will paste all the MeshroomCache in the same file path (/tmp/MeshroomCache), and the images in the path I have right now. But how will I continue the after DepthMap, so I do not lose any progress.

Also in other issue #251, I saw that I should reduce the maxPoints, and how should I do that on GoogleColab. I am sorry, but I am a NOOB at CLI and python.

Desktop (please complete the following and other pertinent information):

  • OS: Google Colab (Free Version) [Using on Windows 10 Pro 64 Bit]
  • Python version - Python 3
  • Qt/PySide version [e.g. 5.12.4]
  • Meshroom version: Meshroom-2019.1.0-linux
@ironman26820
Copy link
Author

Just Now I got this error,
7/12 DepthMap

  • commandLine: aliceVision_depthMapEstimation --input "/tmp/MeshroomCache/StructureFromMotion/099754c5e35d9f436d09bfb804aa2d1264485b4f/sfm.abc" --imagesFolder "/tmp/MeshroomCache/PrepareDenseScene/601629980a54fc1b3e2ce37ec0d499d9a1ef78b6" --downscale 2 --minViewAngle 2.0 --maxViewAngle 70.0 --sgmMaxTCams 10 --sgmWSH 4 --sgmGammaC 5.5 --sgmGammaP 8.0 --refineMaxTCams 6 --refineNSamplesHalf 150 --refineNDepthsToRefine 31 --refineNiters 100 --refineWSH 3 --refineSigma 15 --refineGammaC 15.5 --refineGammaP 8.0 --refineUseTcOrRcPixSize False --exportIntermediateResults False --nbGPUs 0 --verboseLevel info --output "/tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd" --rangeStart 69 --rangeSize 3
  • logFile: /tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd/23.log
  • elapsed time: 0:05:31.138883

7/12 DepthMap

  • commandLine: aliceVision_depthMapEstimation --input "/tmp/MeshroomCache/StructureFromMotion/099754c5e35d9f436d09bfb804aa2d1264485b4f/sfm.abc" --imagesFolder "/tmp/MeshroomCache/PrepareDenseScene/601629980a54fc1b3e2ce37ec0d499d9a1ef78b6" --downscale 2 --minViewAngle 2.0 --maxViewAngle 70.0 --sgmMaxTCams 10 --sgmWSH 4 --sgmGammaC 5.5 --sgmGammaP 8.0 --refineMaxTCams 6 --refineNSamplesHalf 150 --refineNDepthsToRefine 31 --refineNiters 100 --refineWSH 3 --refineSigma 15 --refineGammaC 15.5 --refineGammaP 8.0 --refineUseTcOrRcPixSize False --exportIntermediateResults False --nbGPUs 0 --verboseLevel info --output "/tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd" --rangeStart 72 --rangeSize 3
  • logFile: /tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd/24.log
  • elapsed time: 0:07:24.743936

7/12 DepthMap

  • commandLine: aliceVision_depthMapEstimation --input "/tmp/MeshroomCache/StructureFromMotion/099754c5e35d9f436d09bfb804aa2d1264485b4f/sfm.abc" --imagesFolder "/tmp/MeshroomCache/PrepareDenseScene/601629980a54fc1b3e2ce37ec0d499d9a1ef78b6" --downscale 2 --minViewAngle 2.0 --maxViewAngle 70.0 --sgmMaxTCams 10 --sgmWSH 4 --sgmGammaC 5.5 --sgmGammaP 8.0 --refineMaxTCams 6 --refineNSamplesHalf 150 --refineNDepthsToRefine 31 --refineNiters 100 --refineWSH 3 --refineSigma 15 --refineGammaC 15.5 --refineGammaP 8.0 --refineUseTcOrRcPixSize False --exportIntermediateResults False --nbGPUs 0 --verboseLevel info --output "/tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd" --rangeStart 75 --rangeSize 3
  • logFile: /tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd/25.log
  • elapsed time: 0:00:43.149818
    ERROR:root:Error on node computation: Error on node "DepthMap_1(25)":
    Log:
    Program called with the following parameters:
  • downscale = 2
  • exportIntermediateResults = 0
  • imagesFolder = "/tmp/MeshroomCache/PrepareDenseScene/601629980a54fc1b3e2ce37ec0d499d9a1ef78b6"
  • input = "/tmp/MeshroomCache/StructureFromMotion/099754c5e35d9f436d09bfb804aa2d1264485b4f/sfm.abc"
  • maxViewAngle = 70
  • minViewAngle = 2
  • nbGPUs = 0
  • output = "/tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd"
  • rangeSize = 3
  • rangeStart = 75
  • refineGammaC = 15.5
  • refineGammaP = 8
  • refineMaxTCams = 6
  • refineNDepthsToRefine = 31
  • refineNSamplesHalf = 150
  • refineNiters = 100
  • refineSigma = 15
  • refineUseTcOrRcPixSize = 0
  • refineWSH = 3
  • sgmGammaC = 5.5
  • sgmGammaP = 8
  • sgmMaxTCams = 10
  • sgmWSH = 4
  • verboseLevel = "info"

[17:57:43.150186][info] CUDA-Enabled GPU.
Device information:
- id: 0
- name: Tesla T4
- compute capability: 7.5
- total device memory: 15079 MB
- device memory available: 14939 MB
- per-block shared memory: 49152
- warp size: 32
- max threads per block: 1024
- max threads per SM(X): 1024
- max block sizes: {1024,1024,64}
- max grid sizes: {2147483647,65535,65535}
- max 2D array texture: {131072,65536}
- max 3D array texture: {16384,16384,16384}
- max 2D linear texture: {131072,65000,2097120}
- max 2D layered texture: {32768,32768,2048}
- number of SM(x)s: 40
- registers per SM(x): 65536
- registers per block: 65536
- concurrent kernels: yes
- mapping host memory: yes
- unified addressing: yes
- texture alignment: 512 byte
- pitch alignment: 32 byte

[17:57:43.745351][info] Supported CUDA-Enabled GPU detected.
[17:57:43.792825][info] Found 1 image dimension(s):
[17:57:43.792877][info] - [8000x6000]
[17:57:43.961055][info] Overall maximum dimension: [8000x6000]
[17:57:43.961187][info] Create depth maps.
[17:57:43.961393][info] # GPU devices: 1, # CPU threads: 2
[17:57:43.961430][info] Plane sweeping parameters:
- scale: 2
- step: 6
[17:57:43.961514][info] PlaneSweepingCuda:
- _nImgsInGPUAtTime: 2
- scales: 2
- subPixel: Yes
- varianceWSH: 4
[17:57:43.963224][info] Found only 1/10 nearest cameras for view id: 1613347920
[17:57:43.964780][info] Found only 1/6 nearest cameras for view id: 1613347920
[17:57:43.964850][info] Estimate depth map, view id: 1613347920
tcmalloc: large alloc 1728004096 bytes == 0x29e7c000 @ 0x7f2e6cb23887 0x7f2e6a06aff1 0x7f2e6a06b7b9 0x7f2e6a05f8a8 0x7f2e6a293e5c 0x7f2e6a295a88 0x7f2e6a2978a2 0x7f2e6c59f0fa 0x7f2e6c5a0148 0x7f2e6c5a23ae 0x7f2e6c59a94a 0x7f2e6c595afd 0x7f2e6c5878cf 0x7f2e6c5899a8 0x41206e 0x7f2e61932b97 0x412bc1
tcmalloc: large alloc 1728004096 bytes == 0xa29fc000 @ 0x7f2e6cb23887 0x7f2e6a06aff1 0x7f2e6a06b7b9 0x7f2e6a05f8a8 0x7f2e6a293e5c 0x7f2e6a295a88 0x7f2e6a2978a2 0x7f2e6c59f0fa 0x7f2e6c5a0148 0x7f2e6c5a24f5 0x7f2e6c59a94a 0x7f2e6c595afd 0x7f2e6c5878cf 0x7f2e6c5899a8 0x41206e 0x7f2e61932b97 0x412bc1
number of CUDA devices: 1
0: Tesla T4
CUDA device no 0 for 0
Device 0 memory - used: 139.875000, free: 14939.875000, total: 15079.750000
Device 0 memory - used: 603.875000, free: 14475.875000, total: 15079.750000
Device 0 memory - used: 669.875000, free: 14409.875000, total: 15079.750000
total size of volume map in GPU memory: 65.917969
nDepths 540, nDepthsToSearch 540
ps_computeSimilarityVolume elapsed time: 4.188000 ms
ps_SGMoptimizeSimVolume
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
SGM volume gpu elapsed time: 224.194000 ms
ps_SGMoptimizeSimVolume done
[17:58:17.245938][info] Refine depth map, view id: 1613347920
gpu elapsed time: 669.934998 ms
gpu elapsed time: 670.077026 ms
gpu elapsed time: 672.109985 ms
gpu elapsed time: 662.310974 ms
gpu elapsed time: 153.518005 ms
gpu elapsed time: 154.503006 ms
gpu elapsed time: 155.882996 ms
gpu elapsed time: 152.837006 ms
rc: 75, gpu elapsed time: 270.050995 ms
rc: 75, gpu elapsed time: 271.628998 ms
rc: 75, gpu elapsed time: 272.488007 ms
rc: 75, gpu elapsed time: 266.321991 ms
[17:58:24.986659][info] Found only 0/10 nearest cameras for view id: 1630503138
[17:58:24.987401][info] Found only 0/6 nearest cameras for view id: 1630503138
[17:58:24.987444][info] Estimate depth map, view id: 1630503138
[17:58:24.987453][info] Refine depth map, view id: 1630503138
Segmentation fault (core dumped)

WARNING: downgrade status on node "DepthMap_1(26)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(27)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(28)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(29)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(30)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(31)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(32)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(33)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(34)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(35)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(36)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(37)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(38)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(39)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(40)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(41)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(42)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(43)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(44)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(45)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(46)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(47)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(48)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(49)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(50)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(51)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(52)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(53)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(54)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(55)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(56)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(57)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMap_1(58)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(0)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(1)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(2)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(3)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(4)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(5)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(6)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(7)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(8)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(9)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(10)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(11)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(12)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(13)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(14)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(15)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(16)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "DepthMapFilter_1(17)" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "Meshing_1" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "MeshFiltering_1" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "Texturing_1" from Status.SUBMITTED to Status.NONE
WARNING: downgrade status on node "Publish_1" from Status.SUBMITTED to Status.NONE
Traceback (most recent call last):
File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/cx_Freeze/initscripts/startup.py", line 14, in run
File "/opt/Meshroom/setupInitScriptUnix.py", line 39, in run
File "bin/meshroom_photogrammetry", line 125, in
File "/opt/Meshroom/meshroom/core/graph.py", line 1131, in executeGraph
File "/opt/Meshroom/meshroom/core/node.py", line 271, in process
File "/opt/Meshroom/meshroom/core/desc.py", line 452, in processChunk
RuntimeError: Error on node "DepthMap_1(25)":
Log:
Program called with the following parameters:

  • downscale = 2
  • exportIntermediateResults = 0
  • imagesFolder = "/tmp/MeshroomCache/PrepareDenseScene/601629980a54fc1b3e2ce37ec0d499d9a1ef78b6"
  • input = "/tmp/MeshroomCache/StructureFromMotion/099754c5e35d9f436d09bfb804aa2d1264485b4f/sfm.abc"
  • maxViewAngle = 70
  • minViewAngle = 2
  • nbGPUs = 0
  • output = "/tmp/MeshroomCache/DepthMap/cd191635d5b97fe7a38da50e5bdaad88b5a858bd"
  • rangeSize = 3
  • rangeStart = 75
  • refineGammaC = 15.5
  • refineGammaP = 8
  • refineMaxTCams = 6
  • refineNDepthsToRefine = 31
  • refineNSamplesHalf = 150
  • refineNiters = 100
  • refineSigma = 15
  • refineUseTcOrRcPixSize = 0
  • refineWSH = 3
  • sgmGammaC = 5.5
  • sgmGammaP = 8
  • sgmMaxTCams = 10
  • sgmWSH = 4
  • verboseLevel = "info"

[17:57:43.150186][info] CUDA-Enabled GPU.
Device information:
- id: 0
- name: Tesla T4
- compute capability: 7.5
- total device memory: 15079 MB
- device memory available: 14939 MB
- per-block shared memory: 49152
- warp size: 32
- max threads per block: 1024
- max threads per SM(X): 1024
- max block sizes: {1024,1024,64}
- max grid sizes: {2147483647,65535,65535}
- max 2D array texture: {131072,65536}
- max 3D array texture: {16384,16384,16384}
- max 2D linear texture: {131072,65000,2097120}
- max 2D layered texture: {32768,32768,2048}
- number of SM(x)s: 40
- registers per SM(x): 65536
- registers per block: 65536
- concurrent kernels: yes
- mapping host memory: yes
- unified addressing: yes
- texture alignment: 512 byte
- pitch alignment: 32 byte

[17:57:43.745351][info] Supported CUDA-Enabled GPU detected.
[17:57:43.792825][info] Found 1 image dimension(s):
[17:57:43.792877][info] - [8000x6000]
[17:57:43.961055][info] Overall maximum dimension: [8000x6000]
[17:57:43.961187][info] Create depth maps.
[17:57:43.961393][info] # GPU devices: 1, # CPU threads: 2
[17:57:43.961430][info] Plane sweeping parameters:
- scale: 2
- step: 6
[17:57:43.961514][info] PlaneSweepingCuda:
- _nImgsInGPUAtTime: 2
- scales: 2
- subPixel: Yes
- varianceWSH: 4
[17:57:43.963224][info] Found only 1/10 nearest cameras for view id: 1613347920
[17:57:43.964780][info] Found only 1/6 nearest cameras for view id: 1613347920
[17:57:43.964850][info] Estimate depth map, view id: 1613347920
tcmalloc: large alloc 1728004096 bytes == 0x29e7c000 @ 0x7f2e6cb23887 0x7f2e6a06aff1 0x7f2e6a06b7b9 0x7f2e6a05f8a8 0x7f2e6a293e5c 0x7f2e6a295a88 0x7f2e6a2978a2 0x7f2e6c59f0fa 0x7f2e6c5a0148 0x7f2e6c5a23ae 0x7f2e6c59a94a 0x7f2e6c595afd 0x7f2e6c5878cf 0x7f2e6c5899a8 0x41206e 0x7f2e61932b97 0x412bc1
tcmalloc: large alloc 1728004096 bytes == 0xa29fc000 @ 0x7f2e6cb23887 0x7f2e6a06aff1 0x7f2e6a06b7b9 0x7f2e6a05f8a8 0x7f2e6a293e5c 0x7f2e6a295a88 0x7f2e6a2978a2 0x7f2e6c59f0fa 0x7f2e6c5a0148 0x7f2e6c5a24f5 0x7f2e6c59a94a 0x7f2e6c595afd 0x7f2e6c5878cf 0x7f2e6c5899a8 0x41206e 0x7f2e61932b97 0x412bc1
number of CUDA devices: 1
0: Tesla T4
CUDA device no 0 for 0
Device 0 memory - used: 139.875000, free: 14939.875000, total: 15079.750000
Device 0 memory - used: 603.875000, free: 14475.875000, total: 15079.750000
Device 0 memory - used: 669.875000, free: 14409.875000, total: 15079.750000
total size of volume map in GPU memory: 65.917969
nDepths 540, nDepthsToSearch 540
ps_computeSimilarityVolume elapsed time: 4.188000 ms
ps_SGMoptimizeSimVolume
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
ps_updateAggrVolume
ps_aggregatePathVolume
ps_aggregatePathVolume done
ps_updateAggrVolume done
SGM volume gpu elapsed time: 224.194000 ms
ps_SGMoptimizeSimVolume done
[17:58:17.245938][info] Refine depth map, view id: 1613347920
gpu elapsed time: 669.934998 ms
gpu elapsed time: 670.077026 ms
gpu elapsed time: 672.109985 ms
gpu elapsed time: 662.310974 ms
gpu elapsed time: 153.518005 ms
gpu elapsed time: 154.503006 ms
gpu elapsed time: 155.882996 ms
gpu elapsed time: 152.837006 ms
rc: 75, gpu elapsed time: 270.050995 ms
rc: 75, gpu elapsed time: 271.628998 ms
rc: 75, gpu elapsed time: 272.488007 ms
rc: 75, gpu elapsed time: 266.321991 ms
[17:58:24.986659][info] Found only 0/10 nearest cameras for view id: 1630503138
[17:58:24.987401][info] Found only 0/6 nearest cameras for view id: 1630503138
[17:58:24.987444][info] Estimate depth map, view id: 1630503138
[17:58:24.987453][info] Refine depth map, view id: 1630503138
Segmentation fault (core dumped)

@ironman26820
Copy link
Author

ironman26820 commented Jan 21, 2020

I am downloading the Cache File (/tmp/MeshroomCache), and file do nothing until I get a response from someone. Please help this noob!

@natowi
Copy link
Member

natowi commented Jan 21, 2020

But how will I continue the after DepthMap, so I do not lose any progress.

Just download the cache folder and the mg file and load the mg file from the meshroom gui.

You might need to open the mg file in a text editor and modify the "path": "..." to match your desktop computer path for the images

Found only 1/10 nearest cameras for view id:

The image dataset might not be optimal. What are you trying to reconstruct?

@vinayadatta
Copy link

Solved!

I encountered the same issue as you while running with 451 images on Colab. The status is saved after each successfully computed node and not in between. This means that for example if the computation was halted at DepthMap (21/23), the whole DepthMap node is reset and restarts from the last stable/success node.

So, in order to utilize the free Google Colab and save your progress, the best thing to do is,

  1. Open your meshroom GUI, import your images, and set up all your required parameters. Add a 'Publish' node if missing, and save your "name.md" file.
  2. Next open the name.md file in a text editor and replace your file paths with the ones in your google drive. [Assuming you have your images copied to google drive]. Ex: "C:/users/admin/mesh_files_input" to "/content/drive/mydrive/mesh_files_input".
  3. Now upload the updated name.md file to google drive and run that file for meshroom in Colab.

This will start saving your cache folder alongside your input files in google drive which will not be erased when you restart your runtime.. I am yet to figure out how this actually works but I guess the cache folder is also presumably changed while you change your file paths in iterative mode.. Use Notepad++ or similar..

@ironman26820
Copy link
Author

ironman26820 commented Dec 1, 2020

Solved!

I encountered the same issue as you while running with 451 images on Colab. The status is saved after each successfully computed node and not in between. This means that for example if the computation was halted at DepthMap (21/23), the whole DepthMap node is reset and restarts from the last stable/success node.

So, in order to utilize the free Google Colab and save your progress, the best thing to do is,

  1. Open your meshroom GUI, import your images, and set up all your required parameters. Add a 'Publish' node if missing, and save your "name.md" file.
  2. Next open the name.md file in a text editor and replace your file paths with the ones in your google drive. [Assuming you have your images copied to google drive]. Ex: "C:/users/admin/mesh_files_input" to "/content/drive/mydrive/mesh_files_input".
  3. Now upload the updated name.md file to google drive and run that file for meshroom in Colab.

This will start saving your cache folder alongside your input files in google drive which will not be erased when you restart your runtime.. I am yet to figure out how this actually works but I guess the cache folder is also presumably changed while you change your file paths in iterative mode.. Use Notepad++ or similar..

So this is what I did, I used to create a template pipeline of the settings and then re-use it for different projects.
I edited some parameters I found in the documentation,
--pipeline ./Input/sift_akaze.mg (The template file)
--input ./Input/Images (The folder of the dataset)
--save ./Input/Render.mg (The output pipeline, just for the image dataset)

Ran that in Meshroom first, just to create a pipeline for my dataset. Then use the output (Render.mg) and run that using these parameters
--pipeline /content/Input/Render.mg (The output pipeline just for image dataset, also this would save the progress, so I backed up that too)
--input /content/Input/Images (I think this was not needed, but I still did that)
--cache /content/MeshroomCache (Just in case if I am near the time limit, so I can zip the Cache and save it to Drive)
--output /content/Render (The real reason we use Meshroom for)
Optional --forceStatus (If you are re-running the same thing)
I always wanted to directly save it to Google Drive, but the problem is, my drive is named as My Drive. That space in between gave me a lot of issue in doing it, so I used this way around. I see it shows as mydrive for you, how did you do it?!

This will be the plan from now on if I rename my Gdrive as mydrive:

Put meshroom cache and Render.mg directly to Google Drive, and --forceStatus the way in, so could you explain how you renamed My Drive as mydrive?

@ironman26820 ironman26820 reopened this Dec 1, 2020
@natowi
Copy link
Member

natowi commented Dec 2, 2020

@ironman26820 put double quotes around the path before using it or create a symlink !ln -s "/content/drive/My Drive" /gdrive

@vinayadatta
Copy link

Put meshroom cache and Render.mg directly to Google Drive, and --forceStatus the way in, so could you explain how you renamed My Drive as mydrive?

@ironman26820 I did not rename it actually.. But what you can do is you can either put double quotes as @natowi said or if not you can simply add a '\' before each space like this

/content/drive/My\ Drive/Folder\ with\ space/File\ with\ space.mg

I used the backslash method in my project.. Since '\' is an escape character, any character next to it will be considered literally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants