panorama test without theory

[zebrajack]

since narrow perspective cannot ensure the triplet. we choose 3 panoramas. but I know the theory cannot support the vps in panoramas. but it genrated some results.

Though the result looks like it is not that bad. so does it mean some conincident?

All the dataset can be download below.
https://drive.google.com/open?id=14eerqQs2g0LudnXJsMCDBsMoo0g4sQOl

[ySalaun]

Hi,

Though I don't really know what you expect from me, I can tell you that the vanishing points are wrongly estimated on your pictures (vertical one is correct but horizontal is a mixed of multiple horizontal directions)
However, vanishing points are only used to limit the number of line pairs for calibration estimation. It means that the soft should work (even if slower) with every segment associated to the same VP (even if it's false).

Best,

Yohann

[zebrajack]

yeah. just want to ensure that the VP are wrong and the result is not right.

[haopo2005]

Hi, @ySalaun
We'd like you to help us verify the correctness of global pose from the RECONSTRUCTION application based on our image set (both panorama and perspective image we've tried)
There are not enough feature points in our images and we'd like to use the line features to compute camera pose. And your paper said , it can handle the situation of less feature points.
And the CALIBRATION application has executed as we expected. As you can see, the lines are the final kept feature.
jst@jst-pc:~/share/project/LineSfM-master/build$ ./CALIBRATION -d ../test_image/ -i ../images2.txt -v 1
4
1 /home/jst/share/project/LineSfM-master/test_image/1.PNG
2 /home/jst/share/project/LineSfM-master/test_image/2.PNG
3 /home/jst/share/project/LineSfM-master/test_image/3.PNG
4 /home/jst/share/project/LineSfM-master/test_image/4.PNG
reading intrinsic parameters
reading global poses
computing relative poses
LOAD DETECTIONS AND MATCHES
picture 0: 1

• read points
• read segments
• read vanishing points
  picture 1: 2
• read points
• read segments
• read vanishing points
  picture 2: 3
• read points
• read segments
• read vanishing points
  picture 3: 4
• read points
• read segments
• read vanishing points
  pictures 1 and 2
• read point matches
• read line matches
  pictures 2 and 3
• read point matches
• read line matches
  pictures 3 and 4
• read point matches
• read line matches
  COMPUTE POSES
  pictures 1 and 2
  === BETTER MODEL FOUND ===
  NFA: -33.3503
  #inliers: 82
• points: 15/20
• lines: 19/30
• parallel pairs: 67/143
  === BETTER MODEL FOUND ===
  NFA: -43.6666
  #inliers: 18
• points: 18/20
• lines: 28/30
• parallel pairs: 0/143
  === BETTER MODEL FOUND ===
  NFA: -46.1029
  #inliers: 26
• points: 20/20
• lines: 28/30
• parallel pairs: 6/143
  === BETTER MODEL FOUND ===
  NFA: -53.4207
  #inliers: 49
• points: 20/20
• lines: 23/30
• parallel pairs: 29/143
  === BETTER MODEL FOUND ===
  NFA: -58.7462
  #inliers: 46
• points: 19/20
• lines: 20/30
• parallel pairs: 27/143
  final kept hypothesis: LINES
  WARNING: Logging before InitGoogleLogging() is written to STDERR
  E0919 10:04:00.823555 4555 trust_region_minimizer.cc:91] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5

Solver Summary (v 1.13.0-eigen-(3.3.4)-lapack-suitesparse-(4.4.6)-cxsparse-(3.1.4)-eigensparse-openmp-no_tbb)

                                 Original                  Reduced

Parameter blocks 2 2
Parameters 6 6
Residual blocks 46 46
Residual 46 46

Minimizer TRUST_REGION

Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT

                                    Given                     Used

Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver threads 1 1
Linear solver ordering AUTOMATIC 2

Cost:
Initial 8.625961e-03

Minimizer iterations 5
Successful steps 1
Unsuccessful steps 4

Time (in seconds):
Preprocessor 0.011321

Residual evaluation 0.000000
Jacobian evaluation 0.007050
Linear solver 0.009047
Minimizer 0.016271

Postprocessor 0.000009
Total 0.027602

Termination: FAILURE (Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5)

pictures 2 and 3
=== BETTER MODEL FOUND ===
NFA: -69.6865
#inliers: 128

• points: 32/35
• lines: 30/42
• parallel pairs: 96/214
  === BETTER MODEL FOUND ===
  NFA: -81.3461
  #inliers: 26
• points: 26/35
• lines: 35/42
• parallel pairs: 0/214
  === BETTER MODEL FOUND ===
  NFA: -82.0272
  #inliers: 37
• points: 22/35
• lines: 27/42
• parallel pairs: 15/214
  === BETTER MODEL FOUND ===
  NFA: -90.9075
  #inliers: 30
• points: 30/35
• lines: 34/42
• parallel pairs: 0/214
  === BETTER MODEL FOUND ===
  NFA: -102.702
  #inliers: 99
• points: 29/35
• lines: 26/42
• parallel pairs: 70/214
  === BETTER MODEL FOUND ===
  NFA: -105.49
  #inliers: 31
• points: 31/35
• lines: 29/42
• parallel pairs: 0/214
  === BETTER MODEL FOUND ===
  NFA: -112.041
  #inliers: 34
• points: 30/35
• lines: 27/42
• parallel pairs: 4/214
  === BETTER MODEL FOUND ===
  NFA: -126.755
  #inliers: 88
• points: 31/35
• lines: 26/42
• parallel pairs: 57/214
  final kept hypothesis: LINES
  E0919 10:04:01.145869 4555 trust_region_minimizer.cc:91] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5

Solver Summary (v 1.13.0-eigen-(3.3.4)-lapack-suitesparse-(4.4.6)-cxsparse-(3.1.4)-eigensparse-openmp-no_tbb)

                                 Original                  Reduced

Parameter blocks 2 2
Parameters 6 6
Residual blocks 88 88
Residual 88 88

Minimizer TRUST_REGION

Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT

                                    Given                     Used

Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver threads 1 1
Linear solver ordering AUTOMATIC 2

Cost:
Initial 5.571573e-03

Minimizer iterations 5
Successful steps 1
Unsuccessful steps 4

Time (in seconds):
Preprocessor 0.000013

Residual evaluation 0.000000
Jacobian evaluation 0.000052
Linear solver 0.000032
Minimizer 0.000112

Postprocessor 0.000002
Total 0.000127

Termination: FAILURE (Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5)

pictures 3 and 4
=== BETTER MODEL FOUND ===
NFA: 8.59071
#inliers: 18

• points: 15/21
• lines: 25/30
• parallel pairs: 3/116
  === BETTER MODEL FOUND ===
  NFA: -22.7967
  #inliers: 22
• points: 20/21
• lines: 8/30
• parallel pairs: 2/116
  === BETTER MODEL FOUND ===
  NFA: -34.9052
  #inliers: 20
• points: 20/21
• lines: 22/30
• parallel pairs: 0/116
  === BETTER MODEL FOUND ===
  NFA: -41.2829
  #inliers: 21
• points: 21/21
• lines: 21/30
• parallel pairs: 0/116
  === BETTER MODEL FOUND ===
  NFA: -47.1451
  #inliers: 21
• points: 21/21
• lines: 23/30
• parallel pairs: 0/116
  === BETTER MODEL FOUND ===
  NFA: -47.5883
  #inliers: 21
• points: 21/21
• lines: 22/30
• parallel pairs: 0/116
  === BETTER MODEL FOUND ===
  NFA: -48.1566
  #inliers: 21
• points: 21/21
• lines: 23/30
• parallel pairs: 0/116
  === BETTER MODEL FOUND ===
  NFA: -48.6653
  #inliers: 21
• points: 21/21
• lines: 23/30
• parallel pairs: 0/116
  final kept hypothesis: LINES
  E0919 10:04:01.410118 4555 trust_region_minimizer.cc:91] Terminating: Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5

Solver Summary (v 1.13.0-eigen-(3.3.4)-lapack-suitesparse-(4.4.6)-cxsparse-(3.1.4)-eigensparse-openmp-no_tbb)

                                 Original                  Reduced

Parameter blocks 2 2
Parameters 6 6
Residual blocks 21 21
Residual 21 21

Minimizer TRUST_REGION

Dense linear algebra library EIGEN
Trust region strategy LEVENBERG_MARQUARDT

                                    Given                     Used

Linear solver DENSE_QR DENSE_QR
Threads 1 1
Linear solver threads 1 1
Linear solver ordering AUTOMATIC 2

Cost:
Initial 1.272141e-03

Minimizer iterations 5
Successful steps 1
Unsuccessful steps 4

Time (in seconds):
Preprocessor 0.000010

Residual evaluation 0.000000
Jacobian evaluation 0.000031
Linear solver 0.000017
Minimizer 0.000077

Postprocessor 0.000002
Total 0.000089

Termination: FAILURE (Number of consecutive invalid steps more than Solver::Options::max_num_consecutive_invalid_steps: 5)

PROCESSED IN 0.834883
images 0 and 1
Error in rotation : 120
Error in translation : nan
images 1 and 2
Error in rotation : 120
Error in translation : nan
images 2 and 3
Error in rotation : 120
Error in translation : nan
AVERAGE RESULTS:
Rotation: 120
Translation: nan
jst@jst-pc:~/share/project/LineSfM-master/build$

As followed pic, the global poses of your TEST_IMAGE (4 outdoor images with rich features) are right

And this ours is not correct,

The global poses are extracted before bundle adjustment in the RECONSTRUCTION application. (we'd like the first image's coord to be 0,0,0.)

[haopo2005]

Here is a small batch of our images.

[haopo2005]

Besides we reuse the calibration matrix K from your TEST_IMAGE to compute everything, dose it matter?

[ySalaun]

Hi,

This seems to be a tough scene so you should check:

• point matches: even if 2 only matches are required, they need to be correct
• line matches: the matching does not need to be perfect but segments that are matched together need to belong to the same vanishing point
• vanishing points: you should check in the resulting images if the clusters are correct enough

About the K matrix, the result won't be correct if you use the wrong matrix. However, I'm not sure it will solve your problem in this case.
To compute your own K matrix you should follow the formula below:
K = | focal_length (mm) * picture_height (pixel nb) / sensor_height(mm), 0, 0.5picture_height (pixel nb) |
| 0, focal_length (mm) * picture_width (pixel nb) / sensor_width(mm), 0.5picture_width (pixel nb) |
| 0,0,1 |
(Note that the parameters can be found on your exif (for focal length) and internet (for sensor information))

Best,

Yohann