Remove EXIV2_EXT variable references #1289

kmilos · 2020-09-09T09:28:08Z

Addresses #1273

Tested ok only on mingw64 so far

clanmills · 2020-09-09T11:03:21Z

Thanks for looking. There is still work to be done. I tried to debug some python test code in Visual Studio Code last week and had trouble loading tests/system_tests.py because EXIV2_EXT wasn't defined. And we have to remove mention of it from the documentation. And there's always something we didn't know and didn't expect.

clanmills · 2020-09-09T11:29:29Z

Ah. This is a PR. I thought it was a comment on #1273. I'll ask @LeoHsiao1 to review as he has been involved with the python test code. I'll take care of update README.md (and possibly README-CONAN.md) when we believe the PR is ready to be merged.

kmilos · 2020-09-09T12:02:52Z

I'll take care of update README.md (and possibly README-CONAN.md) when we believe the PR is ready to be merged.

Already in this PR, so this should be it AFAIK ;)

As for the CI failures, that's unfortunately beyond me...

clanmills · 2020-09-09T12:36:14Z

Ah, yes. I see you have worked on the README.md and README-CONAN.md Thanks. There is a reference to EXIV2_EXT in tests/suite.conf and I believe that's why Visual Studio Code refuses to launch/debug the test suite. Just removing that from tests/suite.conf will not solve that. We'll need to investigate the use of the variable binary_extension.

1216 rmills@rmillsmbp:~/gnu/github/exiv2/0.27-maintenance $ find tests -name '*' -type f -exec grep -iH EXIV2_EXT {} \;
tests/suite.conf:binary_extension: EXIV2_EXT
1217 rmills@rmillsmbp:~/gnu/github/exiv2/0.27-maintenance $

I need to investigate the CI (which I didn't implement). The builders frequently fail to download dependencies. And CodeCov often fails good builds for 100% incomprehensible reasons. MinGW/msys2 and Cygwin/64 don't get built by the CI.

I have a macMini at home and use that to build/test the 8 supported platforms (macOS, Ubuntu, cygwin64, MinGW/msys2, msvc2019, solaris, NetBSD and FreeBSD). You know how it is. There's 100,000 lines of code to be maintained. I'm writing a book. And unbelievably I only have one pair hands.

kmilos · 2020-09-09T12:51:24Z

I have changed suit.conf as well, in the hope that removing will be interpreted as an empty string. It works on mingw64 (and Linux CI). There is no more mention of EXIV2_EXT on my branch, only binary_extension

$ grep EXIV2_EXT -w -r *

$ grep binary_extension -w -r *
tests/suite.conf:binary_extension:
tests/suite.conf:exiv2: ${ENV:exiv2_path}/exiv2${ENV:binary_extension}
tests/suite.conf:exiv2json: ${ENV:exiv2_path}/exiv2json${ENV:binary_extension}
tests/suite.conf:tiff_test: ${ENV:exiv2_path}/tiff-test${ENV:binary_extension}
tests/suite.conf:largeiptc_test: ${ENV:exiv2_path}/largeiptc-test${ENV:binary_extension}
tests/suite.conf:easyaccess_test: ${ENV:exiv2_path}/easyaccess-test${ENV:binary_extension}
tests/suite.conf:taglist: ${ENV:exiv2_path}/taglist${ENV:binary_extension}
tests/suite.conf:exiv2exe: exiv2${ENV:binary_extension}
tests/system_tests.py:        config['ENV']['binary_extension'] = config['ENV']['binary_extension'] or '.exe'

Maybe binary_extension: '' in suite.conf (explicit empty string rather than nothing) is enough to fool Visual Studio Code?

clanmills · 2020-09-09T13:21:02Z

I think it's causing problems because it is testing that some executables exist.

1220 rmills@rmillsmbp:~/gnu/github/exiv2/0.27-maintenance $ find tests -type f -exec grep -H binary_extension {} \; 
tests/suite.conf:binary_extension: EXIV2_EXT
tests/suite.conf:exiv2: ${ENV:exiv2_path}/exiv2${ENV:binary_extension}
tests/suite.conf:exiv2json: ${ENV:exiv2_path}/exiv2json${ENV:binary_extension}
tests/suite.conf:tiff_test: ${ENV:exiv2_path}/tiff-test${ENV:binary_extension}
tests/suite.conf:largeiptc_test: ${ENV:exiv2_path}/largeiptc-test${ENV:binary_extension}
tests/suite.conf:easyaccess_test: ${ENV:exiv2_path}/easyaccess-test${ENV:binary_extension}
tests/suite.conf:taglist: ${ENV:exiv2_path}/taglist${ENV:binary_extension}
tests/suite.conf:exiv2exe: exiv2${ENV:binary_extension}
tests/system_tests.py:        config['ENV']['binary_extension'] = config['ENV']['binary_extension'] or '.exe'
1221 rmills@rmillsmbp:~/gnu/github/exiv2/0.27-maintenance $

I'm working on something else at the moment. Let me finish what I'm doing, sync your code and see how this is works with Visual Studio Code. It's possible that everything's great and I'm talking about nothing.

You've caught my by surprise by submitting a PR. I had asked @LeoHsiao1 to work on this and I thought you would test his PR on MinGW/msys2. @LeoHsiao1 has gone to the movies this evening. So, we working backwards. However this might work out perfectly if I conclude your PR is good, we wait for @LeoHsiao1 to review.

kmilos · 2020-09-09T13:24:04Z

You've caught my by surprise by submitting a PR. I had asked @LeoHsiao1 to work on this and I thought you would test his PR on MinGW/msys2. @LeoHsiao1 has gone to the movies this evening. So, we working backwards.

Sorry, you assigned #1273 to me, and I understood I was thus asked to provide a PR... Thankfully the work is small so any duplication won't be significant...

clanmills · 2020-09-09T13:45:24Z

There's been a little misunderstanding. I think this is going to work out well. I'll update you later.

Thanks for getting involved. Most people are very demanding while contributing nothing. You're the opposite. You're contributing something and demanding nothing. Thanks for working on this.

clanmills · 2020-09-09T16:19:17Z

@kmilos Apologies. I have run out of time today and will look at your PR tomorrow.

This afternoon, I had to fix my macMini buildserver to build on all supported platforms. This was working perfectly at the end of June when I shipped 0.27.3. I've been working on my book in July and August. Now that I'm back working on the 0.27-maintenance branch, mysterious things have disturbed the environment on the macMini. I've fixed a cmake issue with Solaris.

I totally rely on the macMini (and ignore the GitHub/CI). So.... tomorrow:

I'll run your PR on all supported platforms.
Work with Visual Studio Code and our python test suite.
Update this issue with my progress.

Apologies for the confusion about the assignment. I should have made you a "reviewer" and only assigned @LeoHsiao1 to the issue. So, you've "swapped seats". You remain assigned to this issue and @LeoHsiao1 is the reviewer.

clanmills · 2020-09-10T07:32:37Z

Mostly good news. I confirm that your code works well on MinGW/msys2, freebsd, netbsd, cygwin64, ubuntu and macOS. There's a build issue on Solaris which is unrelated to your code. 1290.

Sadly msvc/Windows dies a horrible death which I'll investigate that today. I haven't investigated debugging tests/runner.py with Visual Studio Code.

There's something causing a python traceback during $ make python_tests. I believe it's coming from iotest in which we launch a python web-server. I've seen this on MinGW/msys2, Cygwin64 and UNIX. This is probably related to merging 1257 yesterday. I think it's unlikely to be caused by your code. I will investigate.

I'm unhappy about the corpse of binary_extension in the code. I think it's only in tests/suite.conf

kmilos · 2020-09-10T07:59:17Z

I'm unhappy about the corpse of binary_extension in the code. I think it's only in tests/suite.conf

It does indeed stick out. However, it is set to '.exe' for Windows in system_tests.py, and it really is used to test the presence of required binaries.

Maybe it can be fully internalized in system_test.py somehow in a second pass by @LeoHsiao1

clanmills · 2020-09-10T08:16:42Z

It shouldn't be set to .exe in Windows. We remove corpses from the code. If something does nothing, it should not be there. Please remove it and update the PR.

@LeoHsiao1 is to review the code and it's better that he contributes nothing to code that he reviews.

kmilos · 2020-09-10T08:19:45Z

I tried removing everything of course, the test for presence of binaries fails without '.exe' added on Windows. My suggestion is to merge the patch that is known to be working, and take another separate step.

clanmills · 2020-09-10T08:48:05Z

I will investigate the windows/.exe puzzle.

kmilos · 2020-09-10T09:30:57Z

It looks like you have to account for the extension on Windows, and there are suggestions to use the PATHEXT variable rather than assume and hard code anything. I also like the hint that one could skip searching for the file anyway, and catch the exception from Popen (which doesn't require the extension): https://stackoverflow.com/a/379535

LeoHsiao1

This removes the variable EXIV2_EXT from the surface, but do we still need the variable binary_extension?

Many suffixes are retained in the file exiv2/tests/ Suite.conf:

[paths]
exiv2: ${ENV:exiv2_path}/exiv2${ENV:binary_extension}
exiv2json: ${ENV:exiv2_path}/exiv2json${ENV:binary_extension}

I had planned to improve both pieces of code to completely remove binary_extension:

exiv2/tests/system_tests.py

Line 166 in bb771e5

config['ENV']['binary_extension'] = config['ENV']['binary_extension'] or '.exe'

exiv2/tests/system_tests.py

Line 180 in bb771e5

if not os.path.exists(abs_path):

kmilos · 2020-09-10T10:09:02Z

The second commit removes any extension use and search for binaries, only relying on Popen to raise an error. Works or on mingw64.

LeoHsiao1 · 2020-09-10T10:10:13Z

Oh, you did it while I was typing!

clanmills · 2020-09-10T10:18:51Z

Yesterday, I was complaining to Milos that I only have one pair of hands. Wrong! At least 3 pairs here, and two working on C++14 stuff on master.

So great to have you both (@kmilos and @LeoHsiao1 ) working with me. Thank You.

clanmills · 2020-09-10T10:57:13Z

Unbelievable, Windows/MSVC passes. All I did was $ git pull --rebase. Nothing else.

C:\Users\rmills\gnu\github\exiv2\kmilos\build>git status
On branch 0.27-rm-exiv2_ext
Your branch is up to date with 'origin/0.27-rm-exiv2_ext'.

nothing to commit, working tree clean

C:\Users\rmills\gnu\github\exiv2\kmilos\build>cat ..\.git\config
[core]
        repositoryformatversion = 0
        filemode = false
        bare = false
        logallrefupdates = true
        symlinks = false
        ignorecase = true
[remote "origin"]
        url = https://github.com/kmilos/exiv2
        fetch = +refs/heads/0.27-rm-exiv2_ext:refs/remotes/origin/0.27-rm-exiv2_ext
[branch "0.27-rm-exiv2_ext"]
        remote = origin
        merge = refs/heads/0.27-rm-exiv2_ext

C:\Users\rmills\gnu\github\exiv2\kmilos\build>

C:\Users\rmills\gnu\github\exiv2\kmilos\build>cmake --build . --config Release --target tests
...
 ---- Running python_tests ----

  bash -c  . functions.source ; cd ../tests ; if [ ! -z $VERBOSE ]; then verbose=--verbose ;fi ; python3 runner.py $verb
  ose
  ..................................................s...............s............ss............s........................
  ............s...................................................s..........
  ----------------------------------------------------------------------
  Ran 193 tests in 40.765s

  OK (skipped=7)

  ---- Running version_test ----

  exiv2 0.27.3
  exiv2=0.27.3
  platform=windows
  compiler=MSVC

I'm going to cut the grass in the garden and while I'm doing that, the macMini can build/test the supported platforms. After lunch I'll investigate Visual Studio Code on Windows. I don't expect an issue (mad fool).

This smells good!

kmilos · 2020-09-10T10:59:38Z

Bon apetit!

clanmills · 2020-09-10T12:12:08Z

@kmilos Can you "sync" #1290 into your code-base please? It's solaris build fix. I can't remember what git/github calls "sync" - rebasing or something. Luis has done some good work on the CI on 'master' this morning.

I hope we'll merge your stuff into 0.27-maintenance today. Tomorrow, I'll get the CI working better on 0.27-maintenance.

clanmills

Thanks. Visual Studio Code is now debug tests/runner.py on Windows. I see the weirdo pythonic traceback from iotest is visible in Windows. I'll investigate that now. Then, I think we're done on this. I'll approve when I believe we're ready for the merge.

Great Job. Take the rest of the afternoon off!

clanmills · 2020-09-10T15:32:25Z

I'm happy with this. I don't believe the pythonic traceback from tests/bash_tests/testcases.py/iotest() has anything to do with this PR.

@LeoHsiao1 Can you review PR, please? If you're happy, approve and merge.

@kmilos Thank You for working on this. Come back and see us again soon. When this PR has been merged, can you sync/test branch 0.27-maintenance and then close #1273 and #1269.

LeoHsiao1

I tried the PR on Windows and it worked.

kmilos · 2020-09-11T09:24:15Z

Thanks, pleasure to make a small contribution to your efforts.

clanmills · 2020-09-11T09:45:06Z

You're welcome. Please close #1273 and #1269 and we're done.

Remove EXIV2_EXT variable references

8a68658

clanmills requested a review from LeoHsiao1 September 9, 2020 11:27

clanmills assigned kmilos Sep 9, 2020

clanmills added this to the v0.27.4 milestone Sep 9, 2020

Remove remaining vestiges of binary_extension

92fbfc9

LeoHsiao1 reviewed Sep 10, 2020

View reviewed changes

fix_solaris_stack_protection_0.27

b60325a

clanmills reviewed Sep 10, 2020

View reviewed changes

LeoHsiao1 approved these changes Sep 11, 2020

View reviewed changes

LeoHsiao1 merged commit d98d30c into Exiv2:0.27-maintenance Sep 11, 2020

kmilos deleted the 0.27-rm-exiv2_ext branch September 11, 2020 09:28

clanmills mentioned this pull request Sep 16, 2020

Remove EXIV2_EXT from code-base #1273

Closed

clanmills mentioned this pull request Jan 11, 2021

Exiv2 RoadMap #1018

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove EXIV2_EXT variable references #1289

Remove EXIV2_EXT variable references #1289

kmilos commented Sep 9, 2020

clanmills commented Sep 9, 2020

clanmills commented Sep 9, 2020

kmilos commented Sep 9, 2020 •

edited

Loading

clanmills commented Sep 9, 2020

kmilos commented Sep 9, 2020 •

edited

Loading

clanmills commented Sep 9, 2020

kmilos commented Sep 9, 2020 •

edited

Loading

clanmills commented Sep 9, 2020

clanmills commented Sep 9, 2020

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020 •

edited

Loading

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020 •

edited

Loading

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020

LeoHsiao1 left a comment •

edited

Loading

kmilos commented Sep 10, 2020

LeoHsiao1 commented Sep 10, 2020

clanmills commented Sep 10, 2020

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020

clanmills commented Sep 10, 2020

clanmills left a comment •

edited

Loading

clanmills commented Sep 10, 2020 •

edited

Loading

LeoHsiao1 left a comment

kmilos commented Sep 11, 2020

clanmills commented Sep 11, 2020

Remove EXIV2_EXT variable references #1289

Remove EXIV2_EXT variable references #1289

Conversation

kmilos commented Sep 9, 2020

clanmills commented Sep 9, 2020

clanmills commented Sep 9, 2020

kmilos commented Sep 9, 2020 • edited Loading

clanmills commented Sep 9, 2020

kmilos commented Sep 9, 2020 • edited Loading

clanmills commented Sep 9, 2020

kmilos commented Sep 9, 2020 • edited Loading

clanmills commented Sep 9, 2020

clanmills commented Sep 9, 2020

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020 • edited Loading

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020 • edited Loading

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020

LeoHsiao1 left a comment • edited Loading

Choose a reason for hiding this comment

kmilos commented Sep 10, 2020

LeoHsiao1 commented Sep 10, 2020

clanmills commented Sep 10, 2020

clanmills commented Sep 10, 2020

kmilos commented Sep 10, 2020

clanmills commented Sep 10, 2020

clanmills left a comment • edited Loading

Choose a reason for hiding this comment

clanmills commented Sep 10, 2020 • edited Loading

LeoHsiao1 left a comment

Choose a reason for hiding this comment

kmilos commented Sep 11, 2020

clanmills commented Sep 11, 2020

kmilos commented Sep 9, 2020 •

edited

Loading

kmilos commented Sep 9, 2020 •

edited

Loading

kmilos commented Sep 9, 2020 •

edited

Loading

kmilos commented Sep 10, 2020 •

edited

Loading

kmilos commented Sep 10, 2020 •

edited

Loading

LeoHsiao1 left a comment •

edited

Loading

clanmills left a comment •

edited

Loading

clanmills commented Sep 10, 2020 •

edited

Loading