All of the tests failed on my computer.

[karajan1001]

I download the repo, create a venv, and followed the instruction

pip install -r requirements.txt
dvc pull
pytest

But I got

dulwich.errors.HangupException: ssh: Could not resolve hostname https: nodename nor servname provided, or not known

For every single test.

self = <dulwich.protocol.Protocol object at 0x1056a66b0>

    def read_pkt_line(self):
        """Reads a pkt-line from the remote git process.

        This method may read from the readahead buffer; see unread_pkt_line.

        Returns: The next string from the stream, without the length prefix, or
            None for a flush-pkt ('0000').
        """
        if self._readahead is None:
            read = self.read
        else:
            read = self._readahead.read
            self._readahead = None

        try:
            sizestr = read(4)
            if not sizestr:
>               raise HangupException()
E               dulwich.errors.HangupException: The remote server unexpectedly closed the connection.

/opt/homebrew/lib/python3.10/site-packages/dulwich/protocol.py:232: HangupException

During handling of the above exception, another exception occurred:

request = <FixtureRequest for <Function test_sharing[None]>>

    def fill(request):
        item = request._pyfuncitem
        fixturenames = getattr(item, "fixturenames", None)
        if fixturenames is None:
            fixturenames = request.fixturenames

        if hasattr(item, 'callspec'):
            for param, val in sorted_by_dependency(item.callspec.params, fixturenames):
                if val is not None and is_lazy_fixture(val):
                    item.callspec.params[param] = request.getfixturevalue(val.name)
                elif param not in item.funcargs:
>                   item.funcargs[param] = request.getfixturevalue(param)

/opt/homebrew/lib/python3.10/site-packages/pytest_lazyfixture.py:37:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/benchmarks/conftest.py:40: in dvc_git_repo
    clone(url, tmp_path)
/opt/homebrew/lib/python3.10/site-packages/dulwich/porcelain.py:538: in clone
    return client.clone(
/opt/homebrew/lib/python3.10/site-packages/dulwich/client.py:760: in clone
    result = self.fetch(path, target, progress=progress, depth=depth)
/opt/homebrew/lib/python3.10/site-packages/dulwich/client.py:837: in fetch
    result = self.fetch_pack(
self = <dulwich.client.SSHGitClient object at 0x10565c970>, path = '//github.com/iterative/dvc'
determine_wants = <bound method BaseObjectStore.determine_wants_all of <DiskObjectStore('/private/var/folders/pv/474xpngj6w71zbfcsxhdxwc40000gn/T/pytest-of-gao/pytest-99/dvc-git-repo0/.git/objects')>>
graph_walker = <dulwich.object_store.ObjectStoreGraphWalker object at 0x1056a6080>, pack_data = <bound method SpooledTemporaryFile.write of <tempfile.SpooledTemporaryFile object at 0x1056a6170>>
progress = <built-in method write of _io.BufferedWriter object at 0x102a30670>, depth = None

    def fetch_pack(
        self,
        path,
        determine_wants,
        graph_walker,
        pack_data,
        progress=None,
        depth=None,
    ):
        """Retrieve a pack from a git smart server.

        Args:
          path: Remote path to fetch from
          determine_wants: Function determine what refs
            to fetch. Receives dictionary of name->sha, should return
            list of shas to fetch.
          graph_walker: Object with next() and ack().
          pack_data: Callback called for each bit of data in the pack
          progress: Callback for progress reports (strings)
          depth: Shallow fetch depth

        Returns:
          FetchPackResult object

        """
        proto, can_read, stderr = self._connect(b"upload-pack", path)
        with proto:
            try:
                refs, server_capabilities = read_pkt_refs(proto.read_pkt_seq())
            except HangupException:
>               raise _remote_error_from_stderr(stderr)
E               dulwich.errors.HangupException: ssh: Could not resolve hostname https: nodename nor servname provided, or not known

/opt/homebrew/lib/python3.10/site-packages/dulwich/client.py:1151: HangupException

besides if I run pytest --dvc-git-repo 'git@github.com:iterative/dvc.git' --dvc-revs af70c33 tests/benchmarks/cli/commands/test_exp_show.py I will get

tests/benchmarks/conftest.py:54: in dvc_bin
    venv.run(f"pip install git+file://{dvc_git_repo}@{dvc_rev}")
/opt/homebrew/lib/python3.10/site-packages/pytest_virtualenv.py:151: in run
    return super(VirtualEnv, self).run(args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pytest_virtualenv.VirtualEnv object at 0x108d9fa90>, cmd = 'pip install git+file:///private/var/folders/pv/474xpngj6w71zbfcsxhdxwc40000gn/T/pytest-of-gao/pytest-101/dvc-git-repo0@af70c33'
capture = False, check_rc = True, cd = Path('/private/var/folders/pv/474xpngj6w71zbfcsxhdxwc40000gn/T/pytest-of-gao/pytest-101/dvc-venv-af70c330'), shell = True
kwargs = {'env': {'ANTIBODY': '/Users/gao/Library/Caches/antibody', 'AUTOJUMP_ERROR_PATH': '/Users/gao/Library/autojump/errors.log', 'AUTOJUMP_SOURCED': '1', 'BAT_PAGER': 'less -xRF-j12', ...}}
p = <Popen: returncode: 1 args: 'pip install git+file:///private/var/folders/pv/...>, out = None, _ = None
err = CalledProcessError(1, 'pip install git+file:///private/var/folders/pv/474xpngj6w71zbfcsxhdxwc40000gn/T/pytest-of-gao/pytest-101/dvc-git-repo0@af70c33')

    def run(self, cmd, capture=False, check_rc=True, cd=None, shell=False, **kwargs):
        """
        Run a command relative to a given directory, defaulting to the workspace root

        Parameters
        ----------
        cmd : `str` or `list`
            Command string or list. Commands given as a string will be run in a subshell.
        capture : `bool`
            Capture and return output
        check_rc : `bool`
            Assert return code is zero
        cd : `str`
            Path to chdir to, defaults to workspace root
        """
        if isinstance(cmd, string_types):
            shell = True
        else:
            # Some of the command components might be path objects or numbers
            cmd = [str(i) for i in cmd]

        if not cd:
            cd = self.workspace

        with cmdline.chdir(cd):
            log.debug("run: {0}".format(cmd))
            if capture:
                p = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kwargs)
            else:
                p = subprocess.Popen(cmd, shell=shell, **kwargs)
            (out, _) = p.communicate()

            if out is not None and not isinstance(out, string_types):
                out = out.decode('utf-8')

            if self.debug and capture:
                log.debug("Stdout/stderr:")
                log.debug(out)

            if check_rc and p.returncode != 0:
                err = subprocess.CalledProcessError(p.returncode, cmd)
                err.output = out
                if capture and not self.debug:
                    log.error("Stdout/stderr:")
                    log.error(out)
>               raise err
E               subprocess.CalledProcessError: Command 'pip install git+file:///private/var/folders/pv/474xpngj6w71zbfcsxhdxwc40000gn/T/pytest-of-gao/pytest-101/dvc-git-repo0@af70c33' returned non-zero exit status 1.

/opt/homebrew/lib/python3.10/site-packages/pytest_shutil/workspace.py:132: CalledProcessError

[efiop]

@karajan1001 Could you try cloning a dvc repo locally manually and then supply path to it in --dvc-git-repo?

[karajan1001]

It says

pytest --dvc-git-repo '/Users/gao/Code/dvc/' --dvc-revs af70c33 tests/benchmarks/cli/commands/test_exp_show.py                                    -1-[ins][11:15:20]
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --dvc-git-repo --dvc-revs af70c33 tests/benchmarks/cli/commands/test_exp_show.py
  inifile: /Users/gao/Code/dvc/pyproject.toml
  rootdir: /Users/gao/Code/dvc

And if I use pytest --dvc-git-repo '/Users/gao/Code/dvc.git' --dvc-revs af70c33 tests/benchmarks/cli/commands/test_exp_show.py

It says

>               raise NotGitRepository(
                    "No git repository was found at %(path)s" % dict(path=root)
                )
E               dulwich.errors.NotGitRepository: No git repository was found at /Users/gao/Code/dvc.git

.venv/lib/python3.10/site-packages/dulwich/repo.py:1090: NotGitRepository

[dtrifiro]

@karajan1001 did you ever solve this? Since the error was that the--dvc-git-repo flag was not being recognised, I'm assuming this is due to the virtualenv not being set up correctly (possibly using a pytest version not in the virtualenv?)

[karajan1001]

@karajan1001 did you ever solve this? Since the error was that the--dvc-git-repo flag was not being recognised, I'm assuming this is due to the virtualenv not being set up correctly (possibly using a pytest version not in the virtualenv?)

Still the same, I'm in pytest 7.2.0, and which pytest points to the venv directory.

[karajan1001]

pytest --dvc-git-repo='/Users/gao/Code/dvc' --dvc-revs='main' tests works for me.

[efiop]

@karajan1001 Looks like you accidentally specified .git suffix on a dir instead of pointint to the dir or .git within that dir. (incorrect /Users/gao/Code/dvc.git vs correct /Users/gao/Code/dvc or correct /Users/gao/Code/dvc/.git).

So now everything works fine? I suppose the original issue was related to the proxy/vpn/firewall?

[dtrifiro]

Note the =.

pytest --dvc-git-repo path/to/dvc tests/ is broken, whereas pytest -s --dvc-git-repo=$w/dvc tests/ works.

Error is:

ImportError while loading conftest '/Users/dtrifiro/projects/dvc-bench/tests/conftest.py'.
_pytest.pathlib.ImportPathMismatchError: ('tests.conftest', '/Users/dtrifiro/projects/dvc/tests/conftest.py', PosixPath('/Users/dtrifiro/projects/dvc-bench/tests/conftest.py'))

[efiop]

@dtrifiro Good point. Please disregard my last comment, I got confused myself 🙂

[skshetry]

I think this is how pytest parses arguments.

pytest path/to/testdir path/other/
will determine the common ancestor as path and then check for configuration files as follows:

Custom pytest plugin commandline arguments may include a path, as in pytest --log-output ../../test.log args. Then args is mandatory, otherwise pytest uses the folder of test.log for rootdir determination (see also issue 1435). A dot . for referencing to the current working directory is also possible.

[omesser]

pytest --dvc-git-repo='/Users/gao/Code/dvc' --dvc-revs='main' tests works for me.

Closing 🙏