Improve idist to use Tcp and FileStore for init_method

[sdesrozis]

🚀 Feature

idist supports default initilialization env://. It could be useful to support tcp:// and file:// modes (to tackle distributed computations on windows for instance).

Main issue is how to configure these modes ?

[ahmedo42]

How about users customize this using environment variables? like this:

self._init_method = os.eviron["INIT_METHOD"]
dist.init_process_group(backend, init_method=self._init_method, **init_pg_kwargs)

in native.py

What are the drawbacks of this?

[vfdev-5]

@ahmedo42 this is interesting idea. Initially, I was thinking about an argument here:

ignite/ignite/distributed/comp_models/native.py

Lines 78 to 82 in 6d490c8

	init_pg_kwargs = {}
	if timeout is not None:
	init_pg_kwargs["timeout"] = timeout
	dist.init_process_group(backend, init_method="env://", **init_pg_kwargs)

but maybe we can think about two options like that...

[ahmedo42]

So the end result would be something like that?

       def _create_from_backend(self, backend: str, timeout: Optional[int] = None, 
                   init_method:Optional[str] = "env://", **kwargs: Any) -> None:

            if backend == dist.Backend.NCCL and not torch.cuda.is_available():
                raise RuntimeError("Nccl backend is required but no cuda capable devices")

            self.setup_env_vars()

            self._local_rank = int(os.environ["LOCAL_RANK"])
            # for debug purposes
            self._master_port = int(os.environ["MASTER_PORT"])  # type: Optional[int]
            self._master_addr = os.environ["MASTER_ADDR"]  # type: Optional[str]

            init_pg_kwargs = {}
            if timeout is not None:
                init_pg_kwargs["timeout"] = timeout

           if  not _valid_init_method(init_method):
                #raise error 
                
           dist.init_process_group(backend, init_method=init_method, **init_pg_kwargs)

[vfdev-5]

Yes, something like that. Maybe, init_method:Optional[str] = None and we check os.eviron["INIT_METHOD"] and if init_method is not None (priority to the argument) and set up dist.init_process_group.