[Question] The logic of data sampler in data parallel.

[kxzxvbk]

Hi, thanks for your attention.

When reading the source code of transformers, I cannot understand the implementation of _get_train_sampler in trainer.py. Why the default data sampler is RandomSampler rather than DistributedSampler? How does the trainer handle the sampler for data parallel?

reference code: https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L975

[kxzxvbk]

And I notice that in very earlier versions. There are some if cases to check whether the world size > 1 and use a different sampler. Why this can be removed in the newer version?
This change is introduced in this commit: ebd94b0

[Rocketknight1]

Hi @kxzxvbk, this isn't really a bug, but we have a community support space for asking questions like these and getting answers from other users or Hugging Face engineers! https://huggingface.co/spaces/transformers-community/support