Describe the bug
CPU memory utilization grows with training and finally cause OOM when num_workers of Dataloader greater than 0.
Especially when more datasets are used, this mem growth phenomenon becomes more obvious.
Environment
torch 2.3.0+cu121
Other information
No response
Describe the bug
CPU memory utilization grows with training and finally cause OOM when num_workers of Dataloader greater than 0.
Especially when more datasets are used, this mem growth phenomenon becomes more obvious.
Environment
torch 2.3.0+cu121
Other information
No response