Add allgather check for xpu #3605

Job	Run time
run-tests (latest, test_prod)	3m 31s
run-tests (latest, test_core)	4m 45s
run-tests (latest, test_cli)	5m 23s
run-tests (latest, test_big_modeling)	4m 31s
run-tests (latest, test_deepspeed)	57s
run-tests (latest, test_fsdp)	1m 10s
run-tests (latest, test_example_differences)	51s
run-tests (latest, test_checkpoint_step)	6m 40s
run-tests (latest, test_checkpoint_epoch)	6m 22s
run-tests (latest, test_rest)	10m 28s
run-tests (minimum, test_prod)	4m 10s
run-tests (minimum, test_core)	4m 58s
run-tests (minimum, test_cli)	5m 33s
run-tests (minimum, test_big_modeling)	52s
run-tests (minimum, test_deepspeed)	59s
run-tests (minimum, test_fsdp)	51s
run-tests (minimum, test_example_differences)	54s
run-tests (minimum, test_checkpoint_step)	6m 25s
run-tests (minimum, test_checkpoint_epoch)	5m 51s
run-tests (minimum, test_rest)	9m 49s
	1h 25m 0s

Provide feedback