vdpa/mlx5: fix large VM memory region registration
authorXueming Li <xuemingl@nvidia.com>
Thu, 23 Sep 2021 08:11:22 +0000 (16:11 +0800)
committerThomas Monjalon <thomas@monjalon.net>
Mon, 27 Sep 2021 15:24:22 +0000 (17:24 +0200)
commit6e914454d5083046497bbd764ff5ae1547e8c6b1
tree8d6ca6bdce17b985cce2e817c18c47a8825ea38b
parenteb5636e879a568a323cbed4c356162011e7312be
vdpa/mlx5: fix large VM memory region registration

When VM size is larger than 4G (u32) and memory region is larger than 4G,
the 32-bit GCD function overflowed and returned wrong value
that resulted in memory registration failure.

This patch calls 64-bit GCD function to avoid overflow.

Fixes: cc07a42da250 ("vdpa/mlx5: prepare memory regions")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Matan Azrad <matan@nvidia.com>
drivers/vdpa/mlx5/mlx5_vdpa_mem.c