ixgbe: fix bit shift overflow in VMDQ pool setup

Fix issue reported by Coverity.

Coverity ID 13193: Bad bit shift operation (BAD_SHIFT)
large_shift: In expression 1 << pool, left shifting by more than 31 bits
has undefined behavior. The shift amount, pool, is at least 32.

This patch is a rework of register addr selection logic and mask
computation to made it more readable and avoid bit overflow when 32 bit
value is shifted over its size for pool > 31.

Fixes: fe3a45fd4104 ("ixgbe: add VMDq support")

Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index c99143a..a2b170b 100644 (file)
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -4397,9 +4397,19 @@ ixgbe_set_pool_rx(struct rte_eth_dev *dev, uint16_t pool, uint8_t on)
        if (ixgbe_vmdq_mode_check(hw) < 0)
                return -ENOTSUP;
 
-       addr = IXGBE_VFRE(pool >= ETH_64_POOLS/2);
+       if (pool >= ETH_64_POOLS)
+               return -EINVAL;
+
+       /* for pool >= 32, set bit in PFVFRE[1], otherwise PFVFRE[0] */
+       if (pool >= 32) {
+               addr = IXGBE_VFRE(1);
+               val = bit1 << (pool - 32);
+       } else {
+               addr = IXGBE_VFRE(0);
+               val = bit1 << pool;
+       }
+
        reg = IXGBE_READ_REG(hw, addr);
-       val = bit1 << pool;
 
        if (on)
                reg |= val;
@@ -4424,9 +4434,19 @@ ixgbe_set_pool_tx(struct rte_eth_dev *dev, uint16_t pool, uint8_t on)
        if (ixgbe_vmdq_mode_check(hw) < 0)
                return -ENOTSUP;
 
-       addr = IXGBE_VFTE(pool >= ETH_64_POOLS/2);
+       if (pool >= ETH_64_POOLS)
+               return -EINVAL;
+
+       /* for pool >= 32, set bit in PFVFTE[1], otherwise PFVFTE[0] */
+       if (pool >= 32) {
+               addr = IXGBE_VFTE(1);
+               val = bit1 << (pool - 32);
+       } else {
+               addr = IXGBE_VFTE(0);
+               val = bit1 << pool;
+       }
+
        reg = IXGBE_READ_REG(hw, addr);
-       val = bit1 << pool;
 
        if (on)
                reg |= val;