net/iavf: release port upon close