vhost: fix error path when setting memory tables