PostfixがデフォルトでIPv6が有効になってるので

vi /etc/postfix/main.cf

# Enable IPv4, and IPv6 if supported
#inet_protocols = all
inet_protocols = ipv4

に変更。

DovecotもデフォルトでIPv6が有効になってるので

vi /etc/dovecot/dovecot.conf

# A comma separated list of IPs or hosts where to listen in for connections. 
# "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
# If you want to specify non-default ports or anything more complex,
# edit conf.d/master.conf.
#listen = *, ::
listen = *

に変更(*´ｪ`)