DuckCorp » DuckCorp Infrastructure

On toushirou we got:

/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]: 2021-07-06 06:59:59 INFO     MAIN: MTA-STS daemon starting...
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]: 2021-07-06 06:59:59 INFO     MAIN: Starting eventloop...
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]: 2021-07-06 06:59:59 INFO     MAIN: uvloop enabled.
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]: 2021-07-06 06:59:59 INFO     MAIN: Eventloop started.
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]: Traceback (most recent call last):
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/bin/mta-sts-daemon", line 11, in <module>
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     load_entry_point('postfix-mta-sts-resolver==1.0.0', 'console_scripts', 'mta-sts-daemon')()
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/daemon.py", line 123, in main
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     evloop.run_until_complete(amain(cfg, evloop))
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "uvloop/loop.pyx", line 1448, in uvloop.loop.Loop.run_until_complete
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/daemon.py", line 65, in amain
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     await cache.setup()
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/postfix_mta_sts_resolver/redis_cache.py", line 34, in setup
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     self._pool = await aioredis.create_redis_pool(**self._opts)
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/commands/__init__.py", line 201, in create_redis_pool
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     loop=loop)
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/pool.py", line 56, in create_pool
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     await pool._fill_free(override_min=False)
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/pool.py", line 388, in _fill_free
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     conn = await self._create_new_connection(self._address)
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/connection.py", line 129, in create_connection
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     await conn.auth(password)
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/util.py", line 48, in wait_ok
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     res = await fut
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/connection.py", line 183, in _read_data
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     obj = await self._reader.readobj()
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3/dist-packages/aioredis/stream.py", line 94, in readobj
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     await self._wait_for_data('readobj')
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:   File "/usr/lib/python3.7/asyncio/streams.py", line 473, in _wait_for_data
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]:     await self._waiter
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou mta-sts-daemon[27711]: ConnectionResetError: [Errno 104] Connection reset by peer
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou systemd[1]: postfix-mta-sts-resolver.service: Main process exited, code=exited, status=1/FAILURE
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou systemd[1]: postfix-mta-sts-resolver.service: Failed with result 'exit-code'.
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou systemd[1]: postfix-mta-sts-resolver.service: Consumed 709ms CPU time.
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou systemd[1]: postfix-mta-sts-resolver.service: Service RestartSec=100ms expired, scheduling restart.
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou systemd[1]: postfix-mta-sts-resolver.service: Scheduled restart job, restart counter is at 1.
/var/log/daemon.log.3.gz:Jul  6 06:59:59 Toushirou systemd[1]: postfix-mta-sts-resolver.service: Consumed 709ms CPU time.

After 5 times the service seem to not have been brought back.

Is it a problem because redis was restarted? We need to investigate.

Severity high since it blocked outgoing emails on this server.