Project

General

Profile

Bug #775

Ninjabot doesn't handle unreachable network

Added by Pierre-Louis Bonicoli 5 months ago. Updated 4 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Service :: Supervision
Start date:
2022-07-10
Due date:
% Done:

100%

Estimated time:
Patch Available:
Yes
Confirmed:
No
Branch:
Entity:
DuckCorp
Security:
No
Help Needed:

Description

Ninjabot was unable to reconnect after encountering a temporarily unreachable network:

Jul 07 00:40:11 orthos.duckcorp.org ninjabot[1608725]: <= {} None PINGĀ ['irc2.duckcorp.org']
Jul 07 00:41:31 orthos.duckcorp.org ninjabot[1608725]: [126B blob data]
Jul 07 00:42:08 orthos.duckcorp.org ninjabot[1608725]: [132B blob data]
Jul 07 00:46:31 orthos.duckcorp.org ninjabot[1608725]: [129B blob data]
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]: Traceback (most recent call last):
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:   File "/opt/ninjabot/venv/bin/ninjabot", line 8, in <module>
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:     sys.exit(ninjabot.cli())
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:   File "/opt/ninjabot/venv/lib/python3.9/site-packages/ninjabot/ninjabot.py", line 38, in cli
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:     client.start()
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:   File "/opt/ninjabot/venv/lib/python3.9/site-packages/py_irc/irc.py", line 99, in start
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:     buf = self.socket.recv(4096)
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:   File "/usr/lib/python3.9/ssl.py", line 1226, in recv
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:     return self.read(buflen)
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:   File "/usr/lib/python3.9/ssl.py", line 1101, in read
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]:     return self._sslobj.read(len)
Jul 07 00:58:02 orthos.duckcorp.org ninjabot[1608725]: OSError: [Errno 101] Network is unreachable
Jul 07 04:56:32 orthos.duckcorp.org ninjabot[1608725]: [127B blob data]
Jul 07 04:56:32 orthos.duckcorp.org ninjabot[1608725]: Connection broke up
Jul 07 04:56:32 orthos.duckcorp.org ninjabot[1608725]: Attemting to connect to irc.milkypond.org
Jul 07 04:56:32 orthos.duckcorp.org ninjabot[1608725]: Connected to irc.milkypond.org

The bot wasn't connected at 04:56:32, a manual restart of the service was required.

History

#1

Updated by Pierre-Louis Bonicoli 5 months ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 50
  • Patch Available set to Yes
  • Confirmed changed from No to Yes
  • Security set to No
#2

Updated by Pierre-Louis Bonicoli 4 months ago

  • Status changed from In Progress to Resolved
  • % Done changed from 50 to 100
  • Confirmed changed from Yes to No

I just updated ninjabot:

$ UPGRADE=1 ansible-playbook playbooks/tenants/duckcorp/monitoring.yml -vv --diff
[...]
TASK [Install NinjaBot for Prometheus] **************************************************************************************************************************************
task path: ansible/playbooks/tenants/duckcorp/monitoring.yml:79
Tuesday 09 August 2022  01:18:30 +0200 (0:00:00.376)       0:00:32.083 ******** 
NOTIFIED HANDLER Restart NinjaBot for Orthos
changed: [Orthos] => changed=true 
  cmd:
  - /opt/ninjabot//venv/bin/pip3
  - install
  - -U
  - git+https://gitlab.crans.org/ninjabot/ninjabot-prometheus.git
  name:
  - git+https://gitlab.crans.org/ninjabot/ninjabot-prometheus.git
  requirements: null
  state: latest
  stderr: |2-
      Running command git clone -q https://gitlab.crans.org/ninjabot/ninjabot-prometheus.git /tmp/user/0/pip-req-build-nnocwhje
      Running command git clone -q https://gitlab.crans.org/ninjabot/ninjabot-core.git /tmp/user/0/pip-install-opw7zuxm/ninjabot-core_72af2ece38f84f90879d046dec81efc1
      Running command git clone -q https://gitlab.crans.org/ninjabot/pyirc.git /tmp/user/0/pip-install-opw7zuxm/py-irc_a08f51e0c0c94775bc50e36201e38553
  stderr_lines: <omitted>
  stdout: |-
    Collecting git+https://gitlab.crans.org/ninjabot/ninjabot-prometheus.git
      Cloning https://gitlab.crans.org/ninjabot/ninjabot-prometheus.git to ./pip-req-build-nnocwhje
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
        Preparing wheel metadata: started
        Preparing wheel metadata: finished with status 'done'
    Collecting ninjabot-core@ git+https://gitlab.crans.org/ninjabot/ninjabot-core.git@master
      Cloning https://gitlab.crans.org/ninjabot/ninjabot-core.git (to revision master) to ./pip-install-opw7zuxm/ninjabot-core_72af2ece38f84f90879d046dec81efc1
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
        Preparing wheel metadata: started
        Preparing wheel metadata: finished with status 'done'
    Requirement already satisfied: flask>=2.0 in /opt/ninjabot/venv/lib/python3.9/site-packages (from ninjabot-prometheus==0.0.3) (2.0.2)
    Requirement already satisfied: python-dateutil>=2.8 in /opt/ninjabot/venv/lib/python3.9/site-packages (from ninjabot-prometheus==0.0.3) (2.8.2)
    Collecting py_irc@ git+https://gitlab.crans.org/ninjabot/pyirc.git@0.0.4
      Cloning https://gitlab.crans.org/ninjabot/pyirc.git (to revision 0.0.4) to ./pip-install-opw7zuxm/py-irc_a08f51e0c0c94775bc50e36201e38553
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
        Preparing wheel metadata: started
        Preparing wheel metadata: finished with status 'done'
    Requirement already satisfied: pyyaml>=6.0 in /opt/ninjabot/venv/lib/python3.9/site-packages (from ninjabot-core@ git+https://gitlab.crans.org/ninjabot/ninjabot-core.git@master->ninjabot-prometheus==0.0.3) (6.0)
    Requirement already satisfied: Jinja2>=3.0 in /opt/ninjabot/venv/lib/python3.9/site-packages (from flask>=2.0->ninjabot-prometheus==0.0.3) (3.0.3)
    Requirement already satisfied: Werkzeug>=2.0 in /opt/ninjabot/venv/lib/python3.9/site-packages (from flask>=2.0->ninjabot-prometheus==0.0.3) (2.0.2)
    Requirement already satisfied: itsdangerous>=2.0 in /opt/ninjabot/venv/lib/python3.9/site-packages (from flask>=2.0->ninjabot-prometheus==0.0.3) (2.0.1)
    Requirement already satisfied: click>=7.1.2 in /opt/ninjabot/venv/lib/python3.9/site-packages (from flask>=2.0->ninjabot-prometheus==0.0.3) (8.0.3)
    Requirement already satisfied: MarkupSafe>=2.0 in /opt/ninjabot/venv/lib/python3.9/site-packages (from Jinja2>=3.0->flask>=2.0->ninjabot-prometheus==0.0.3) (2.0.1)
    Requirement already satisfied: six>=1.5 in /opt/ninjabot/venv/lib/python3.9/site-packages (from python-dateutil>=2.8->ninjabot-prometheus==0.0.3) (1.16.0)
    Building wheels for collected packages: ninjabot-core, py-irc
      Building wheel for ninjabot-core (PEP 517): started
      Building wheel for ninjabot-core (PEP 517): finished with status 'done'
      Created wheel for ninjabot-core: filename=ninjabot_core-0.0.7-py3-none-any.whl size=4235 sha256=8aa7bc7a799016113077a385bf3305c2fbc1aa18f08be734c32f279fa70400bd
      Stored in directory: /tmp/user/0/pip-ephem-wheel-cache-3zk05372/wheels/89/f9/c9/f3f60cb219fa451662229a7bebd9d2f2d472a542eb7bff9683
      Building wheel for py-irc (PEP 517): started
      Building wheel for py-irc (PEP 517): finished with status 'done'
      Created wheel for py-irc: filename=py_irc-0.0.4-py3-none-any.whl size=5429 sha256=12d3f2123b76c4f3e3beaf3d31196df9ffd3d8c13a1f670208f30ae254fa4da6
      Stored in directory: /tmp/user/0/pip-ephem-wheel-cache-3zk05372/wheels/2e/f4/85/0c3aecd8d110f66fd013e732d955f52afce883740e5c78ad1b
    Successfully built ninjabot-core py-irc
    Installing collected packages: py-irc, ninjabot-core
      Attempting uninstall: py-irc
        Found existing installation: py-irc 0.0.3
        Uninstalling py-irc-0.0.3:
          Successfully uninstalled py-irc-0.0.3
      Attempting uninstall: ninjabot-core
        Found existing installation: ninjabot-core 0.0.6
        Uninstalling ninjabot-core-0.0.6:
          Successfully uninstalled ninjabot-core-0.0.6
    Successfully installed ninjabot-core-0.0.7 py-irc-0.0.4
  stdout_lines: <omitted>
  version: null
  virtualenv: /opt/ninjabot//venv
[...]

Also available in: Atom PDF