Project

General

Profile

Enhancement #689

Log aggregation and monitoring

Added by Marc Dequènes 3 months ago. Updated 3 months ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Service :: Supervision
Start date:
2020-04-04
Due date:
% Done:

0%

Estimated time:
Patch Available:
Confirmed:
No
Branch:
Entity:
DuckCorp
Security:
Help Needed:

Description

I could not find a simple system really doing something similar to logcheck on an aggregation server.

As I want to take advantage of the proper split of information we can get in a journal (currently systemd journald) instead of a long line of text formatted in various ways, I was thinking about simply using systemd-journal-remote to collect all remote logs (sent by clients with systemd-journal-upload). I do not feel the need for a fancy UI and I would prefer to avoid installing a web service (currently the monitoring has one but we plan to split), so simply using journalctl with the --merge option should be sufficient for our need.

Then to be consistent we would also need to replace logcheck to do centralized filtering and alerting. I found some projects but most dead of not having a very nice approach. On project caught my eyes though: https://github.com/twaugh/journal-brief
I like the idea to use the Python systemd binding to directly access the journal and be able to filter using any field. Then we could run it in batch or continuously and think how we would like alerts to reach us. It could be interesting to use the priority of the message to send through vwrious means XMPP/IRC for urgent matters and email reports for the rest.

When we're happy with the new system we then need to decide if we keep rsyslog or downsize its config.

History

#1

Updated by Marc Dequènes 3 months ago

  • Description updated (diff)

Also available in: Atom PDF