DuckCorp Projects: Issueshttps://projects.duckcorp.org/https://projects.duckcorp.org/favicon.ico?16699090422022-03-15T21:29:07ZDuckCorp Projects
Redmine DuckCorp Infrastructure - Bug #759 (In Progress): redmine instances don't send any notificationhttps://projects.duckcorp.org/issues/7592022-03-15T21:29:07ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p>Since the redmine instances are hosted within a LXC container, email notifications are no longer sent.</p>
<p>It looks like the issue comes from the Redmine configuration and 127.0.0.1:25 being used within the container.</p>
<p>The following configuration update isn't sufficient:<br /><pre>
--- /etc/redmine/dc/configuration.yml 2022-03-15 22:28:00.095274510 +0000
+++ /etc/redmine/dc/configuration.yml.new 2022-03-15 22:27:44.102827009 +0000
@@ -4,8 +4,8 @@
email_delivery:
delivery_method: :smtp
smtp_settings:
- address: 127.0.0.1
- domain: ''
+ address: 10.0.7.1
+ domain: 'projects.duckcorp.org'
enable_starttls_auto: false
port: 25
</pre><br />due to the grey listing configuration:<br /><pre>
Mar 15 23:12:37 Toushirou postfix/smtpd[1597691]: connect from unknown[10.0.7.2]
Mar 15 23:12:37 Toushirou postfix/smtpd[1597691]: 4KJ71x5crKz4Bs: client=unknown[10.0.7.2]
Mar 15 23:12:37 Toushirou postfix/cleanup[1597693]: 4KJ71x5crKz4Bs: message-id=<redmine.journal-2400.20220315221237.3bd6c5f55c0c0d17@projects.duckcorp.org>
Mar 15 23:12:38 Toushirou postfix/cleanup[1597693]: 4KJ71x5crKz4Bs: milter-reject: END-OF-MESSAGE from unknown[10.0.7.2]: 4.7.1 Try again later; from=<issues@projects.duckcorp.org> to=<[redacted]@ir5.eu> proto=ESMTP helo=<projects.duckcorp.org>
</pre></p>
<p><a class="user active user-mention" href="https://projects.duckcorp.org/users/3">@Marc Dequènes</a> should the grey listing be disabled for 10.0.7.2 or is there another way ?</p> DuckCorp Infrastructure - Enhancement #745 (New): ban IPs that try to authenticate with a nonexis...https://projects.duckcorp.org/issues/7452021-11-24T14:03:15ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p>Fail2ban should block the following attemps:<br /><pre>
Nov 24 15:06:46 Toushirou dovecot[1308700]: auth: ldap(<redacted>,XXX.237.103.19): unknown user
Nov 24 15:07:00 Toushirou dovecot[1308700]: auth: ldap(<redacted>,XXX.237.103.19): unknown user
Nov 24 15:07:20 Toushirou dovecot[1308700]: auth: ldap(<redacted>,XXX.237.103.19): unknown user
Nov 24 15:07:30 Toushirou dovecot[1308700]: auth: ldap(<redacted>,XXX.237.103.19): unknown user
Nov 24 15:07:44 Toushirou dovecot[1308700]: auth: ldap(<redacted>,XXX.237.103.19): unknown user
Nov 24 15:08:04 Toushirou dovecot[1308700]: auth: ldap(<redacted>,XXX.237.103.19): unknown user
</pre></p>
<p>Some numbers in order to support the new filter (the oldest entry in the journal is 7 days old):<br /><pre>
root@Toushirou:~# # count all entries
root@Toushirou:~# journalctl -g '(auth:.*unknown)' | wc -l
5032
root@Toushirou:~# # check the regex
root@Toushirou:~# journalctl -g '(auth:.*unknown)' | sed -n 's/.*ldap([^,]\+,\([^,)]\+\)\(,<[^>]\+>\)\?):.*/\1/p' | sort | uniq -c | sort -nr | awk '{print $1}' | paste -sd+ | bc
5029
root@Toushirou:~# # display the most used IPs
root@Toushirou:~# journalctl -g '(auth:.*unknown)' | sed -n 's/.*ldap([^,]\+,\([^,)]\+\)\(,<[^>]\+>\)\?):.*/\1/p' | sort | uniq -c | sort -nr | awk '{print $1}' | head -n 10
741
566
467
362
307
182
177
174
167
161
# There are 697 different IPs, the twenty most used produce 85% of the login failure.
</pre></p> DuckCorp Infrastructure - Enhancement #657 (In Progress): OOB management for Toushirouhttps://projects.duckcorp.org/issues/6572019-07-04T13:48:43ZMarc Dequènesduck@duckcorp.org
<p>This involves, plugging the cable, network configuration with Hivane, configuration on the server and documentation.</p>
<p>As Toushirou recently crashed for no reason that could be explained I raised the priority a bit.</p> DuckCorp Infrastructure - Bug #656 (New): Raise the backup system from the deadhttps://projects.duckcorp.org/issues/6562019-07-04T13:45:01ZMarc Dequènesduck@duckcorp.org
<p>Old files from Nicecity-OLD needs to be retrieved and put in the right place.</p>
<p>Then we need to resume the work around the new backup system which was left unfinished in <a class="issue tracker-2 status-7 priority-4 priority-default parent" title="Enhancement: Change Backup System (In Progress)" href="https://projects.duckcorp.org/issues/497">#497</a>.</p>
<p>If there's any reason to change our mind that's the right moment. I personally have no problem with the current plan.<br />If we go on then we need to create a new simpler burp role as explained by Pilou on IRC.</p> DuckCorp Infrastructure - Enhancement #652 (In Progress): Orfeo would like a brand new bodyhttps://projects.duckcorp.org/issues/6522019-05-08T16:47:00ZMarc Dequènesduck@duckcorp.org
<p>It is a followup of <a class="issue tracker-2 status-3 priority-4 priority-default closed parent" title="Enhancement: Toushirou would like a brand new body (Resolved)" href="https://projects.duckcorp.org/issues/537">#537</a> for Orfeo only.</p>
<p>Orfeo is old too and even if we do not need more power now it crashed last year for an undetermined reason and we should think of the future.</p>
<p>I'm still looking into the possibility of hosting it on a Elwing container using LXD. My internet connection is better even if not wonderful. And my complicated network config and Hivane L2TP tunnel are stable now. As we might never have the ability to change the machine in the current hosting I guess it's even more an interesting possibility to explore.</p> mkcert - Bug #597 (New): Please review Ansible Vault supporthttps://projects.duckcorp.org/issues/5972017-09-24T06:21:13ZMarc Dequènesduck@duckcorp.org
<p>It has been made as less intrusive as possible to avoid introducing bugs. Most probably a rework of the path variables would be better but this is not the goal of this PR,</p> DuckCorp Infrastructure - Review #591 (In Progress): Commit eb15755f0416e4a07c8e585ec42db79051edb78fhttps://projects.duckcorp.org/issues/5912017-08-29T12:57:21ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<ul>
<li>ansible/host_vars/Thorfinn/backup.yml: should not <code>/usr/share/rbot/plugins</code> be added to <code>backup_important_custom</code> (I noted 'manual changes tracking in rbot plugins') ?</li>
<li>ansible/host_vars/Elwing/backup.yml: should not <code>/data/Vingilot_backup</code>, <code>/data/elwing_sys</code>, <code>/data/share/Data-Important</code> be added ?</li>
<li>s/backup_exclude/backup_excludes/ (same for backup_exclude_regex)</li>
<li>burp role should handle <code>exclude_regex</code> too</li>
</ul> DuckCorp Infrastructure - Review #585 (In Progress): backup_duckhttps://projects.duckcorp.org/issues/5852017-08-29T11:59:20ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eumkcert - Review #579 (New): Please review error handling fixeshttps://projects.duckcorp.org/issues/5792017-07-24T10:51:36ZMarc Dequènesduck@duckcorp.orgDuckCorp Infrastructure - Review #518 (In Progress): Review branch backuphttps://projects.duckcorp.org/issues/5182017-04-03T11:58:27ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p>This is a start: the committed configuration backups only PostgreSQL databases hosted on Toushirou.</p>
<p>Other hosts/directories will be added latter.</p> DuckCorp Infrastructure - Enhancement #497 (In Progress): Change Backup Systemhttps://projects.duckcorp.org/issues/4972016-10-16T12:48:58ZMarc Dequènesduck@duckcorp.org
Bacula has some drawbacks which do not improve over the last years, mainly:
<ul>
<li>horrible CLI, difficult to search through backups and volumes</li>
<li>retention is awfully complex to setup</li>
<li>consolidated backups never worked with file volumes</li>
<li>cannot resume, so if a full backup with a lot of data fails midway, useless incomplete volumes pile-up and eat all the available space</li>
</ul>
We may use another system for laptop/user backup, like duplicity for eg. On a trusted centralized backup system for server I would list these criteria:
<ul>
<li>secure transfer (TLS, SSH…)</li>
<li>delta transfer</li>
<li>compression</li>
<li>resumable</li>
<li>incremental backup: either full-incremental without full backup (except initial setup), or consolidation</li>
<li>open fifo option: useful to backup databases without reserving a huge local space, allows on-the-fly backup stream with unmodified tools (mysqldump, pg_dump…)</li>
<li>proper retention settings: we should be able to express this: keep 1 backup per day during 7 days, then 1 per week during 4 weeks, then 1 per month during 1 year</li>
<li>long time restoration: backup format breaks infrequently and either new software can read old formats or a straightforward command can convert them to the new format</li>
<li>maintained: at least one maintenance release per year, no critical bug without at least a workaround for more than a month</li>
<li>CLI</li>
</ul>
Also, would-be-nice features but we can live without it:
<ul>
<li>deduplication</li>
<li>single entrypoint when different category of data are to be saved (different retention for eg): single daemon and open port</li>
<li>exclude dir if contains file: allows user to exclude their own dirs, like we did with Bacula, just 'touch .nobackup' and the backup software will skip the dir</li>
</ul>
<p>Backup scheduling:<br /><img src="https://projects.duckcorp.org/attachments/download/85/backup_scheduling.jpg" loading="lazy" style="width: 100%;" alt="" /></p> Bip - Enhancement #343 (New): Allow to blreset all queries or all channelshttps://projects.duckcorp.org/issues/3432014-07-24T00:21:01ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p><code>blreset</code> command allows to reset backlog of an entire connection, a chan, a query.</p>
<p>Be able to reset all queries or all channels would be a nice feature.</p> Bip - Bug #342 (New): 'list connections' command doesn't display status of channelshttps://projects.duckcorp.org/issues/3422014-07-24T00:13:06ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p>It seems that output of <code>list connections</code> command should use a suffix on channels without backlog: <a class="source" href="https://projects.duckcorp.org/projects/bip/repository/bip/entry/src/bip.c#L1395">source:src/bip.c#L1395</a>, but this is not the case.</p>
<p><code>list connections</code> doesn't display a suffix on any channel:</p>
<pre>
02:04:18 Pilou | list connections
[...]
02:04:18 -bip | * milkypond to milkypond as "pilou" (pilou!pilou) :
02:04:18 -bip | Options:
02:04:18 -bip | Channels (* with key, ` no backlog) #test #milkypond #DuckCorp
02:04:18 -bip | Status: connected !
</pre> Bip - Bug #341 (New): 'bip list connections' command should display querieshttps://projects.duckcorp.org/issues/3412014-07-24T00:01:23ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p>The command <code>bip list connections</code> lists channels for all connections.</p>
<p>Queries could be listed too.</p> Bip - Bug #262 (In Progress): using ircnet bip crashes if a channel has two nicks different only ...https://projects.duckcorp.org/issues/2622011-11-13T15:30:32ZPierre-Louis Bonicolipierre-louis.bonicoli@ir5.eu
<p>nitram reported:<br /><pre>
as soon as i join a channel that has the nicks "~mc" and "mc" on ircnet bip crashes with "11-11-2011 15:55:44 FATAL: Element with key mc already in hash b9495ce0
</pre></p>
<p>I found two problems:</p>
<p><strong>First</strong>, in <code>irc_353</code> function ( <a class="source" href="https://projects.duckcorp.org/projects/bip/repository/bip/revisions/a46b8bd2/entry/src/irc.c#L1362">source:src/irc.c@a46b8bd2#L1362</a> ) we discard '~' character when storing operator/voice mask foreach nickname. For example if the irc server send<br /><code>'ircnet.optilian.net' ':ircnet.optilian.net 353 pilou = #plopplopplop :pilou ~lolll219 lolll219 '</code> (user <code>pilou</code> joining <code>ircnet.optilian.net</code> where tho users <code>~lolll219</code> <code>lolll219</code> are here)<br />we store operator/voice mask of <code>lolll219</code> twice (once for <code>~lolll219</code> and another for <code>lolll219</code>).</p>
<p>This lead to many errors:</p>
<ul>
<li>if either <code>lolll219</code> or <code>~lolll219</code> have a not empty operator/voice mask, then problem reported by nitram appears: the second <code>hash_insert</code> fails.</li>
</ul>
<ul>
<li>when <code>~lolll219</code> or <code>lolll219</code> send irc <code>part</code> command, <code>irc_part</code> function ( <a class="source" href="https://projects.duckcorp.org/projects/bip/repository/bip/revisions/a46b8bd2/entry/src/irc.c#L1498">source:src/irc.c@a46b8bd2#L1498</a> ) encounters problem.<br />If <code>~lolll219</code> quit then his operator/voice mask can not be found (it was not stored) and then <code>irc_part</code> return <code>ERR_PROTOCOL</code>:<br /><pre>
13-11-2011 14:38:22 ERROR: [ircnet] Error in protocol, closing...
13-11-2011 14:38:22 ERROR: [ircnet] reconnecting in 0 seconds
</pre></li>
</ul>
<ul>
<li>If <code>lolll219</code> quit then an assertion fails, indeed the <code>lolll219</code> key is present twice in the operator/voice mask hash:<br /><pre>
13-11-2011 14:37:29 FATAL: 80b3288 appears twice in list
</pre></li>
</ul>
<p><strong>Second</strong> problem: it should not be possible to store two identical key in one hash. <code>list_remove_if_exists</code> function ( <a class="source" href="https://projects.duckcorp.org/projects/bip/repository/bip/revisions/a46b8bd2/entry/src/util.c#L370">source:src/util.c@a46b8bd2#L370</a> ) - called by <code>irc_part</code> - verify this assertion and the assertion fails.</p>
<p>Currently insertion of two identical keys occurs because instead of checking if the hash contains already an identical key, we check if the value corresponding to this key is NULL or not ( <a class="source" href="https://projects.duckcorp.org/projects/bip/repository/bip/revisions/a46b8bd2/entry/src/util.c#L566">source:src/util.c@a46b8bd2#L566</a> ):</p>
<pre>
void hash_insert(hash_t *hash, const char *key, void *ptr)
[...]
if (hash_get(hash, key))
fatal("Element with key %s already in hash %x\n", key, hash);
</pre>
<p>So it's possible to store many identical key associated to 0/NULL value.</p>
<p>And the associated value for the key in operator/voice mask hash can be 0/NULL:<br /><pre>
long int ovmask = 0;
[...]
hash_insert(&channel->ovmasks, nick, (void *)ovmask);
</pre></p>