Actions
Enhancement #782
closed
Migrate to Debian Bookworm
Start date:
2023-06-28
Due date:
% Done:
100%
Estimated time:
Patch Available:
Confirmed:
No
Branch:
bookworm
Entity:
DuckCorp
Security:
Help Needed:
No
Description
Quack,
This ticket is tracking the migration steps:- ✅ custom/backported packages:
- ✅ identify packages that are still needed => spoolinger, python-certbot-dns-rfc2136
- ✅ setup suite in custom repo
- ✅ update packages and upload them
- ✅ add non-free-firmware component to get CPU microcode updates; Nicecity also needs firmware-realtek unfortunately)
- 🔳 needed software/config changes (to update in Ansible):
- ✅ apache2: none
- ✅ atheme-services: none
- ✅ dovecot: expire plugin removed: use mailbox/autoexpunge instead
- ✅ icecast2: none
- ✅ inspircd: pcre module is now pcre2
- 🔳 logcheck: rsyslog now defaults to “high precision timestamps” and we need to update regexs in our custom filters -> see #785
- ✅ lxd: remove, we'll be using podman+quadlets instead (not needed for redmine anymore)
- ✅ mailman3: none
- ✅ mediawiki: config check needed
- ✅ mariadb-server: innodb_large_prefix and innodb_file_format are deprecated and can simply be removed (we used the new default already)
- ✅ matrix-synapse: missing, hopefully should come as backports like it did for Bullseye
- ✅ openldap: none
- ✅ passenger: none
- ✅ postfix: none
- ✅ postgresql: none
- ✅💡 proftpd: missing, maybe it's time to deprecate it for good
- ✅ prometheus: lots of changes but current config should be fine
- ✅ redis-server: none
- ✅ redmine: switch back to the package
- ✅ roundcube: config has changed slightly
- ✅ xl2tpd: control command renamed
- …
- ✅ upgrade: (follow the OS_Upgrade procedure) (suggested order)
- ✅ Elwing
- ✅ Nicecity
- ✅ Orhos
- ✅ Thorfinn
- ✅ Jinta
- ✅ Toushirou
- ✅ Orfeo
- 🔳 post-check: (possibly moved into specific ticket later)
- 🔳 check services still missing systemd config (we might be able to remove some workaround in Ansible) -> see #786
- 🔳 do we still need rsyslog? maybe for logcheck -> see #784
- 🔳 remove obsolete logcheck filters (maybe start again from scratch and pull back rules we had when we hit them?) -> see #785
- ✅ remove obsolete log files:
- /var/log/mail.{info,warn,err}
- /var/log/lpr.log
- /var/log/{messages,debug,daemon.log}
- anacron: check is it needs reenabling for software that did not switch to systemd
Updated by Marc Dequènes 10 months ago
- Subject changed from Migrate to Bookworm to Migrate to Debian Bookworm
- Description updated (diff)
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
- Status changed from New to In Progress
- Branch set to bookworm
Updated by Marc Dequènes 10 months ago
On Nicecity we have a kernel trace:
Jun 29 11:15:57 Nicecity kernel: i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem Jun 29 11:15:57 Nicecity kernel: [drm] Skipping LVDS initialization for Gigabyte GA-D525TUD Jun 29 11:15:57 Nicecity kernel: ------------[ cut here ]------------ Jun 29 11:15:57 Nicecity kernel: i915 0000:00:02.0: Useless DMI match. Internal LVDS support disabled by VBT Jun 29 11:15:57 Nicecity kernel: WARNING: CPU: 0 PID: 336 at drivers/gpu/drm/i915/display/intel_lvds.c:840 intel_lvds_init+0x59b/0x760 [i915] Jun 29 11:15:57 Nicecity kernel: Modules linked in: i915(+) evdev drm_buddy video wmi iTCO_wdt intel_pmc_bxt coretemp drm_display_helper iTCO_vendor_support sha512_ssse> Jun 29 11:15:57 Nicecity kernel: CPU: 0 PID: 336 Comm: (udev-worker) Not tainted 6.1.0-9-amd64 #1 Debian 6.1.27-1 Jun 29 11:15:57 Nicecity kernel: Hardware name: Gigabyte Technology Co., Ltd. D525TUD/D525TUD, BIOS F9 12/14/2012 Jun 29 11:15:57 Nicecity kernel: RIP: 0010:intel_lvds_init+0x59b/0x760 [i915] Jun 29 11:15:57 Nicecity kernel: Code: 08 75 af 48 8b 7b 08 48 8b 5f 50 48 85 db 75 03 48 8b 1f e8 b7 f3 91 c6 48 89 da 48 c7 c7 70 85 e4 c0 48 89 c6 e8 55 f4 2d c6 <0f> Jun 29 11:15:57 Nicecity kernel: RSP: 0018:ffffbf2f006b3a48 EFLAGS: 00010282 Jun 29 11:15:57 Nicecity kernel: RAX: 0000000000000000 RBX: ffff96df00b54eb0 RCX: 0000000000000000 Jun 29 11:15:57 Nicecity kernel: RDX: 0000000000000002 RSI: ffffffff8833fa66 RDI: 00000000ffffffff Jun 29 11:15:57 Nicecity kernel: RBP: ffff96df03cd02b8 R08: 0000000000000000 R09: ffffbf2f006b38b8 Jun 29 11:15:57 Nicecity kernel: R10: 0000000000000003 R11: ffffffff88ad43a8 R12: ffff96df03cd02c0 Jun 29 11:15:57 Nicecity kernel: R13: 0000000000000001 R14: 0000000000000000 R15: ffff96df00c2b000 Jun 29 11:15:57 Nicecity kernel: FS: 00007fe6e3ac28c0(0000) GS:ffff96df3bc00000(0000) knlGS:0000000000000000 Jun 29 11:15:57 Nicecity kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 29 11:15:57 Nicecity kernel: CR2: 00007f6a1979ae20 CR3: 0000000103f5c000 CR4: 00000000000006f0 Jun 29 11:15:57 Nicecity kernel: Call Trace: Jun 29 11:15:57 Nicecity kernel: <TASK> Jun 29 11:15:57 Nicecity kernel: ? delay_tsc+0x8f/0xc0 Jun 29 11:15:57 Nicecity kernel: intel_modeset_init_nogem+0x10ba/0x1150 [i915] Jun 29 11:15:57 Nicecity kernel: ? _raw_spin_unlock_irq+0x1b/0x40 Jun 29 11:15:57 Nicecity kernel: ? intel_irq_postinstall+0x43a/0x610 [i915] Jun 29 11:15:57 Nicecity kernel: i915_driver_probe+0x5c5/0xe50 [i915] Jun 29 11:15:57 Nicecity kernel: ? i915_pci_probe+0x94/0x1d0 [i915] Jun 29 11:15:57 Nicecity kernel: local_pci_probe+0x3e/0x80 Jun 29 11:15:57 Nicecity kernel: pci_device_probe+0xc3/0x230 Jun 29 11:15:57 Nicecity kernel: really_probe+0xdb/0x380 Jun 29 11:15:57 Nicecity kernel: ? pm_runtime_barrier+0x50/0x90 Jun 29 11:15:57 Nicecity kernel: __driver_probe_device+0x78/0x120 Jun 29 11:15:57 Nicecity kernel: driver_probe_device+0x1f/0x90 Jun 29 11:15:57 Nicecity kernel: __driver_attach+0xce/0x1c0 Jun 29 11:15:57 Nicecity kernel: ? __device_attach_driver+0x110/0x110 Jun 29 11:15:57 Nicecity kernel: bus_for_each_dev+0x84/0xd0 Jun 29 11:15:57 Nicecity kernel: bus_add_driver+0x1ae/0x200 Jun 29 11:15:57 Nicecity kernel: driver_register+0x89/0xe0 Jun 29 11:15:57 Nicecity kernel: i915_init+0x1f/0x7f [i915] Jun 29 11:15:57 Nicecity kernel: ? 0xffffffffc0f4e000 Jun 29 11:15:57 Nicecity kernel: do_one_initcall+0x56/0x220 Jun 29 11:15:57 Nicecity kernel: do_init_module+0x4a/0x200 Jun 29 11:15:57 Nicecity kernel: __do_sys_finit_module+0xac/0x120 Jun 29 11:15:57 Nicecity kernel: do_syscall_64+0x58/0xc0 Jun 29 11:15:57 Nicecity kernel: ? syscall_exit_to_user_mode+0x17/0x40 Jun 29 11:15:57 Nicecity kernel: ? do_syscall_64+0x67/0xc0 Jun 29 11:15:57 Nicecity kernel: ? exit_to_user_mode_prepare+0x147/0x1d0 Jun 29 11:15:57 Nicecity kernel: ? syscall_exit_to_user_mode+0x17/0x40 Jun 29 11:15:57 Nicecity kernel: ? do_syscall_64+0x67/0xc0 Jun 29 11:15:57 Nicecity kernel: ? do_syscall_64+0x67/0xc0 Jun 29 11:15:57 Nicecity kernel: ? do_syscall_64+0x67/0xc0 Jun 29 11:15:57 Nicecity kernel: entry_SYSCALL_64_after_hwframe+0x63/0xcd Jun 29 11:15:57 Nicecity kernel: RIP: 0033:0x7fe6e3d4d4f9 Jun 29 11:15:57 Nicecity kernel: Code: 08 89 e8 5b 5d c3 66 2e 0f 1f 84 00 00 00 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> Jun 29 11:15:57 Nicecity kernel: RSP: 002b:00007ffec70cfe08 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 Jun 29 11:15:57 Nicecity kernel: RAX: ffffffffffffffda RBX: 000055af12abe3e0 RCX: 00007fe6e3d4d4f9 Jun 29 11:15:57 Nicecity kernel: RDX: 0000000000000000 RSI: 00007fe6e3ee0efd RDI: 0000000000000016 Jun 29 11:15:57 Nicecity kernel: RBP: 00007fe6e3ee0efd R08: 0000000000000000 R09: 000055af12aba430 Jun 29 11:15:57 Nicecity kernel: R10: 0000000000000016 R11: 0000000000000246 R12: 0000000000020000 Jun 29 11:15:57 Nicecity kernel: R13: 0000000000000000 R14: 000055af12b7dfe0 R15: 000055af116eae4f Jun 29 11:15:57 Nicecity kernel: </TASK> Jun 29 11:15:57 Nicecity kernel: ---[ end trace 0000000000000000 ]--- Jun 29 11:15:57 Nicecity kernel: i915 0000:00:02.0: [drm] Initialized overlay support. Jun 29 11:15:57 Nicecity kernel: [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
I can't se the display so maybe there's an impact but the rest of the initialization goes on and the machine seem to work fine. Could not find the same exact problem on the Internet, so I'm not sure what to do with that.
Updated by Marc Dequènes 10 months ago
On Nicecity openipmi fails to start:
Jun 29 11:18:32 Nicecity kernel: ipmi_si: IPMI System Interface driver Jun 29 11:18:32 Nicecity kernel: ipmi_si: Unable to find any System Interface(s) Jun 29 11:18:32 Nicecity openipmi[3581]: Starting ipmi drivers ipmi failed!
Why do we even have this installed in the first place?
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
- % Done changed from 0 to 20
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
On Orthos the ninjabot Python venv needed to be upgraded to the new Python major version:
python3 -m venv --upgrade /opt/ninjabot//venv
Updated by Marc Dequènes 10 months ago
On Orthos same problem as Nicecity with openipmi.
Updated by Marc Dequènes 10 months ago
On Elwing logrotate failed, affecting Orfeo, Jinta and Thorfinn too:
un 30 00:00:01 Elwing systemd[1]: Starting logrotate.service - Rotate log files... Jun 30 00:00:01 Elwing logrotate[84610]: /usr/sbin/invoke-rc.d: 538: /etc/init.d/rsyslog: not found Jun 30 00:00:01 Elwing logrotate[84607]: invoke-rc.d: initscript rsyslog, action "rotate" failed. Jun 30 00:00:01 Elwing logrotate[84524]: error: error running non-shared postrotate script for /var/log/burp-client.log of '/var/log/burp-client.log ' Jun 30 00:00:02 Elwing systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE Jun 30 00:00:02 Elwing systemd[1]: logrotate.service: Failed with result 'exit-code'. Jun 30 00:00:02 Elwing systemd[1]: Failed to start logrotate.service - Rotate log files.
In /etc/logrotate.d/burp is does a invoke-rc.d rsyslog rotate. Now that the sysv config has vanished this cannot work therefore I deleted the file on all hosts.
This config is not in the "new" Ansible repo and I cannot updte the rules accordingly.
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
- % Done changed from 20 to 40
Updated by Marc Dequènes 10 months ago
Investing problems with the Apache on Toushirou: it is very slow and often times out. It can be experienced from another Internet access, is visible in the monitoring, and can even be reproduced locally to some extent, therefore this is not network related. There is no resource shortage, no lack of entropy, and nothing useful in the logs so far. I'll keep investigating.
Updated by Marc Dequènes 10 months ago
On Toushirou this was a DNS problem: Hivane resolvers did not allow Toushirou's Nerim IPs to resolve but with the nameserver rotation I fail to see it during my tests.
Updated by Marc Dequènes 10 months ago
Fixed redmine registration, this is package bug and I'm preparing an official upload but in the meanwhile I built a package for us.
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
- % Done changed from 40 to 70
Fixed a few things (see repo).
openipmi is not needed anymore, therefore I removed it.
Toushirou should be in good shape now.
Updated by Marc Dequènes 10 months ago
I had to update the database scheme s/postgres/postgresql/ in mailman.cfg in database/url because the scheme was deprecated in sqlalchemy. I'm not sure if this is my fault or if the package used it by mistake since it seems this part was generated (we do not own this file in Ansible, only make a small change).
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
- % Done changed from 70 to 90
phppgadmin does not support PG15 in Bookworm, what a shame.
Updated by Marc Dequènes 10 months ago
Orfeo has one disk down:
ioc0 vol_id 2 type IM, 2 phy, 67 GB, state DEGRADED, flags ENABLED ioc0 phy 0 scsi_id 4 SEAGATE ST973402SS S229, 68 GB, state ONLINE, flags NONE ioc0 phy 1 scsi_id 3 SEAGATE ST973401LSUN72G 0556, 68 GB, state MISSING, flags OUT_OF_SYNC
I'll defer rebooting the machine.
Updated by Marc Dequènes 10 months ago
Made some more changes and merged the branch.
Updated by Marc Dequènes 10 months ago
- Description updated (diff)
- Status changed from In Progress to Resolved
- % Done changed from 90 to 100
Actions