Project

General

Profile

OS Upgrade » History » Version 28

Marc Dequènes, 2023-07-11 07:01

1 1 Marc Dequènes
h1. OS Upgrade
2
3
This is a list of steps as guidelines for OS upgrades.
4
5 16 Marc Dequènes
* review APT pinnings in Ansible
6 1 Marc Dequènes
* create a temporary _group_vars/<new-suite>/system.yml_ in a git branch:
7
8
<pre>
9
---
10
system:
11
  distribution:
12
    debian:
13 17 Marc Dequènes
      codename: bookworm
14 1 Marc Dequènes
    dc:
15 17 Marc Dequènes
      codename: bookworm
16 1 Marc Dequènes
17
</pre>
18
19 18 Marc Dequènes
* check if there other places where the old suite name is used and update them
20 3 Marc Dequènes
* (I often choose Elwing first) ansible-playbook --diff -l Elwing -t apt playbooks/common.yml | tee /tmp/dc.log
21 27 Marc Dequènes
* run selected playbooks that update pinnings and the other suite names (run _-t certificates playbooks/common.yml_ for bookworm)
22 24 Marc Dequènes
* _apt upgrade_ and then _apt full-upgrade_ (check if removed packages are problematic):
23 9 Marc Dequènes
** accept new version of:
24
*** /etc/services and copy-paste content of _local services_ from _/etc/services.dpkg-old_ to avoid having to redeploy all services
25
*** /etc/grub.d/10_linux but see warning below
26
*** /etc/ssh/ssh_config (we use ssh_config.d in Ansible now) BUT NOT sshd_config!
27 1 Marc Dequènes
** do not accept new versions for:
28 20 Marc Dequènes
*** /etc/apt-cacher-ng/acng.conf
29 23 Marc Dequènes
*** /etc/apache2/*
30 22 Marc Dequènes
*** /etc/chrony/chrony.conf
31 23 Marc Dequènes
*** /etc/default/grub
32 20 Marc Dequènes
*** /etc/logrotate.d/*
33
*** /etc/oidentd.conf
34 2 Marc Dequènes
*** /etc/smartd.conf
35 1 Marc Dequènes
*** /etc/snmp/snmp.conf
36
*** /etc/sudoers
37 2 Marc Dequènes
*** /etc/ulogd.conf
38 1 Marc Dequènes
** check the diff manually for other files
39 24 Marc Dequènes
* _apt autoremove_
40
* _apt clean_
41 17 Marc Dequènes
* if PHP FPM: (to avoid having to redeploy all vhosts) (example for PHP 7.4->8.2)
42
** rm /etc/php/8.2/fpm/pool.d/www.conf
43
** cp /etc/php/7.4/fpm/pool.d/* /etc/php/8.2/fpm/pool.d/
44
** sed -i 's/7\.4/8.2/g' /etc/php/8.2/fpm/pool.d/*
45
** systemctl restart php8.2-fpm.service
46
** sed -i 's/7\.4/8.2/g' /etc/apache2/sites-enabled/*.conf.d/php.conf
47 13 Marc Dequènes
** systemctl restart apache2
48 17 Marc Dequènes
** purge facts_cache/<host> before running Ansible to detect the new major version
49 10 Marc Dequènes
** run common web playbook _playbooks/tenants/duckcorp/web.yml_ with _-t web-common_
50 19 Marc Dequènes
* run the _playbooks/common.yml_ playbook
51 15 Marc Dequènes
* on MX1 servers, run the _playbooks/tenants/duckcorp/mail.yml_ playbook with _-t antispam_ (to switch the Rspamd repo to the new suite)
52 14 Marc Dequènes
* run the _playbooks/tenants/duckcorp/accounts.yml_ playbook
53 26 Marc Dequènes
* run the _playbooks/common_post.yml_ playbook
54
* run the _playbooks/tenants/duckcorp/monitoring.yml_ playbook
55 25 Marc Dequènes
* run the _playbooks/cleanup.yml_ playbook
56 7 Marc Dequènes
57 1 Marc Dequènes
58 7 Marc Dequènes
It is critical that the common playbook is run successfully before rebooting. Especially _/etc/grub.d/10_linux_ must contain the _--unrestricted_ option and the GRUB config must be regenerated or the server will block at the GRUB screen waiting for a login.
59 14 Marc Dequènes
60
* reboot
61 24 Marc Dequènes
* check failed services: _systemctl --failed_
62 28 Marc Dequènes
63
Later:
64
65
* check _apt-forktracer_ output