DuckCorp » DuckCorp Infrastructure

I use duplicity personally, not sure how it compares to other software you're mentioning. Will check when I have some time.

Here is a comparison of existing backup programs:
https://wiki.archlinux.org/index.php/Synchronization_and_backup_programs

It seems burp does not intent to keep compatibility between major releases which could be a problem when trying to access old backups. Not sure if there is a way to automatically update all of them in an easy way after major releases though...

using the previous experience and the feature chart Arnau linked, I listed the criteria I think we should have.

Arnaud Fontaine wrote:

It seems burp does not intent to keep compatibility between major releases which could be a problem when trying to access old backups. Not sure if there is a way to automatically update all of them in an easy way after major releases though...

Burp version in Jessie is 1.3.48, Burp version in Stretch will be 2.0.54. The are two Burp protocols (protocol 1, protocol 2), the second one is under development and is not stable yet. The default protocol in 2.0.54 is the first one.

At the beginning development of protocol 2, it was possible to switch from protocol 1 to protocol 2, I don't have an up to date information on that subject.

So this this a retranscription of the Bacula settings, but with some adaptations (obsolete paths, new paths, forgotten paths). So we should check this list twice. Also I simplified the retention using 3 classes only. I wonder if we should also save the logs (~9GB total). Also databases could probably go into class 2. All of this is up to discussion.

Class 1: Lightweight critical data, keep everyday for 2 months and weekly for 4 months and monthly for 6 months (1 year total):¶

• /boot
• /etc
• /var/lib/dpkg/available
• /var/lib/dpkg/diversions
• /var/lib/dpkg/statoverride
• /var/lib/dpkg/status
• /var/spool/cron
• /var/backups
• /root
• /usr/local

Additional for Elwing:¶

• /var/lib/opendnssec
• /var/lib/softhsm

Additional for Orfeo:¶

• /var/lib/opendnssec
• /var/lib/softhsm

Additional for Korutopi:¶

• /var/lib/postgresql/zbx_config_backup.sql

Class 2: Lightweight-Heavyweight important data, keep everyday for 2 weeks and weekly for 6 weeks, and monthly for 4 months (6 months total):¶

• /home

Additional for Elwing:¶

• /var/www

Additional for Thorfinn:¶

• /srv/bouncer
• /srv/www

Additional for Orfeo:¶

• /var/lib/minbif
• /var/lib/prosody
• /srv/www
• /var/local/vmail
• /var/lib/mailman
• /var/spool/dspam

Additional for Toushirou:¶

• /srv/vcs
• /srv/www
• /srv/projects
• /var/local/stuffcloud-data

Class 3: Lightweight-Heavyweight less important data, keep everyday for 1 week and weekly for 3 weeks, and monthly for 5 months (6 months total):¶

• /opt

Additional for Elwing:¶

• /usr/local/share/mibs
• /srv/tftp
• /var/lib/smokeping

Additional for Jinta:¶

• /var/games/minetest-server

Additional for Orfeo:¶

• /var/lib/roundcube/plugins
• /usr/local/share/roundcube-plugins/

Additional for Toushirou:¶

• /var/lib/smokeping
• /srv/ftp/ftp.duckcorp.org/private/
• /srv/ftp/ftp.duckcorp.org/public/

• class 1: 30/11
• class 2: 14/3/2
• class 3: 7/3/4

What do you think about this?

As for scheduling I think that's very difficult to find nice ranges, with all the data we have it can take some time and BW. So I think the best is to target the week's best range and limit the BW to an acceptable rate for a fiber connection and the poor I/Os some of the servers can produce.

Marc Dequènes wrote:

So I'll try to roughly translate the retention into Burp's keep parameter.:

• class 1: 30/11
• class 2: 14/3/2
• class 3: 7/3/4

What do you think about this?

Fine.

Translated:

• class 1
  • before: keep everyday for 2 months and weekly for 4 months and monthly for 6 months (1 year total)
  • now: keep everyday for 1 month and monthly for 11 months (1 year total)

• class 2
  • before: keep everyday for 2 weeks and weekly for 6 weeks, and monthly for 4 months (6 months total)
  • now: keep everyday for 2 weeks and every 2 weeks for 6 weeks, and and every month and a half for 3 months (6 months total)
  • now: keep everyday for 2 weeks, and every 2 weeks for 6 weeks, and every 2 months for 2 months (6 months total)

• class 3
  • before: keep everyday for 1 week and weekly for 3 weeks, and monthly for 5 months (6 months total)
  • now: keep everyday for 1 week and weekly for 3 weeks, and every 3 weeks for 3 months (4 months total)

For the scheduling, we will adapt: it depends on backup operation duration. Burps logs every duration, it will be easy to improve scheduling.

There were changes recently, due to Stretch migration and obsolete things, so I updated the directory list above.

• cleaning up the packages, configs, directories… of the previous system
• updating the documentation (https://admin.duckcorp.org/index.php/Services/Backup) and clarify the restoration steps (also what's the best way to restore a database, without writing the full dump locally is possible as some DB might be very big and not fit in our limited space)