Project

General

Profile

Enhancement #563

Let's be an Openinfra!!!

Added by Marc Dequènes almost 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Category:
Org :: Admin
Start date:
2017-06-22
Due date:
% Done:

100%

Patch Available:
Confirmed:
No
Branch:
prepare_vault
Entity:
DuckCorp
Security:
Help Needed:

Description

So here is a list of files containing secrets:

  • ansible/group_vars/shell_servers/vars.yml
  • pki/keys/*/*.key
  • doc/accounts.md

Please check if I forgot anything.

We have git history with secrets and I think it would be too risky to rely on history filtering. Also we have signed commits so it is most probably not possible anyway. So I would suggest creating a new repo, public this time, and use Ansible Vault because it is simple (see https://gitlab.com/osas/community-cage-infra-ansible/commit/f6c9f8ab54169b78ef8e97d20a8b0b95ab5a0ca0).

Also admin.duckcorp.org should be open too. Content needs to be checked of course. I think we should put the documentation inside the new admin repo and commit doc updates along code changes. Also we should generate it. We could use Nanoc, try Jekyll…

As for the content which is very sensitive and not needed for Ansible and tools, we could just keep the current repo. For example doc/accounts.md could stay inside. Security sensitive content from admin.duckcorp.org if any could sit here. Also I was thinking about sensitive user-related topics, like the list of anonymous donors or private information we could not discuss on the ML…

[informative but irrelevant to the work in this BR] Now that Elwing is less and less involved in DC proper, I would separate DC and DL admin. With shared roles we can still make it simple and it is easier to delegate just one or the other. Daneel, when resurrected, would then follow the same path. In case we need to host services inside DL, then we could use a LXC or VM.

I think this is something we can do right now, even if we create repos for the current roles and phase them out later for better version. We don't need to work on a doc builder right know. I think this would avoid loosing more history.

So let's discuss this and polish details if we agree on something.


Related issues

Related to DuckCorp Infrastructure - Enhancement #599: Use letsencrypt for public-facing websites… maybe more Resolved 2017-09-26

History

#1 Updated by Marc Dequènes almost 2 years ago

  • Description updated (diff)

#2 Updated by Marc Dequènes almost 2 years ago

I would also suggest to open the bug tracker to the public. We could review bugs for sensitive information and either hide them by editing the content if it does not destroy the BR's meaning (like using <this-kind-of-data> instead of the real sensitive data…) or use the private flag in Redmine if not possible or too complicated. We should not take chances so using the flag more/only would be ok. I don't think there is much sensitive content anyway.

I still have the idealistic hope people could contribute, but at least it would make sysadmin work more transparent and also clarify how much energy we put in it.

#3 Updated by Marc Dequènes over 1 year ago

We could then close the `Ducklings Volunteer Activities` initiative, which was honestly a big failure. We might still decide to tag some easy bugs if anyone is willing to help (Redmine does not support tags but a searchable custom field should do the trick).

#4 Updated by Marc Dequènes over 1 year ago

  • Status changed from New to In Progress
  • Branch set to prepare_vault

I have checked the tickets and flagged private all the sensitive ones.

In the `prepare_vault` branch I've been preparing for the repository split by protecting sensitive data:
  • users and entities files contains passwords, email addresses and personal data, thus have been fully encrypted
  • files in `host_vars` and `group_vars` have been either fully encrypted or split when they contain sensitive data
  • PKI secret materials are fully encrypted and the `mkcert` tool has been patched to add Ansible Vault support (see #597)
  • other files, especially older in-repo Ansible roles, have been checked (no changes needed)

All encrypted files have git attributes. All YAML encrypted files are suffixed by `.vault.yml`. The key is stored in the `duckcorp-infra.vault_pass` file and can be used easily by setting the `ANSIBLE_VAULT_PASSWORD_FILE` environment variable.

All files in the current repository are meant to be copied over in a brand new `duckcorp-infra` repo except for:
  • participants
  • duckcorp-infra.vault_pass
  • doc/accounts.md

I already improved the documentation of the Ansible part. I plan to do the same for the top directory.

Please review so I can go ahead with opening the infra materials and tickets.

#5 Updated by Marc Dequènes over 1 year ago

As for the documentation we could use the website. All the user doc would fit. The admin could would probably be transfered in the `duckcorp-infra` repo and does not necessarily needs to be generated. Also with time I hope to replace all manual operations using Ansible, so only debug and rescue tips, and more textual description of the infra should stay.

Btw I just realized the website's repo is hidden. I did not see any sensitive content. I will have another look later and open it if no problem. The tickets in the Redmine subproject does not contain any sensitive material and could be opened alongside the admin project.

#6 Updated by Marc Dequènes over 1 year ago

  • % Done changed from 0 to 30

I reworked roles and workflows in Redmine to clarify and simplify. Also checked the new ACLs and removed the obsolete Worker (and not very pleasantly named) role.

I opened the Admin and website projects.

I created the new repo with the finished doc, moved the content, and added some doc in the old repo too.

#7 Updated by Marc Dequènes over 1 year ago

  • Assignee changed from DC Admins to Marc Dequènes
  • % Done changed from 30 to 40

I clarified the tooling around Ansible Vault.

I phazed out the Ducklings Volunteer Activities project and moved back related tickets in this project.

I moved the website repository in the open. I also adapted repositories permissions, but we should improve the overall system to ease contibutions (see #158).

I renamed this very project, improved documentation, and began moving non-obsolete content from the Admin wiki in here.

#8 Updated by Marc Dequènes over 1 year ago

I am checking users.duckcorp.org:
  • fixed some obsolete information
  • the architecture diagrams are broken (problem with the new MW plugin for graphviz); maybe we could just forget about this and link to the Supervision website (when recreated) as it contains diagrams
  • I need to check the services pages for sensitive information

When this is done we can open the site.

#9 Updated by Marc Dequènes over 1 year ago

  • % Done changed from 40 to 50

I finished checking users.duckcorp.org, and fixed a few more obsolete information. The only sensitive point was the first-time password for Minetest, which has been changed; users would have to ask an administrator for it. The site is now readable by everyone, edition is limited as before.

#10 Updated by Marc Dequènes over 1 year ago

  • Description updated (diff)

#11 Updated by Marc Dequènes over 1 year ago

To clarify the remaining tasks:
  • phase-out the admin wiki and move non-obsolete information elsewhere (this project's wiki seems appropriate)
  • add Let's Encrypt support for the following sites (to really enable external users to easily access the opened data):
    • www.duckcorp.org
    • projects.duckcorp.org
    • users.duckcorp.org
    • vcs.duckcorp.org
    • vcs-git.duckcorp.org
    • vcs-git-viewer.duckcorp.org
    • repository.duckcorp.org
    • lists.duckcorp.org (for the public user list)

#12 Updated by Marc Dequènes over 1 year ago

  • % Done changed from 50 to 60
I already did some work on the admin wiki, most notably:

#13 Updated by Marc Dequènes over 1 year ago

  • % Done changed from 60 to 70

The listed sites now use Let's Encrypt.

#14 Updated by Marc Dequènes over 1 year ago

  • Related to Enhancement #599: Use letsencrypt for public-facing websites… maybe more added

#15 Updated by Marc Dequènes about 1 year ago

  • % Done changed from 70 to 90

The old admin wiki is almost empty now. Documentation is dispatched, based on use, on the user wiki, in the Ansible rules, and in the new Procedures / Notes section on this project's wiki.

#16 Updated by Marc Dequènes about 1 year ago

  • Status changed from In Progress to Resolved
  • % Done changed from 90 to 100

All DONE!!!

Also available in: Atom PDF