New stable DebOps release, v2.3.0

Hello everyone, Another late release, again due to a busy schedule; sorry about that. This time there's nothing ground breaking, just enchancements to existing roles and multiple bugfixes. Due to this, we stay with v2.x.y series. Hopefully the next release around August-September will bring new scripts. New DebOps release, v2.3.0 -------------------------- You can find the new version of DebOps on: GitHub: https://github.com/debops/debops/releases/tag/v2.3.0 PyPI: https://pypi.python.org/pypi/debops/2.3.0 Galaxy: https://galaxy.ansible.com/debops/debops/ You can upgrade the Python-based installation by running the command: pip3 install --upgrade debops Installation instructions can be found here: https://docs.debops.org/en/stable-2.3/introduction/install.html The brief Changelog can also be found on the documentation page: https://docs.debops.org/en/stable-2.3/news/changelog.html Complete, detailed changelog can be viewed using the 'git log' command. You can use the 'git log --no-merges' command to skip the "boring" merge commits. The DebOps documentation has a separate page which details important changes from previous release in the Ansible inventory or on the remote hosts which you might need to perform manually: https://docs.debops.org/en/stable-2.3/news/upgrades.html The Python packages available on PyPI, as well as the tarballs available on GitHub are signed with my GPG key. You can get it from the OpenPGP keyserver network using the command: gpg --keyserver hkp://pool.sks-keyservers.net \ --recv-keys 27067A91D620EE91D50309D92DCCF53E9BC74BEC I wasn't able to finish the new DebOps scripts yet, so the v3.0.0 release is delayed; check the documentation for the new planned release dates. The v2.3.0 development cycle finished with ~364 commits (without merges), Here's a brakdown of committers this time around: 272 Maciej Delmanowski 23 Thomas Blein 16 Robin Schneider 13 Stefan G. Weichinger 11 David Härdeman 8 Imre Jonk 3 Julien Lecomte 3 Jérémy Rosen 3 Nicolas Quiniou-Briand 2 Patryk Ściborek 2 Émile Morel 1 KoS M. Walder 1 Pedro Lucas 1 Pedro Luis López Sánchez 1 Stuart Mumford 1 Thomas Lotze 1 anzil 1 prk0ghy 1 velvetant Thanks to everyone for bugfixes, new functionalities and improvements. See you around in the commits next time! It looks like the 6 months between releases are becoming the norm, so I updated the release schedule. Hopefully this will let me keep the next releases on time. For the curious, at the moment each version takes about half an hour to make (update Changelog, create tags and push them to GitHub, make Python packages and release them on PyPI, make a new release on GitHub, create and publish Ansible Collections on Galaxy), so with 4 supported versions that's about 3 hours of focused work. New stable releases take a bit longer, including time to write the annoucement and setting up new git branches on GitHub, ReadTheDocs and Docker Hub. Goals reached since previous release ------------------------------------ During the latest development cycle I had to put the new scripts on the backburner and work on other things, for example Elasticsearch, Kibana and SimpleSAMLphp (not merged yet). During the summer I will have a bit more free time, so I'm planning to work on the new scrpts then. New upstream releases --------------------- Ansible v4.0.0 has been released, congratulations to the entire team! The changes to the version string required an update in the `debops` script, but apart from that the DebOps playbooks and roles should work with the new release just fine. If you encounter any issues, don't hesitate to create issues in the bug tracker, or send them to the mailing list. Ready-made fixes are not necessary, but are appreciated. New Debian stable release (Bullseye, 11) should be done in the coming weeks. I didn't have time to test DebOps with the new release yet, but if you are planning to do so, please let me know about any issues with the new release. The 'rsnapshot' package has been removed from the next Debian Stable release[1] and it will most likely be dropped from Debian at some point unless the upstream finds new maintainers. It might be a good time to look for a replacement. Borgbackup and Restic seem to be popular candidates, there's also Bacula/Bareos backup solution which is targeted towards larger, distributed installations. The 'autopostgresqlbackup' package will not be included in the Debian Bullseye release due to a bug that hasn't been fixed before the freeze[2]. It might be available via bullseye-backports at some point. For now, the 'postgresql_server' role will not configure automatic snapshots on Debian Bullseye. If you are curious, Michael Prokop has written an excellent blog post[3] about upcoming changes in Debian Bullseye, it's worth the read. A Reddit thread[4] about it also exists, if you're looking for even more news. [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=986709 [2]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956083 [3]: https://michael-prokop.at/blog/2021/05/27/what-to-expect-from-debian-bull... [4]: https://www.reddit.com/r/debian/comments/nmibww/what_to_expect_from_debia... Community highlights -------------------- Due to recent news about FreeNode changing ownership, I created new '#debops' IRC channels on the Libera.Chat[5] and OFTC[6] IRC networks. There's also a Matrix room[7] which will be used to eventually bridge the IRC communities together (currently only FreeNode IRC channel is bridged). [5]: https://libera.chat/ [6]: https://oftc.net/ [7]: https://matrix.to/#/#debops:matrix.org DebOps is now used by the 'ansible-lint' project to test impact of the upcoming lint rule changes against existing Ansible ecosystem.[8] To ensure that DebOps doesn't break the 'ansible-lint' testing pipeline, I changed the 'make lint' test to use 'ansible-lint' in autodiscovery mode, the same one that is used by their CI workflow. [8]: https://github.com/ansible-community/ansible-lint/pull/1500 Julien Lecomte has created a custom Ansible inventory plugin called 'ansible-roster'. While in normal Ansible inventory users create inventory groups and add hosts in them, in 'ansible-roster', users define specific hosts first, and then assign inventory groups to them. The plugin is designed with DebOps in mind and can make management of complicated environments easier. It's available on GitLab[9] as well as through PyPI[10]. [9]: https://gitlab.com/ansible-kheops/plugins/roster [10]: https://pypi.org/project/ansible-roster/ New release highlights ---------------------- As usual, there are many interesting changes and new features in a DebOps release. I suggest that you check out the Changelog of the latest release; below you can find a list of highlights: - DebOps now has the 'extrepo' role which provides support for the "extrepo" third-party APT repository management tool.[11] The "extrepo" database[12] contains details about hundreds of third-party APT repositories and is curated by Debian Developers. Wthin DebOps, "extrepo" is currently used to configure the Elastic APT repositories, used by 'elasticsearch', 'kibana' and 'filebeat' roles; other roles with third-part applications will be converted in the future. - Support for SSH access to hosts was Improved. The 'authorized_keys' role was overhauled and is now focused on managing specific SSH identities on multiple UNIX accounts instead of managing UNIX accounts with multiple SSH keys. The difference is subtle, but noticeable. Check the role documentation for detauls and examples. Additionally, DebOps now officially supports management of hosts using a dedicated UNIX account (for example 'ansible') instead of creating separate UNIX accounts for each system administrator. The 'system_users' role documentation has the details. - The 'dovecot' and 'postldap' roles LDAP support was refreshed and now by default the roles create configuration which utilizes the 'mailservice.schema' LDAP schema included in the 'slapd' role. If you use LDAP for mail management, you should check the new configuration before applying the roles in production environment. [11]: https://packages.debian.org/bullseye/extrepo [12]: https://salsa.debian.org/extrepo-team/extrepo-data Plans for the next development cycle ------------------------------------ In the next release cycle, during Summer, I want to focus on finishing the rewrite of the new DebOps scripts. The gist of the changes is that the new scripts will support multiple Ansible inventories in a single project, with focus on primary "system" inventory for privileged access, and separate "unprivileged" inventories for various applications. This in turn will allow for new set of playbooks and roles focused on unprivileged operations which use Ansible without full 'sudo' privileged access. New scripts will also introduce full support for Ansible Collections; DebOps monorepo will be converted to a Collection format as well. Ansible v4.0.0 brings support for role arguments specification and validation[13]. Adding that support to DebOps roles should significantly improve usability, this will be a second goal in the next release cycle. Most of the specs are already written in the documentation, they just need to be translated into format accepted by Ansible; when the role specs are done they could in turn be used to re-generate documentation using a structured format. [13]: https://docs.ansible.com/ansible-core/devel/user_guide/playbooks_reuse_ro... Take care, Maciej