New DebOps stable release - v2.0.0
by Maciej Delmanowski
Hello everyone,
It's 2020, the time has come to release DebOps v2.0.0!
New DebOps release, v2.0.0
--------------------------
You can find the new version of DebOps on:
GitHub: https://github.com/debops/debops/releases/tag/v2.0.0
PyPI: https://pypi.python.org/pypi/debops/2.0.0
Galaxy: https://galaxy.ansible.com/debops/debops/
You can upgrade the Python-based installation by running the command:
pip3 install --upgrade debops
The DebOps v2.0.0 release should have full support for Ansible Collections. If
you plan to use DebOps this way, let me know if there are any issues.
Installation instructions can be found here:
https://docs.debops.org/en/stable-2.0/introduction/install.html
The brief Changelog can also be found on the documentation page:
https://docs.debops.org/en/stable-2.0/news/changelog.html
Complete, detailed changelog can be viewed using the 'git log' command. You can
use the 'git log --no-merges' command to skip the "boring" merge commits.
The DebOps documentation has a separate page which details important changes
from previous release in the Ansible inventory or on the remote hosts which
you might need to perform manually:
https://docs.debops.org/en/stable-2.0/news/upgrades.html
The Python packages available on PyPI, as well as the tarballs available on
GitHub are signed with my GPG key. You can get it from the OpenPGP keyserver
network using the command:
gpg --keyserver hkp://pool.sks-keyservers.net \
--recv-keys 27067A91D620EE91D50309D92DCCF53E9BC74BEC
This development cycle was shorter, only 2 months instead of the usual three.
The focus was more on cleanup and improvements in Ansible Galaxy support, so
the number of commits since the v1.2.0 release is pretty small, only ~394.
Here's the breakdown of the committers in the v2.0.0 release, without merges:
293 Maciej Delmanowski
50 Rainer 'rei' Schuth
26 Robin Schneider
7 Sandro Jäckel
6 Imre Jonk
4 Gaudenz Steinlin
3 Alin Alexandru
2 Anton Nesterov
1 Andreas Palm
1 Andrew Rowson
1 Pedro Luis López Sánchez
Thanks to everyone involved for helping shape up this project, and see you in
the commits.
Goals reached since previous release
------------------------------------
At the v1.2.0 release I decided to focus on cleaning up DebOps codebase and
improving support for Ansible Collections[1]. How did that turn out? A quick
round-up:
- Removal of the [debops_*] inventory groups from playbooks - done.
- Migration of hard role dependencies to 'import_role' tasks - done.
- Cleanup of old non-namespaced Ansible tags - done, I think. There might be
some stragglers.
- Switch to using 'dpkg_divert' Ansible module for diverting/revering files
- not done, unfortunately. Perhaps in the next release cycle.
- Clean up of 'lookup("password")' paths inside of the 'secret/' directory
- done, I think. If you notice any passwords or other data in directories
named after host FQDNs when you don't use them in the inventory, let me
know.
- Cleanup of Ansible local facts - not done as well. This will probably take
some more time.
- Conversion of 'script' tasks into Ansible modules - also not done yet.
- Update of upstream releases of various applications - not done. I'll
probably try and do it early in the v2.1.0 release cycle though.
So, as usual not all of the planned things were achieved, but at least there's
some progress. :-) I wonder how the next release cycle will work out.
[1]: https://lists.debops.org/pipermail/debops-users/2019-December/000247.html
Fixed Ansible Collection is finally here
----------------------------------------
The major version number change was done due to the improved Ansible
Collection support. All of the supported DebOps roles were renamed to make
them compatible with Collections, the existing playbooks also gained
Collection support and can be used with all versions of the DebOps
installation methods now. The roles included in the Collection are slightly
modified to allow use of lookup and filter plugins inside the collection but
otherwise they should work the same from any of the installation methods
- Galaxy, Python package or git monorepo. And hopefully your data center will
not change into an active supervolcano in the process.
If you plan to use DebOps roles via Ansible Collection and find any issues
with them, let me know. I would also like to know how many people plan to use
each installation method (Python package, Ansible Collection, git monorepo),
so that project development could be effectively focused in the future.
One debatable issue is use of third-party Ansible Collections within DebOps
itself - for example there's a NetBox community collection[2] that contains
various Ansible modules which interface with NetBox. Using them inside DebOps
roles would be problematic for git monorepo or Python package users, but an
alternative would be somehow vendoring them inside of the DebOps monorepo. I'm
not sure about the best solution for that, and I'd like to know your opinion.
[2]: https://galaxy.ansible.com/netbox_community/ansible_modules
Switching to manual
-------------------
I've had an annoying issue for a long time while using DebOps on my
infrastructure - when I was writing configuration in the inventory variables
I always had to go to the web documentation and look up a role to find the
required syntax of a particular variable. Using the 'debops-defaults' command
in this case is usually not enough because it shows empty list variables most
of the time. An alternative would be to find the documentation page in the git
repository which is also tedious. So I decided to try the classic UNIX method
and generate man(1) pages for each DebOps role from the existing
documentation using Sphinx manual builder.
And it worked! Somewhat, at least. The manual sections are in a rather awkward
order (and I had to reorder the online documentation pages as well to make
manpages more sane) and the default variables of the roles are not included
due to horrible results from the converted YAML files, but most important
sections are there. The manual pages are included out of the box in the DebOps
Python package - after installing it systemwide (not sure about the virtualenv
install) you can run 'man debops.postfix' command to see the manual for the
'postfix' DebOps role. The other manual pages are named in a similar format,
'debops.<role>'.
I would like to improve the manual pages of the roles in the future, but I'm
not sure about the impact on online documentation. Hopefully as long as the
section order and contents are consistent for all roles, any changes to them
will be acceptable. Some more manual pages will probably also show up, for
example manuals for the various 'debops*' scripts.
Plans for the next development cycle
------------------------------------
Because during last two months DebOps development was focused mostly on
existing roles and cleanup of the project, for the next three months I would
like to include as much new Ansible roles as possible. There are a few of
these lingering in the GitHub pull requests - if you have a role waiting for
a merge, check if it still works and rename it if necessary. When I have some
time, I'll try to take some of the pull requests, update them as needed and
merge into DebOps. There are also still old debops-contrib roles which need to
be renamed and refreshed, so I'll try to tackle at least a few of them before
next release.
I'm working on designing a concept of a "wrapper role", that would allow
switching between different base roles without the need to change the
playbooks. Currently in my sights I have a 'firewall' role which would support
'ferm' and in the future 'firewalld' roles, as well as a revamped 'ntp' role
which would allow switching between different NTP service roles like
'openntpd', 'chrony', 'timesyncd' and so on. Perhaps a 'webserver' role could
support switching between 'nginx' and 'apache' roles, whenever they are
finally refreshed. The method that allows this unfortunately isn't very
flexible - roles have to be switched globally for all hosts in the inventory
via newly implemented "global variables" - but at least it's possible.
I'm also refreshing the 'mailman' role to finally switch it to support Mailman 3,
with w forum-like web interface. This should make the DebOps mailing lists
a bit more user-friendly and easier to participate in.
I hope that I'll finally manage to tackle the rest of the ELK stack, namely
Logstash and Filebeat roles, to better support logging. The Elastcicsearch and
Kibana roles might accidentally be refreshed in the process as well, we will
see.
So, that would be it for now. Have a nice 2020, and see you in the commits.
Maciej
4 years, 2 months
