Some of the subscribers that use the '(a)gmail.com' domain might not have
received the previous message due to the mail server using 6to4 tunnel and
Google blocking mails over IPv6 without a PTR record. The tunnel is now
disabled, so mail delivery to these accounts should work again. Sorry for the
On Aug 16, Reto Gantenbein wrote:
On Tue, 2017-08-15 at 20:36 +0200, Maciej Delmanowski wrote:
> I'm not saying that all of the DebOps roles should be wiped from Ansible
> Galaxy completely. The current project model can be summarized as "create
> roles separately and then combine them in a bundle". What I'm proposing to
> do is reverse that into something like "create a consistent bundle of roles
> and expose the interesting ones on Ansible Galaxy".
What's the difference between a DebOps Ansible role and an Ansible
role. Except a few DebOps-specific modules it's mainly conventions.
That said, I see the development approach of DebOps more like the
nowadays often discussed microservice architecture. There are
individual pieces each one solving a specific task and respecting some
conventions, but each can be developed separately even by different
teams. The same approach is also written down in the DebOps policy
about organizational structure. Changing this to a centralized approach
would in my opinion:
- Lower the chance that 3rd-party developers would want to "donate"
their roles to DebOps as this would reduce their control and they would
need to hope that their role is an "interesting" one or they develop
and keep it separately to control what is on Galaxy.
It's hard to tell what the current donation rate of the roles to DebOps is. In
the main organization there are maybe ~10 roles that were developed
independently before being added to DebOps. The 'debops-contrib' organization
currently has 32 roles, but they are not directly integrated with the main
DebOps project and can be considered as maintained separately, just grouped in
a GitHub organization. Perhaps if the "conversion rate" for roles moving from
'debops-contrib' to main DebOps organization was higher, the number of roles
donated that way would increase.
That number is low partially due to the current DebOps Policy that states that
each role requires a maintainer, and maintainers need to be DebOps Developers.
There are currently two Developers in the project (drybjed, ypid), becoming
one requires the use of a GPG key to sign your commits, a period of time while
you did that to see if you are consistent, and presence of that GPG key on
for authentication purposes.
So far nobody directly asked me to become a project Developer (a few people
added or tried to add their GPG keys to the project's keyring, as
Contributors). From this I can theorize that either the requirements for
becoming a DebOps Developer are too steep, or people are perfectly fine with
adding their modifications through GitHub pull requests which are then merged
by existing project Developers. Becoming a Developer and thus a maintainer of
the role in the main 'debops' organization lets you push commits to it
directly, merge pull requests and appararently manage all of the
organization's roles in Ansible Galaxy because the GitHub authorization is not
mapped to Ansible Galaxy authorization.
As for the role being "interesting" enough to be published separately on
Ansible Galaxy - this process will become highly automated, since the roles
would not need to be tagged with GPG signatures (the original code will be
signed and present in the main DebOps repository). If demand is high enough,
all of the DebOps roles could be published that way on Galaxy using a bot.
What will change is the direction the code is sourced from - instead of
pulling all of the separate roles together into one directory structure, roles
will be extracted from one "source of truth" git repository into separate
- The role maintainer model has to be given up (at least privilege
wise). Either one is a main developer of DebOps and has access to
everything or one has no rights.
On the contrary, with just one repository, you can fork it on GitHub and gain
access to everything that way. Your changes could then be pulled to the main
repository from your fork. If your fork is popular enough that users start
using it instead of the main DebOps repository, you win.
There could be even a separate DebOps-next repository where anybody that wants
to, has access and commit rights. Then, when a feature is developed enough, it
could be cherry-picked and added to the main DebOps repository.
In fact that would be the normal way of adding new roles or updating existing
ones. I think that existing model of Developers, Maintainers and Contributors
could be scrapped altogether - there would only be people that have commit
access to the main repository, or not.
- It's harder to follow development and see issues of individual
as everything will be mixed together.
I think that this ties with the case where multiple roles need to be modified
together to work properly (think project-wide firewall changes, or the recent
Postfix updates). In the current model, each role requires a separate PR to
handle the required changes, in one repository model all the needed changes
could be grouped together in one PR, developed over time to encompass
everything that's involved in that PR, and committed at once.
Looking at issues for individual roles could be handled by properly using
labels and/or tagging issue titles. In the current model I stopped using
GitHub issue labels altogether because each repository needed to have them set
up separately. I guess that this could be handled by writing scripts that
manage labels in all DebOps repositories via GitHub API, but that never
happened (lack of manpower, I suppose).
> In the curren state this "freeze" would need to be
handled in the
> 'debops-playbooks' repository. Since the roles in separate git
> repositories, only mechanism that is valid to use to freeze on specific
> role versions is Ansible Galaxy 'requirements.yml' file, which would
> specify all of the roles and the desired versions.
In the current state yes. However, I would understand a freeze with the
current layout to be a tarball of a repository state with a version and
each repository tagged with this version. For stabilizing, there can be
a branch of this version where bug fixes can be applied. Obviously the
requirements.yml file can follow whatever versions it wants to.
That still requires a DebOps Developer and role maintainer to tag lots of git
repositories and GPG sign their tags. Unless the number of Developers and thus
maintainers increase, I don't see that being a feasible model as the number of
roles in separate git repositories gets bigger.
> Let's say that DebOps offers two "stable" branches,
1.x and 2.x, each one
> with a frozen set of roles. there's also separate 'master' branch which
> gets updated before being frozen. Now, a 'debops.nginx' role has some
> updates and a new version is released, now that version needs to be updated
> in the 'debops-playbooks' 'master' branch. And then
> 'debops.gitlab', etc., etc. Each release of a role requires a parallel
> update in 'debops-playbooks'. Note that right now an update in the playbook
> repository is needed only when a new role is added, or a playbook of an
> existing role is updated; otherwise 'debops-playbooks' is not changed all
> that much.
There is nothing like individual releases of roles.
Isn't each siged git tag a release of a role? Ansible Galaxy certainly uses
git tags as versions, and you can use that in your 'requirements.yml' file to
select specific role version. Not to mention that this model becomes hard to
maintain when different roles are released with different frequency, and the
code that is used to pass configuration to other roles becomes desynchronized.
For example, what version of the 'debops.nginx' role is compatible with
'debops-contrib.roundcube' role? When 'debops.nginx' role is changed, you
might need to update 'debops-contrib.roundcube' role to keep up.
Even it might contradict a bit to my distributed approach that I
above, but a release is primarily a DebOps release. It still can be a all or
nothing approach even with multiple repositories.
Yes, it can be with an increased overhead on the project maintainer to
synchronize role versions in 'requirements.yml' file included in the
'debops-playbooks'. This is an external mechanism that could be dropped in
favor of using git branches and tags in a single repository. Not to mention
that switching from one release to another would require either pinning the
role git repositories as submodules to 'debops-playbooks' (not recommended,
this was tried with documentation, very slow), modifying the DebOps scripts to
checkout each role repository with the desired version, doing something like
Nix and deploying each DebOps reelease in a separate subdirectory in the
filesystem. Any other approaches to do that? Switching releases using a signle
repository is just a 'git checkout' away.
> Without changing the development model of the project and
> "freezes" like the above, that would put even more pressure on the project
> maintainers and add even more work. Can this be automated? Not without
> automating commit signing...
Maybe a "simple" script to do the relevant part which is executed by
the project leader would be sufficient? Close the CHANGES.rst and
update the requirements.yml?
Sure, but that doesn't solve the issue of increasing number of separate role
git repositories. If that number would coincide with increasing number of
people that maintain them, then that could be a solution.
> There was an attempt to do this by a Debian Developer a while ago
> the relevant Debian bugs in my initial post). Unfortunately this fell apart
> because of how the documentation is currently managed in DebOps via git
> submodules - this resulted in lots of duplicated role code in the
> 'debops-doc' package which couldn't be passed through Debian lintian.
> Keeping everything in one repository wouldn't have that issue and
> a 'debian/' directory with build instructions could be included as well.
Yes, I'm aware of this and I can feel with him as I'm doing a lot of
packaging work too. I feel the main issue was, that he tried to build
the package out of git directly which is a pain because of the
distributed repositories. I think a lot could be simplified if there is
a release tarball as mentioned above which can be used as basis.
That release tarball was already done as 'debops-playbooks' package. The issue
Daniel had with packaging 'debops-doc' was that the documentation repository
included submodules of all the roles and playbooks which duplicated code
a second time and required a massive 'debian/copyright' file to cover
everything. Redesigning the documentation model used by DebOps could most
likely solve this.
As you can see, I'm a user of individual roles and a contributor
individual roles (mostly in contrib). The entire DebOps might become
interesting to me when there is something like a stable release that is
well tested. I'm happy to support you in trying to achieve this but I
only have a few hours per week that I can spend on this. So for the
time being, I use the roles separately in the same way as other Ansible
Therefore this seems to be a chicken-and-egg situation. You won't be
interested in using DebOps as a whole unless a stable release is available,
and stable releases are right now unfeasible because of the separate role
model, so you stay with the separate role model until a stable release is
I knew that this might be an important issue for users of separate roles.
Consider this - in the new development model, all roles (from main DebOps
repository and from 'debops-contrib') would be present in one subdirectory of
the repository. In your 'ansible.cfg', you could then include that
subdirectory in the 'roles_path' configuration parameter, which would let you
access the roles separately in the same way as you do today. They would not be
pulled via 'ansible-galaxy', but instead via 'git pull', but you could
the desired stable release by checking out the desired git branch. This is
pretty similar to how DebOps roles are now maintained in the
'~/.local/share/debops/debops-playbooks/roles/' directory and accessed in
Ansible by adding that directory in the 'ansible.cfg' configuration file.
At the end you are the project leader and DebOps is where it is
because of all your hard work and your decisions. Just keep going...
Thanks. Going with the current development model is starting to be slower as
number of roles increases. I hoped that some of the DebOps users would be
interested in the project enough to become its Developers and help with
maintaining the roles, but it seems that is not the case. Because of that, I'm
looking for a different way to manage the project which might be better in the
Ansible Galaxy does not benefit the project apart from use case in testing on
Travis, since roles and playbooks are distributed directly from GitHub.
I think that changing the way that Ansible Galaxy uses DebOps roles from an
active participant to a passive consumer would help DebOps development.
Providing stable and consistent releases might help with the project's
adoption and increase number of users, which hopefully will convert to
a higher number of contributors.