Re: [debops-users] RFC: Merge all DebOps git repositories into one

Some of the subscribers that use the '(a)gmail.com' domain might not have received the previous message due to the mail server using 6to4 tunnel and Google blocking mails over IPv6 without a PTR record. The tunnel is now disabled, so mail delivery to these accounts should work again. Sorry for the inconvenience. On Aug 16, Reto Gantenbein wrote: It's hard to tell what the current donation rate of the roles to DebOps is. In the main organization there are maybe ~10 roles that were developed independently before being added to DebOps. The 'debops-contrib' organization currently has 32 roles, but they are not directly integrated with the main DebOps project and can be considered as maintained separately, just grouped in a GitHub organization. Perhaps if the "conversion rate" for roles moving from 'debops-contrib' to main DebOps organization was higher, the number of roles donated that way would increase. That number is low partially due to the current DebOps Policy that states that each role requires a maintainer, and maintainers need to be DebOps Developers. There are currently two Developers in the project (drybjed, ypid), becoming one requires the use of a GPG key to sign your commits, a period of time while you did that to see if you are consistent, and presence of that GPG key on https://keybase.io/ for authentication purposes. So far nobody directly asked me to become a project Developer (a few people added or tried to add their GPG keys to the project's keyring, as Contributors). From this I can theorize that either the requirements for becoming a DebOps Developer are too steep, or people are perfectly fine with adding their modifications through GitHub pull requests which are then merged by existing project Developers. Becoming a Developer and thus a maintainer of the role in the main 'debops' organization lets you push commits to it directly, merge pull requests and appararently manage all of the organization's roles in Ansible Galaxy because the GitHub authorization is not mapped to Ansible Galaxy authorization. As for the role being "interesting" enough to be published separately on Ansible Galaxy - this process will become highly automated, since the roles would not need to be tagged with GPG signatures (the original code will be signed and present in the main DebOps repository). If demand is high enough, all of the DebOps roles could be published that way on Galaxy using a bot. What will change is the direction the code is sourced from - instead of pulling all of the separate roles together into one directory structure, roles will be extracted from one "source of truth" git repository into separate satellite repositories. On the contrary, with just one repository, you can fork it on GitHub and gain access to everything that way. Your changes could then be pulled to the main repository from your fork. If your fork is popular enough that users start using it instead of the main DebOps repository, you win. There could be even a separate DebOps-next repository where anybody that wants to, has access and commit rights. Then, when a feature is developed enough, it could be cherry-picked and added to the main DebOps repository. In fact that would be the normal way of adding new roles or updating existing ones. I think that existing model of Developers, Maintainers and Contributors could be scrapped altogether - there would only be people that have commit access to the main repository, or not. I think that this ties with the case where multiple roles need to be modified together to work properly (think project-wide firewall changes, or the recent Postfix updates). In the current model, each role requires a separate PR to handle the required changes, in one repository model all the needed changes could be grouped together in one PR, developed over time to encompass everything that's involved in that PR, and committed at once. Looking at issues for individual roles could be handled by properly using labels and/or tagging issue titles. In the current model I stopped using GitHub issue labels altogether because each repository needed to have them set up separately. I guess that this could be handled by writing scripts that manage labels in all DebOps repositories via GitHub API, but that never happened (lack of manpower, I suppose). That still requires a DebOps Developer and role maintainer to tag lots of git repositories and GPG sign their tags. Unless the number of Developers and thus maintainers increase, I don't see that being a feasible model as the number of roles in separate git repositories gets bigger. Isn't each siged git tag a release of a role? Ansible Galaxy certainly uses git tags as versions, and you can use that in your 'requirements.yml' file to select specific role version. Not to mention that this model becomes hard to maintain when different roles are released with different frequency, and the code that is used to pass configuration to other roles becomes desynchronized. For example, what version of the 'debops.nginx' role is compatible with 'debops-contrib.roundcube' role? When 'debops.nginx' role is changed, you might need to update 'debops-contrib.roundcube' role to keep up. Yes, it can be with an increased overhead on the project maintainer to synchronize role versions in 'requirements.yml' file included in the 'debops-playbooks'. This is an external mechanism that could be dropped in favor of using git branches and tags in a single repository. Not to mention that switching from one release to another would require either pinning the role git repositories as submodules to 'debops-playbooks' (not recommended, this was tried with documentation, very slow), modifying the DebOps scripts to checkout each role repository with the desired version, doing something like Nix and deploying each DebOps reelease in a separate subdirectory in the filesystem. Any other approaches to do that? Switching releases using a signle repository is just a 'git checkout' away. Sure, but that doesn't solve the issue of increasing number of separate role git repositories. If that number would coincide with increasing number of people that maintain them, then that could be a solution. That release tarball was already done as 'debops-playbooks' package. The issue Daniel had with packaging 'debops-doc' was that the documentation repository included submodules of all the roles and playbooks which duplicated code a second time and required a massive 'debian/copyright' file to cover everything. Redesigning the documentation model used by DebOps could most likely solve this. Therefore this seems to be a chicken-and-egg situation. You won't be interested in using DebOps as a whole unless a stable release is available, and stable releases are right now unfeasible because of the separate role model, so you stay with the separate role model until a stable release is available... I knew that this might be an important issue for users of separate roles. Consider this - in the new development model, all roles (from main DebOps repository and from 'debops-contrib') would be present in one subdirectory of the repository. In your 'ansible.cfg', you could then include that subdirectory in the 'roles_path' configuration parameter, which would let you access the roles separately in the same way as you do today. They would not be pulled via 'ansible-galaxy', but instead via 'git pull', but you could select the desired stable release by checking out the desired git branch. This is pretty similar to how DebOps roles are now maintained in the '~/.local/share/debops/debops-playbooks/roles/' directory and accessed in Ansible by adding that directory in the 'ansible.cfg' configuration file. Thanks. Going with the current development model is starting to be slower as number of roles increases. I hoped that some of the DebOps users would be interested in the project enough to become its Developers and help with maintaining the roles, but it seems that is not the case. Because of that, I'm looking for a different way to manage the project which might be better in the long run. Ansible Galaxy does not benefit the project apart from use case in testing on Travis, since roles and playbooks are distributed directly from GitHub. I think that changing the way that Ansible Galaxy uses DebOps roles from an active participant to a passive consumer would help DebOps development. Providing stable and consistent releases might help with the project's adoption and increase number of users, which hopefully will convert to a higher number of contributors. Cheers, Maciej