Re: Wrapper roles and interchangeable services

On lip 27, Robin Schneider wrote: It was a few years ago. Since then I learned some new tricks with Ansible, and Ansible itself evolved with 'import_role' task, so that the "wrapper role" in my opinion can be designed better. I think that DebOps is slowly approaching a point where more complex deployments might require more than one non-idempotent Ansible playbook runs to finish configuration. The individual services of course run idempotently as standalone, but various roles rely on Ansible local facts defined by other roles more and more, which means that 2-3 full runs might be required to "stabilize" the final infrastructure. It's a hard problem to solve with a linear deployment strategy we use with Ansible. Something more dynamic and parallel[1] could make the deployments with eventual consistency, when various services "come online" during execution and affect other services. A completely different configuration management system comes to mind, namely mgmt[2]. I wonder how far along the project is from a stable release, and if the developers would consider a configuration scheme as flexible as Ansible so that DebOps could be ported to it. I guess we will see eventually; in the meantime multiple playbook runs that finish in a consistent and idempotent state will have to be the norm. [1]: https://purpleidea.com/blog/2016/01/18/next-generation-configuration-mgmt/ [2]: https://github.com/purpleidea/mgmt No, that's not what I envisioned with the "wrapper roles". They are only something akin to "switches" of sorts that run a specified service role with the dependent variables that role listens to passed on. In a 'nginx' vs 'apache' case, you still will have to define say, 'nginx__dependent_servers' and 'apache__dependent_servers' variables in a role like 'owncloud', with the syntax required by the respective roles. There might be issues with some variables being unacessible between the complimentary roles, so the public interface of a given service role will have to be considered very carefully to work around that. But since DebOps has strict separation of variables between roles and no direct access, I think that we are ready for this. The largest body of DebOps documentation comes from a time where each Ansible role was in its own repository, and its documentation lived in the same git repository. Then in the 'debops/docs' repository we added the role repositories as submodules and combined everything together into a whole. Perhaps if DebOps went with a single repository all the way from the beginning, documentation structure would evolve to have common sections shared between similar roles, and more unique roles with their own custom documentation would be an exception. I'm not really sure though - over the years I learned that each puzzle in the Linux ecosystem has its own idea of configuration models, what parameters are needed, etc. Usually what is common is only the file format that is used, be it YAML, INI, or something similar. And describing an INI configuration format is usually not enough to tell how a particular application utilizes it for its own needs. I think that this level of documentation (per-role details) always will be needed due to the above issue. But that's only a low level thing, which DebOps has to do anyway. The result is a more universal YAML-based configuration format designed around Ansible inventory structure. What we need is a higher-level documentation based on that format, which looks at the whole infrastructure from a holistic point of view. Essentially a DebOps equivalent of Debian Handbook[3]. At this level, documentation could focus on presenting similarities in various applications and how to configure them using DebOps. But, I'm pretty busy working on the lower level right now. Somebody else would probably have to start working on a higher level documentation; but we talked about this with the idea of "blueprints" in the other thread some time ago. [3]: https://debian-handbook.info/ I envision that the "infrastructure as code" solution such as DebOps should encode as much automation as possible between different low level operating system services. I don't really want to think about low-level details each time I set up a new service anymore. In the container-based world, it seems that Kubernetes Operators[4] seem to be a solution for this problem. We made pretty good progress with Ansible on the standalone host so far via Ansible local facts, but there's still some way to go in a cluster environment. In this case, something like SaltStack's Event System[5] really starts to look seductive... I wonder if we can find a lightweight alternative without all the other configuration-related baggage that could be co-opted for use with Ansible. Probably some kind of universal RPC service that could be used to execute authenticated and authorized commands. Pub/Sub support would bring more flexibility as well. [4]: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/ [5]: https://docs.saltstack.com/en/master/topics/event/events.html Great. So I'll look into implementing something, probably starting with NTP and see how it works in practice. This probably calls for a survey among DebOps users to see who uses what. LimeSurvey[4] role could probably come in handy, and it seems to be a popular application for this kind of thing. I'll add it to the todo list. [4]: https://www.limesurvey.org/en/ -- Maciej