Re: Check-in

On lip 06, Imre Jonk wrote: Hello! Indeed, I am. :-) The "ginas" name was a backronym, from "ginas is not a server" - since it was meant to manage a cluster from the start, contrary to other such projects being worked on at the time (2013) which focused on everything done on single hosts. But, a better name came along and I switched it. I think that Docker is a great solution in a narrow field of stateless applications, as long as you have everything that is needed underneath Docker for it to work under control. There are of course solutions for that like Docker Machine (more for development), Rancher, now Kubernetes and other orchestration platforms, but these are great when you use them as managed services. When you try to host them yourself, the problems of setting everything from the ground up come back again. And without solid foundation, it's only a matter of time when something breaks under Docker and everything comes down. Building everything properly takes time and effort. Congratulations! :-) This is very impressive! I suspect that you had to write at least dozen additional roles to manage things that are not yet in DebOps itself, but some of that effort could probably be offloaded to already existing roles like nginx, ferm, sysctl, postfix, etc. That's what I was aiming for with the project from the start and I'm very glad that you were able to make it work. I would love to see some of these things in DebOps someday - putting roles upstream will definitely benefit other users and will make the code resilient and easier to maintain when big changes happen. Of course dumping everything at once will probably break some things, and you probably would want to clean up secrets or stuff meant only for your own environment, so it will take time. Still, if you plan to contribute some of your solutions to DebOps, that will be awesome. Did you just found a way to beat https://xkcd.com/1319/? I guess sharing automation solution between organizations will do it... It's great to hear that a project that I started long ago helped you in such a way. This tells me that my decision to make it open by default and work on it essentially "from outside in" to ensure that other people would be able to mold it for their own needs was a correct one. Of course don't forget that DebOps is just essentially an icing and a cherry on a very big cake which is other Open Source and Free Software projects, especially the whole Debian ecosystem, whthout which DebOps probably would not exist today, and Ansible which is THE tool which made such a thing possible in the first place. I'm not sure if DebOps could be recreated using other configuration engines such as Puppet or Saltstack - maybe, maybe not. I was aware pretty early that making DebOps an open project worked on by lots of people from different organizations will eventually require some kind of stewardship over it to keep it going. Usually projects like these end up with a foundation of some sort behind them (Apache Foundation, Mozilla Foundation, GNOME Foundation, I can keep going...), or a separate corporation is formed to perform the role of a steward (Software in the Public Interest, Software Freedom Conservancy). I always thought that at some point DebOps could join the Debian project in some way, not just as a package in the repositories, but as a platform to give users as a stepping stone for "what to do after installing Debian". But I feel that it is still too early for that to happen, there are lots of things to clean up and improve. Some roles haven't been revisited in years due to lack of time or user interest. At the moment DebOps is in a quasi-like state where more and more people start depending on it, but the project itself is not really a legal entity. I'm currently covering the bills for the DNS domain and VPS hosting where various sevices are kept, like this mailing list for example. These costs are manageable right now, so I'm not worried about requiring any donations, especially that this would require me to set up some kind of company to handle taxes in Poland, and other such things I'm currently not in a capacity to handle myself. Various other services like Travis-CI, GitHub, GitLab CI, ReadTheDocs.org provide a free tier which seems to be sufficient for now. DebOps is intentionally designed to be as much self-contained as possible - users are able to set up their own GitLab CI pipeline to faciliate the tests, code and documentation linting performed on Travis-CI for each pull request can be done locally as well after all needed tools are installed (need to make this process easier). I intentionally don't want to depend on any specific SaaS or cloud service to keep the project self-contained and focused on selfhosting. First of all, thanks for starting this thread! I know that I'm really bad at mailing list discussions, I still need to work on some kind of initiative skills... After the mailing list server was upgraded to Mailman 3 I hoped that the rate of discussions here would increase a bit, so it's good to see such an e-mail. I will have to write down some messages around current problems I see in various roles in the project, but usually I prefer to just work on fixing them instead. I'm afraid that this impacts the development culture around DebOps - not much discussion about problems and how to fix them collectively. Perhaps this can be improved in the future. I would like to hear the answers to that question as well! One problem I can definitely see impacting the project is keeping the codebase secure and trustworthy. I tried to solve this problem by limiting the number of people that can merge pull requests to the main repository, requiring (but not enforcing as of yet) signed commits to allow code authorship tracking, carefully reviewing each pull request and trying it out when needed before merging, but I think that this made the development process very slow, some pull requests are waiting years to be resolved. Sorry about that. I'm not really sure how to solve this problem yet. Most likely more people would have to be involved in the review and testing process, with some kind of trusted relationship formed between those who can merge PRs into master. I guess a real life meeting of some sort would be a first step towards that, to estabilish such relationships Web-of-Trust style, similar to how Debian does it. They seem to handle much larger project just fine this way, and probably are a good example to follow. Now that I think about it, DebOps actually is a combination of two things - the existing Ansible and Python codebase which lets you manage infrastructure, and a virtual "blueprint" of an IDEAL infrastructure the project strives to implement - what software to use for various services, the configuration order of different services, what protocols to use, etc. Perhaps such a "blueprint" could help with development and design of DebOps codebase and provide guidelines for role authors to follow. I know that there are some beginnings of it in various parts of the DebOps documentation that describe how the roles should be written, etc. but that's probably not enough to be such a blueprint. I've meant to write an Admin Guide in the docs to describe to how to set up an infrastructure with DebOps but it still hasn't happened yet. Imre, you seem to have done a very good job without it - perhaps a more in-depth guide on how you modeled your infrastructure and what choices you made along the way would be a good starting point? I'm in a position where I have been actively using DebOps at my workplace for several years and essentially doing my work "through it" as much as possible. I definitely don't plan to stop that any time soon - I don't see myself going back to managing Linux infrastructure entirely by hand again, and switching to a different project like this is probably impossible - I don't see any modern alternatives that would handle the scope that I need. So, in essence - I'm stuck with you for the forseeable future. ;-) But if I stop for any reason, I plan to give reins of the project to somebody else, of course. Who will that be, I'm not sure yet. I'll probably look closely at the list of contributors to the project[1], not only at the number of commits but at recent activity as well. Or perhaps over time we can form a larger group of people that can maintain the project as a team. Actually, I was wondering if creating a new mailing list for such people interested in maintaining the project and giving it direction would be a good idea. Say, 'debops-devel'? I'm sure that not every user would be interested in such discussions, this would let them have a choice if they want to keep tabs on them or not. [1]: https://github.com/debops/debops/graphs/contributors It is, a bit by my own design, and this probably creates a bottleneck in the speed of changes and new features impelemnted in DebOps. I'm not sure what pace of development would be expected from such a project; Debian Stable releases happen about every two years and it is considered very slow by Linux community standards. I implemented a four stable releases per year model which kind of fell apart a bit over time, totally by my own hand. I need to better take care of it in the future, but if this model proves unsustainable I'm not against changing it. I guess more discussions about it will be needed. Hmm, maybe a bit. I definitely feel guilty of not handling incoming pull requests or issues on GitHub in a timely manner, I will have to work on that. Other than this, the scope of what the project does really broadened over the years and the development pace slowed down as a result. On each stable release I try to set a few goals for myself for the next one, some of them are made, some not. I definitely try to focus on lower-level stuff, roles that are used in many different places (think firewall, LDAP, various databases) rather than specific user-facing applications so that development efforts can be multiplied over multiple roles at a time. On top of all that is all of the non-Ansible related things and management of the project itself (DNS, various services used for DebOps development, etc.). Of course on top of that is my regular workplace stuff which sometimes needs to take precedence. It's definitely good to hear that your employer is aware that it's not all in your fingertips, but you use open projects for your work. I think that's an excellent example to follow. Currently most of the real-time communications within the project is done on IRC, but there's no logging involved and the format of an IRC conversation is probably not very good for some things. More focus on the mailing lists would probably benefit most current and future DebOps users, especially that we now have proper search UI in Mailman 3. An online meetup is an interesting idea, this would definitely help put some faces to the nicks. I'm up for it if anyone else wants to meet up. We can arrange a time and method later via the mailing list; it should be easy to do using some of the many online services. What I would be interested in is access to more varied infrastructure for development. At the moment I have limited resources at home, and development boxes that are available to me at work are extremely slow (How slow? the common playbook on my home workstation takes about 4 minutes to finish. On the development server at work? 35 minutes.). Due to what I have available, I'm doing my development using LXC containers which introduces a bias in what I'm currently working on. Some things are not possible using containers exclusively, for example NFS infrastructure needs at least VM-level setup since you cannot mount filesystems in unprivileged containers. A VM management platform like OpenNebula requires either hardware-level access or virtualization nesting which introduces speed penality as well (that for example would grind my development boxes to a halt). It would be interesting to have access to some boxes and a subnet, of course if latency over SSH is reasonable. Publicity... I really suck at it. And then there's the mentality on the Net (for example Reddit) where advertising your own work can be considered akin to shilling... So I just don't do it. But if you think that DebOps is worth it, and you have some good ideas on how to do it - be my guest. :-) Let me know if you need some pointers about various aspects of the project. Just be aware that this will bring more users to DebOps which will result in more issue reports and pull requests which might be good or bad - who knows. :) Again, thanks for starting the thread and have a nice day, Maciej