On lip 06, Imre Jonk wrote: Hello! Indeed, I am. :-) The "ginas" name was a backronym, from "ginas is not a server" - since it was meant to manage a cluster from the start, contrary to other such projects being worked on at the time (2013) which focused on everything done on single hosts. But, a better name came along and I switched it. I think that Docker is a great solution in a narrow field of stateless applications, as long as you have everything that is needed underneath Docker for it to work under control. There are of course solutions for that like Docker Machine (more for development), Rancher, now Kubernetes and other orchestration platforms, but these are great when you use them as managed services. When you try to host them yourself, the problems of setting everything from the ground up come back again. And without solid foundation, it's only a matter of time when something breaks under Docker and everything comes down. Building everything properly takes time and effort. Congratulations! :-) This is very impressive! I suspect that you had to write at least dozen additional roles to manage things that are not yet in DebOps itself, but some of that effort could probably be offloaded to already existing roles like nginx, ferm, sysctl, postfix, etc. That's what I was aiming for with the project from the start and I'm very glad that you were able to make it work. I would love to see some of these things in DebOps someday - putting roles upstream will definitely benefit other users and will make the code resilient and easier to maintain when big changes happen. Of course dumping everything at once will probably break some things, and you probably would want to clean up secrets or stuff meant only for your own environment, so it will take time. Still, if you plan to contribute some of your solutions to DebOps, that will be awesome. Did you just found a way to beat https://xkcd.com/1319/? I guess sharing automation solution between organizations will do it... It's great to hear that a project that I started long ago helped you in such a way. This tells me that my decision to make it open by default and work on it essentially "from outside in" to ensure that other people would be able to mold it for their own needs was a correct one. Of course don't forget that DebOps is just essentially an icing and a cherry on a very big cake which is other Open Source and Free Software projects, especially the whole Debian ecosystem, whthout which DebOps probably would not exist today, and Ansible which is THE tool which made such a thing possible in the first place. I'm not sure if DebOps could be recreated using other configuration engines such as Puppet or Saltstack - maybe, maybe not. I was aware pretty early that making DebOps an open project worked on by lots of people from different organizations will eventually require some kind of stewardship over it to keep it going. Usually projects like these end up with a foundation of some sort behind them (Apache Foundation, Mozilla Foundation, GNOME Foundation, I can keep going...), or a separate corporation is formed to perform the role of a steward (Software in the Public Interest, Software Freedom Conservancy). I always thought that at some point DebOps could join the Debian project in some way, not just as a package in the repositories, but as a platform to give users as a stepping stone for "what to do after installing Debian". But I feel that it is still too early for that to happen, there are lots of things to clean up and improve. Some roles haven't been revisited in years due to lack of time or user interest. At the moment DebOps is in a quasi-like state where more and more people start depending on it, but the project itself is not really a legal entity. I'm currently covering the bills for the DNS domain and VPS hosting where various sevices are kept, like this mailing list for example. These costs are manageable right now, so I'm not worried about requiring any donations, especially that this would require me to set up some kind of company to handle taxes in Poland, and other such things I'm currently not in a capacity to handle myself. Various other services like Travis-CI, GitHub, GitLab CI, ReadTheDocs.org provide a free tier which seems to be sufficient for now. DebOps is intentionally designed to be as much self-contained as possible - users are able to set up their own GitLab CI pipeline to faciliate the tests, code and documentation linting performed on Travis-CI for each pull request can be done locally as well after all needed tools are installed (need to make this process easier). I intentionally don't want to depend on any specific SaaS or cloud service to keep the project self-contained and focused on selfhosting. First of all, thanks for starting this thread! I know that I'm really bad at mailing list discussions, I still need to work on some kind of initiative skills... After the mailing list server was upgraded to Mailman 3 I hoped that the rate of discussions here would increase a bit, so it's good to see such an e-mail. I will have to write down some messages around current problems I see in various roles in the project, but usually I prefer to just work on fixing them instead. I'm afraid that this impacts the development culture around DebOps - not much discussion about problems and how to fix them collectively. Perhaps this can be improved in the future. I would like to hear the answers to that question as well! One problem I can definitely see impacting the project is keeping the codebase secure and trustworthy. I tried to solve this problem by limiting the number of people that can merge pull requests to the main repository, requiring (but not enforcing as of yet) signed commits to allow code authorship tracking, carefully reviewing each pull request and trying it out when needed before merging, but I think that this made the development process very slow, some pull requests are waiting years to be resolved. Sorry about that. I'm not really sure how to solve this problem yet. Most likely more people would have to be involved in the review and testing process, with some kind of trusted relationship formed between those who can merge PRs into master. I guess a real life meeting of some sort would be a first step towards that, to estabilish such relationships Web-of-Trust style, similar to how Debian does it. They seem to handle much larger project just fine this way, and probably are a good example to follow. Now that I think about it, DebOps actually is a combination of two things - the existing Ansible and Python codebase which lets you manage infrastructure, and a virtual "blueprint" of an IDEAL infrastructure the project strives to implement - what software to use for various services, the configuration order of different services, what protocols to use, etc. Perhaps such a "blueprint" could help with development and design of DebOps codebase and provide guidelines for role authors to follow. I know that there are some beginnings of it in various parts of the DebOps documentation that describe how the roles should be written, etc. but that's probably not enough to be such a blueprint. I've meant to write an Admin Guide in the docs to describe to how to set up an infrastructure with DebOps but it still hasn't happened yet. Imre, you seem to have done a very good job without it - perhaps a more in-depth guide on how you modeled your infrastructure and what choices you made along the way would be a good starting point? I'm in a position where I have been actively using DebOps at my workplace for several years and essentially doing my work "through it" as much as possible. I definitely don't plan to stop that any time soon - I don't see myself going back to managing Linux infrastructure entirely by hand again, and switching to a different project like this is probably impossible - I don't see any modern alternatives that would handle the scope that I need. So, in essence - I'm stuck with you for the forseeable future. ;-) But if I stop for any reason, I plan to give reins of the project to somebody else, of course. Who will that be, I'm not sure yet. I'll probably look closely at the list of contributors to the project[1], not only at the number of commits but at recent activity as well. Or perhaps over time we can form a larger group of people that can maintain the project as a team. Actually, I was wondering if creating a new mailing list for such people interested in maintaining the project and giving it direction would be a good idea. Say, 'debops-devel'? I'm sure that not every user would be interested in such discussions, this would let them have a choice if they want to keep tabs on them or not. [1]: https://github.com/debops/debops/graphs/contributors It is, a bit by my own design, and this probably creates a bottleneck in the speed of changes and new features impelemnted in DebOps. I'm not sure what pace of development would be expected from such a project; Debian Stable releases happen about every two years and it is considered very slow by Linux community standards. I implemented a four stable releases per year model which kind of fell apart a bit over time, totally by my own hand. I need to better take care of it in the future, but if this model proves unsustainable I'm not against changing it. I guess more discussions about it will be needed. Hmm, maybe a bit. I definitely feel guilty of not handling incoming pull requests or issues on GitHub in a timely manner, I will have to work on that. Other than this, the scope of what the project does really broadened over the years and the development pace slowed down as a result. On each stable release I try to set a few goals for myself for the next one, some of them are made, some not. I definitely try to focus on lower-level stuff, roles that are used in many different places (think firewall, LDAP, various databases) rather than specific user-facing applications so that development efforts can be multiplied over multiple roles at a time. On top of all that is all of the non-Ansible related things and management of the project itself (DNS, various services used for DebOps development, etc.). Of course on top of that is my regular workplace stuff which sometimes needs to take precedence. It's definitely good to hear that your employer is aware that it's not all in your fingertips, but you use open projects for your work. I think that's an excellent example to follow. Currently most of the real-time communications within the project is done on IRC, but there's no logging involved and the format of an IRC conversation is probably not very good for some things. More focus on the mailing lists would probably benefit most current and future DebOps users, especially that we now have proper search UI in Mailman 3. An online meetup is an interesting idea, this would definitely help put some faces to the nicks. I'm up for it if anyone else wants to meet up. We can arrange a time and method later via the mailing list; it should be easy to do using some of the many online services. What I would be interested in is access to more varied infrastructure for development. At the moment I have limited resources at home, and development boxes that are available to me at work are extremely slow (How slow? the common playbook on my home workstation takes about 4 minutes to finish. On the development server at work? 35 minutes.). Due to what I have available, I'm doing my development using LXC containers which introduces a bias in what I'm currently working on. Some things are not possible using containers exclusively, for example NFS infrastructure needs at least VM-level setup since you cannot mount filesystems in unprivileged containers. A VM management platform like OpenNebula requires either hardware-level access or virtualization nesting which introduces speed penality as well (that for example would grind my development boxes to a halt). It would be interesting to have access to some boxes and a subnet, of course if latency over SSH is reasonable. Publicity... I really suck at it. And then there's the mentality on the Net (for example Reddit) where advertising your own work can be considered akin to shilling... So I just don't do it. But if you think that DebOps is worth it, and you have some good ideas on how to do it - be my guest. :-) Let me know if you need some pointers about various aspects of the project. Just be aware that this will bring more users to DebOps which will result in more issue reports and pull requests which might be good or bad - who knows. :) Again, thanks for starting the thread and have a nice day, Maciej

On lip 08, Imre Jonk wrote: Good to know. I will have to mine that for some ideas. I can see some friction in that the current "feedback loop" looks like this: - Debian Stable is released - DebOps is upgraded to support new Debian Stable release - Issues are found in Debian which are mitigated in DebOps over time - Hopefully the workarounds to issues find their way into next Debian Stable Because DebOps is focused on Debian Stable and not the development release, the feedback loop being outside Debian itself works pretty well for Stable users. I'm not sure how that would change if DebOps is incorporated into the Debian development process, but perhaps something like this - a "feedback loop" of sorts which can be used to develop solutions that can then be incorporated into Debian would be benefical to the community. It's similar to how Debian treats upstream applications with fixes found in Debian being pushed upstream so that eventually Debian patches can be dropped entirely. This is what happens with downstream distributions like Ubuntu or Mint, but DebOps is not a Linux distribution per se. Yes, it works pretty similarly in Poland as well. But I think that we can leave this as-is for now. If online meetings work out to something bigger and a large number of interested people show up, we can then think about a solution. I would say don't worry about it. :) The official Omnibus packages probably have a more reliable upgrade path than the role in DebOps. Omnibus also provides various dependencies internally which might be important in production if you want to get the latest GitLab release; the version in DebOps will not be updated as frequently. And you should still be able to use GitLab Runners managed by DebOps with the Omnibus GitLab install. I was invided to CCC a few times already, unfortunately at the time the event was happening around Christmas which I usually spend with my family. So it's a very hard pick... Let's see how the situation evolves, though. 2021 will probably be a major milestone for everybody. Very well. I'll try to write some more focused messages in the near future. I've started to write down the things I'm working on in a Project page (kanban-like) on GitHub: https://github.com/debops/debops/projects/5 Small steps, but hopefully this will help improve the project. Thanks! Of course I don't want to force anybody to do this if they don't want to. Pull request reviews are pretty easy to do on GitHub and you don't need rw access to the repository to do them. Every little bit helps. :-) Sounds good to me, you can take a lead if you want. Any preference in regards to the platform? https://meet.jit.si/ is probably an interesting choice, and we can test its capabilities at the same time. We can prepare a room on the day before and send a separate e-mail with the link to the mailing list.. You forgot - I'm already there. ;-) As for the proposed setup - access to the IPv6 network will be interesting since I don't have that now. But otherwise it seems that this would converge to another set of LXC containers. But it will be interesting to see the network layout. That's true. Looking forward to the blog post. :-) Cheers, Maciej

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, TL;DR: The other DebOps Developer here. SaltStack at work for SUSE SLES systems. DebOps with Ansible for my private self hosting. I am not leaving DebOps and therefore also not Ansible for the foreseeable future so you will have to stick with me :) I don’t regret the learning curve and the commitment to excellence of DebOps when I joined early 2015. I at first answered this here but I guess it is better to move it out into an issue to not make this email even longer. Tracked in: https://github.com/debops/debops/issues/1415 Agreed. And let me use that opportunity to thank mainly Maciej for his never ending great work on DebOps! Not to forget maintaining it which is not to underestimate work that hardly anybody thanks one for. Thank you! if you were to cease spending time on DebOps due to other priorities or unforeseen circumstances? That would mean one DebOps Developer less. But on the good side, I don’t hold any keys/accesses exclusively that are needed to continue DebOps. But moving forward, there should be more support for Macjei here. Not so much currently now that all my work on DebOps is done in my spare time, and I am thankful for that. I recently turned off email notifications for GitHub activity which helped me greatly stay focused. Now I have dedicated (spare) time where I work on my off-work projects. I try to maintain a few DebOps roles that I really care about. They are mostly the security, crypto, self-hosting kind of roles like debops.cryptsetup and debops.etesync for example. Also, I see myself as an independent reviewer of all of the core code changes and a lot of roles that enter master, for years. And there is cryptographic proof of it. It is fine. But I cannot take much more responsibility right now due to other projects I maintain also, other side projects and so on. Something which would cause me stress is being in real-time chat all the time like IRC which is why I much prefer structured hands-on discussion in issues and PR. So you don’t see me in IRC. Sorry for that. Impressive setup at work, Imre! Also that you can work mostly with FOSS. Writing proper DebOps roles takes time as we have high standards and long documentation detailing it so I think just put the unfinished roles somewhere and link them in the issue tracker or as PR. This at least prevents someone else needing such a role from writing it from scratch and instead improving the existing role bringing it a bit closer to being DebOps ready. Also, one thing I would propose is more hands-on code reviews. Instead of commenting "this thing should be changed", maybe the reviewer can just fix it themselves if there is no need for discussion (typos or the like) or even propose changes that need discussions. I hope the learning effect is the same. If contributors try to exploit this we could stop it on case-by-case basis. I often do this in the form of checking out a PR hands-on, working on it and opening a PR against the fork. I do this in the hope of increasing productivity for everyone involved. I am also not really eager to take the time for the tax/legal stuff right now. But I also understand that something like this could help the project move forward. I still want DebOps t-shirts :) Ref: https://github.com/debops/docs/issues/221 But I much rather work on technical stuff than do marketing/design. I have the hope that people who know what they want will find DebOps on their own. I guess that is how I got involved. But I know of course that this is wishful thinking. Very true :) 2020-07-15T18:00Z works for me and I will try to make it. I would also propose Jitsi. - -- Live long and prosper Robin `ypid` Schneider -- https://me.ypid.de/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEE7eE3HRuH0o2l6AUVhv2YC78aQPgFAl8HBs4ACgkQhv2YC78a QPgumhAAsw3AtzVD93K17JM1QzaTVHMN4HHR7aknTr7Q8NWexHiUDgxa0gYtNOcF a/97A76x/6IFPfgJ38JnRApuBdcnPCMwjkYIC164+PIBFahhLILbTBglJ7IkMg2/ WsC9RRMKbS3r1CkIeA/B8uC+Mc0MonFzGOP5eTVCcQN7N/EQjzGkeR3DPabWLonL L/s903zdt3YUoY41Kt3UERC6DP/ZYQywTpkGF4jPkNNKGKf4o+TXB5DB8FJGJnBN d00xo7eDuhHk50tMrpZyTkG2bT4LqDJglFzO5zFBph2diZfULW9XkZucbilQpYcO j9vT+YFjw29ia1u+Buvti/p/4ki4peBJxcHgVjnSWYtMFc1FfTbZaJ+Nf1CBO61Q rKNiBwaZC+ABT8TeuqbG209rroZ/iUNDXeI//jVPdvkLyMGQJ9eUHTszUZRxpL2k +aBVHqLQ6e2gHV92GqivXaT6Mibxwejow14TOIZXRoK9hki9shZ0MpAecDsAk6pf AOJCl/n5uQ34QafwJpRvtlXJja7tnZpknhm1j3MmJr/SsT592u6F37SYBS70zy1E m3N+s3ybDPrm7qtN5Vvebev8d6t6wZnZfcNt2uTZSVD/lPM1N29jXZLfqnWuOphw XoxV4N/6h5GlVqxhW0DJk9hKCZ/xWlZRv2PV+pOOR415yb5xrJY= =Fo1O -----END PGP SIGNATURE-----

On 06/07/2020 19:44, Imre Jonk wrote: Hello Imre, Dam here. Thank you for this awesome thread. Debops is truly a gem. It's definitely one of my dearest open source projects along with Debian, QubesOs, Ansible, Nextcloud, LineageOs, Firefox and a few others. Definitely in my top ten. I'm still not in that situation but I'm longing for it. you are not alone but the fact that it's open source reduces a lot the pressure. --- https://dam.venturin.net

Hi Maciej, On Wed, 2020-07-08 at 21:04 +0200, Imre Jonk wrote: Well that subnet assignment took quite a bit longer than expected, but it's done. You now have root(a)debops.ciphermail-demo.com (that domain name is just one we have laying around, feel free to change it). I pulled your public SSH key from GitHub. Here are the SSH host key fingerprints: 256 SHA256:Nrw40T164/ZsBW9braPacXCsVa01D0tV22dBOBTtQ64 root@debops (ECDSA) 256 SHA256:5oveoUe60p4qxMIXwY7CogzFkbj6j2JX9gYOXLhHWfk root@debops (ED25519) 2048 SHA256:oI0WQ6C8kDTbj+gOtnd0T5N5b5KypAYglPLUtpMNv7U root@debops (RSA) You can freely use IP addresses in 87.233.82.200/29 and 2001:9a8:17b::/64, except for the first address in each range, those are the gateway addresses. Let me know if you need changes to the reverse DNS records. I hope you'll find a good use for it (like testing IPv6 support in DebOps or something). Let me know if you have any questions regarding the VM or network. Imre

On Wed, 2020-07-08 at 21:04 +0200, Imre Jonk wrote: The Dovecot bug we discussed before (the one that caused sieve script synchronization with dsync to fail) has been fixed in Debian 10.6! We no longer have to use custom-built packages for this. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=928492 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=930919 Thanks Noah :)