TL;DR: we need to talk about keeping DebOps the awesome and healthy
project that it is today, for many years to come. Please voice any
concerns you may have. Also, some battle history, cuz' Maciej is a
sucker for war stories.
Let me start off this lengthy email by congratulating everyone involved
in this project with the sheer progress that we made since DebOps
0.8.1, which is the first DebOps release I got to use. In production
mind you, which may or may not have pressed Maciej to pull forward his
1.0.0 release .
There has been even more progress since genesis  (is that where the
'ginas' name came from, Maciej?). I had just started my System and
Network Engineering education at the AUAS and landed my first job as a
Debian (or initially, Ubuntu) system administrator at Bits of Freedom.
It would have been good to know a thing or two about Ansible and IT
automation back then. But I got an O'Reilly book on that subject a few
months into the job, so that was nice.
After some time I had been using Ansible for about half a year at Bits
of Freedom, but wrote all playbooks from scratch. They weren't exactly
the best quality and I still felt like I was working hard, not smart. I
had trouble reusing my own roles and had difficulty solving some
automation problems which resulted in me still doing a lot of manual
sysadmin work. This of course felt more like putting out fires instead
of real progress. I thought that Docker could save me, but no, that
only made it worse because now I still had the exact same automation
problems, but with the added complexity of container lifecycle
Then, somewhere at the end of february (or beginning of march) 2019, I
decided to give this "data center in a box" thingy a try. I had just
landed my second sysadmin job at CipherMail, where the previous admin
had been doing everything by hand. Long story short, I turned off the
very last legacy, hand-managed VM last week. We now run ~45 servers
across multiple data center locations and cloud providers. There are
high availability setups in there, like our DNS, LDAP, RADIUS, mail and
web clusters. We have firewalls, load balancers, DNSSEC, Let's Encrypt
and IPv6 everywhere. Our office wifi has WPA2 Enterprise with EAP-TTLS
for authentication against our LDAP database. Our VPN setup is
incredibly easy to enroll for new users and allows login with LDAP
username, password and Yubikey OTP. Simply removing a user from our
LDAP database immediately revokes their access to almost every IT
system we have. Everything is all meticulously monitored with Icinga
and its agents. Security patching takes minutes instead of hours. And
the best thing? It is *all* automated. And that leaves me with lots of
time to automate even more, like our continuous integration setup and
an upcoming cloud service. I actually have the time to visit customers
on-site now, which happened last week for the first time and was really
inspiring to me!
I truly feel like I am in full control of the whole IT infrastructure
at my place of work, which is something that I had definitely not felt
before. That data center in a box thingy, combined with some additional
roles I wrote, has been a major game changer for me. It became the only
open source project that I regularly contribute to, although I don't
put nearly as much time in it as I would want. The small but capable
community around it is just great. I feel indebted to many of you,
especially to Maciej.
Which is why I never want this adventure to end.
Honestly, even the thought of the possibility of this great project
ceasing to exist makes me sweat. I know that DebOps has brought a lot
of joy to many system administrators and homelab hobbyists out there.
CipherMail has also benefited greatly from the steady development over
the last few years. Of course there won't be an immediate operational
or security issue for us if all development would cease, but our
automation progress would most certainly be heavily impacted by it.
So consider this a check-in round. What I would like is an honest
'temperature' reading of all contributors. Not your actual temperature
(although it would be nice to hear that you're COVID-free), but more
how you currently see the future of DebOps and your involvement. Is
there anything you would like to see improved? And, even more important
but not as pleasant to discuss, what would happen if you were to cease
spending time on DebOps due to other priorities or unforeseen
circumstances? Do you feel like the project is (heavily) dependent on
you? Is your involvement causing you any sort of stress?
Lastly I'd like to lend a proverbial ear to any suggestion or comment
on this email. I want to know if I can do something more to ensure the
continued success of this project. I am certainly able to arrange
things like publicity and sponsoring of infrastructural services, and
will probably be able to organize a meetup for all of us once these
tougher times are behind us. Or maybe we should do regular online
meetups. Let me know what you think. My employer and I both recognize
the importance of our shared adventure.
Yours truly, and truly yours,
The meeting tonight starts at 18:00 UTC, feel free to join using the same URL
as last time. Don't have it (anymore)? No problem, just send me an email and
I'll reply with the details some 15 minutes before we start.
I just noticed that the pki config on a server got skewed. Our pki
config has always worked fine. On trying to run it now with
$ debops -v service/pki -l affected.server
TASK [cron : Manage cron environment variables] *
fatal: [server.name]: FAILED! =>
The task includes an option with an undefined variable. The error
was: dict_keys object has no element 0
The error appears to be in
line 89, column 3, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- name: Manage cron environment variables
But obviously I have never edited this file.
To test another role, I tried running the service/nginx role, which
finishes fine. How do I get the pki role working again? Any suggestion
I totally forgot about the meeting which is starting in 10 minutes.
Unfortunately I've fallen ill (nothing serious, I just need rest) and won't be
attending. Feel free to do the meeting without me at the usual Jitsi URL. Good
evening and until next time!
I'm working on a Prometheus Node Exporter DebOps compatible role: it set
ups /etc/services, ferm and apache via those three debops dependencies.
GPL3 apart, what is DebOps position and strategy on including
contributed roles in the debops repository?