[debops-users] run additional task after common.yml
by Erik Schwalbe
Hi,
Is it possible to automatically run additional tasks after execution of common.yml?
for example:
ansible/playbooks/test-vm.yml
---
- name: additional test-vm tasks
hosts: test-vm
...
and execute this playbook automatically with debops -l test-vm
Perhaps it is possible to add a host_vars variable
Thanks and regards,
Erik
6 years, 5 months
- 2
- 1
[debops-users] DebOps repository merge and current project status
by Maciej Delmanowski
Hello everyone,
Yesterday the number of pending DebOps pull requests on GitHub reached 0.
Therefore, today I merged the current code from most of the DebOps roles and
some other repositories. The new repository is available at:
https://github.com/debops/debops.git
The repository contains code from the 'master' branch of most of the DebOps
roles as of 2017-10-21. This repository should now be considered as a base
DebOps repository, and all changes to the existing roles, or any new roles
and other code, should be directed to it.
Existing DebOps repositories ('debops-tools', Ansible roles, documentation,
etc.) haven't been changed and they are still available. The 'debops' scripts
installed from PyPI will still use the old repositories to install DebOps; the
same goes with installing individual roles via Ansible Galaxy.
How the repository merge was done
---------------------------------
For the curious about the migration process from multiple git repositories to
a single "monorepo": the main obstacle was the same directory and file names
in Ansible role repositories (ie. 'defaults/main.yml', etc.). The easy
solution would be to just copy the files directly to the new repository, but
I wanted to preserve the git commits in the new monorepo, for posterity and
history.
First, I downloaded all of the existing DebOps roles via 'debops-update'
script to a new directory to have a clean workspace. Then in each git
repository, I performed the commands (an example from the 'debops.users'
role);
mkdir -p ansible/roles/debops.users
git mv CHANGES.rst COPYRIGHT defaults docs .gitignore LICENSE meta \
README.md tasks templates .travis.yml UPGRADE.rst \
ansible/roles/debops.users
git commit -m 'Move role code to a subdirectory'
This ensured that the code of each Ansible role and other DebOps repositories
(debops-playbooks, debops-policy, debops-api, debops-tools) would be placed in
a separate directory after the merge in the monorepo.
After the code in all of the relevant repositories has been moved out of the
way, I switched to the main monorepo and started merging the repositories one
by one:
git remote add -f ansible-users ../roles/debops.users
git checkout -b role-users ansible-users/master
git checkout master
git merge --no-ff role-users
With this, the existing git commits in the separate repositories have been
merged directly into the 'master' branch of the new monorepo. There's no
submodules, no subdirs, all of the code is directly in this repository.
The git tags were an issue after the merge - since git allows only one unique
tag with the same name, and DebOps roles used the same names in tags (v0.1.0,
etc.), merging them over was impossible - renaming would just result in a new
tag. Therefore I decided to remove all of the existing tags and start from
scratch. Since the last tagged version was v0.5.1, I tagged the current code
as v0.6.0 to avoid collision with the old repositories. Hopefully this will
quickly go to v1.0.0, but we will see how it goes. :-)
The size of the '~/.local/share/debops' directory on my workstation sits now
at 168940 kilobytes (165M) as reported by 'du'; the new repository is 40556
kilobytes (40M). There are 9314 commits at the moment, but that's only part of
th whole project - some repositories like 'test-suite' or 'examples' hadn't
been merged. You can check out some of the statistics of the repository on
GitHub.
I omitted some of the existing DebOps roles during the merge:
- 'debops.backporter' was due to be deprecated. The role was created before
Debian Jessie has been a Stable release, to aid with backporting Ruby 2.1
packages to Debian Wheezy, required at the time by GitLab. Since then,
Jessie has become Stable and is now Oldstable, GitLab is installable
directly on current Debian Stretch, and concept of automatically backporting
Debian packages from Testing distribution to Stable has been dropped in
DebOps.
- 'debops.gitlab_ci' role was created when GitLab CI was a separate
application. Since then, GitLab CI has been merged with the main GitLab
application and the role was deprecated.
- 'debops.gitlab_ci_runner' was the remote Runner for old version of GitLab
CI. It was replaced by GitLab CI Multi Runner some time ago, and the role
was deprecated in favor of 'debops.gitlab_runner' which supports the new
version.
- 'debops.kvm' role was used to set up a KVM environment on a host. This
functionality has been moved to the 'debops.libvirtd' role and the KVM role
was deprecated.
- 'debops.monkeysphere' role was in the project almost from the beginning and
was meant to aid in configuration of The Monkeysphere Project on the managed
hosts. I haven't used it in a long time, and I hadn't heard about anyone
else using it either, so I decided that this is as good moment as any to
deprecate and remove it. The support for Monkeysphere might return someday.
- 'debops.mysql' role was some time ago deprecated and replaced by the
'debops.mariadb_server' and 'debops.mariadb' roles.
- 'debops.php5' role has been deprecated in favor of the 'debops.php' role.
Apart from the existing DebOps roles located in the 'ansible/roles/'
directory, in the main repository you can currently find:
- 'lib/debops-api' - contents of the https://github.com/debops/debops-api/
repository.
- 'lib/debops-policy' - contents of the
https://github.com/debops/debops-policy/ repository.
- 'lib/debops-tools' - contents of the https://github.com/debops/debops-tools
repository.
- 'ansible/playbooks' - contents of the
https://github.com/debops/debops-playbooks repository.
The project playbooks are currently located in an awkward
'ansible/playbooks/playbooks/' subdirectory because I wanted to preserve the
existing git commits of that repository. They will be moved to the parent
subdirectory at some point.
Current state of the project
----------------------------
At the moment, the new repository does not have any Travis-CI or GitLab tests.
Because of that I don't want to rush with more involved code changes just yet.
I'm looking into writing tests for specific DebOps roles and their
combinations using GitLab CI, based either on Vagrant-LXC or Vagrant-libvirt
test environments. Tests based on Docker are insufficient for DebOps, since
various playbooks and roles can manage multiple services at once - Docker
traiditonally uses one container per service which clashes with the
DebOps-managed environment.
Quick evaluation of Vagrant-LXC based test environment is promising, apart
from the more low-level roles like management of network interfaces since
'systemd' inside LXC containers doesn't allow for bringing the network
interfaces correctly in Debian Stretch; due to that a test environment based
on Vagrant-libvirt and a KVM backend seems to be the best fit for these roles.
As for the Travis-CI tests, I want to create a general suite of tests which
will check the project's documentation and validity of the Ansible code
without executing the playbooks and roles directly.
At the moment the main project repository cannot be installed or updated using
the 'debops-update' command, and is unusable via the 'debops' command without
some manual intervention. If you want to base your environment on the roles in
the new repository, easiest way to do it would be to symlink the
'ansible/roles/' and 'ansible/playbooks/playbooks/' directories from the main
repository to the respective 'ansible/roles/ and 'ansible/playbooks/'
directories in a given DebOps project directory.
The next steps
--------------
There are currently 281 open issues across the old DebOps roles. They will be
resolved over time by fixes in the main DebOps repository. Please create any
new issues or pull request against the 'debops/debops' repository on GitHub
and not the old, separate role repositories.
Cleanup of the old codebase is a good and easy way to start. Removal of
the redundant LICENSE, .travis.yml files and the like. Perhaps even the README
files in the role directories can go as well, they will be preserved in the
git history, and they aren't exposed as much in the new repository.
The documentation from role directories, playbook directory and separate
policy directory should be moved to the the new 'docs/' directory, with
a Sphinx build environment. When this is ready, the new documentation will be
available at https://docs.debops.org/ as usual, via ReadTheDocs.
A few people on IRC expressed interest in updating the 'debops-tools' scripts.
It's a good opportunity to write them from scratch with a new command line
interface based on subcommands. I imagine that it will take a while before
this is ready though. For now, you can just clone the repository directly to
install it.
I plan to look into GitLab-CI tests and prepare the tests for as many roles as
I can using the Vagrant-LXC environment, so that updates can be easily
checked. I don't plan to create a public GitLab instance for that yet, instead
you will be able to deploy your own test environment based on DebOps. At the
same time, I plan to remove the example playbooks from the role
'docs/playbooks/' directories and use the main project playbooks as examples
in the documentation (if possible, not sure yet), and for role testing.
I want to reach out to a few Debian Developers interested in packaging DebOps
in Debian. With the new repository it should be much easier to do than before.
Hopefully with their help, a 'debian/' directory can be added in the
repository so that users can build their own '.deb' DebOps packages and
install the project that way.
Well, that's it for now. Let me know if you have any questions or find issues
with the new repository.
Cheers,
Maciej
6 years, 6 months
- 4
- 4
- ← Newer
- 1
- Older →