12:25 p.m.
On 13 Aug 2017, at 9:38 PM, Maciej Delmanowski
<drybjed(a)drybjed.net> wrote:
Hello everyone,
TL;DR: I want to merge all the Ansible roles, playbooks, documentation, tests
and software repositories back into one DebOps git repository to improve
development process. I want to be sure that it’s done for the right reasons.
<snip>
What's the current state of the project
---------------------------------------
In hindsight, some of these decisions were good (mostly related to the role
code design and inter-role communication, at least in my opinion), and some
were bad, but unavoidable within the selected development framework (using git
submodules for documentation results in a very slow performance of the 'docs'
repository). I think that the most glaring issue and the easiest to spot is the
installation or update of all the git repositories that contain DebOps
playbooks and roles. For comparsion, I cloned the Ansible repository (~139 MB)
to a new directory and timed it:
$ time git clone https://github.com/ansible/ansible
4,90s user 1,18s system 65% cpu 9,345 total
Running the 'debops-update' script, which clones the 'debops-playbooks'
repository and based on that, clones all of the DebOps role repositories (all
of which has ~152 MB):
$ time debops-update
2,20s user 0,81s system 2% cpu 2:13,13 total
This is 10s for cloning 1 repository vs 2:13s for cloning 121 git repositories
(updates of existing repositories are slightly faster). And this is usually
done each time to get the latest changes, otherwise you would need to know what
git repository changed and pull the changes manually.
What you *could* do, is to thread/parallelize the gits, especially as they are all going
to all make ssh/https connection(s) to pull,
and that would speed up things, but I know: It’ll complicate error handling and won’t add
that much benefit ;)