[Plone-testing-team] new jenkins master server status

Gil Forcada gil.forcada at freitag.de
Sat Dec 13 00:23:49 UTC 2014


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

Timo (is always really nice to talk and discuss with you :) and me
meet for 2-3 hours yesterday to talk about:
- - the status of the new plone master server
- - the setup of a pipeline of jenkins jobs to give better feedback to
developers
- - the status of using jenkins-job-builder[1]


What's already done:

- - a testing server[2] is up and running
- - is configured with ansible playbook plone.jenkins_server[3] (which
actually uses ansible-jenkins[4] to install and setup jenkins)
- - the ansible playbook already integrates with GitHub (via a jenkins
plugin) thought that brings some problems[5]
- - a simple pipeline of jenkins jobs using jenkins-job-builder is
available[6], although as the current pipeline approach does not fit
our needs we need to change that (see first action items)



Action items is (so that you are welcome to work on it):

- - investigate the new workflow plugin[7] and prototype it so that two
parallel builds use, for each pipeline of jenkins jobs their own
buildout.coredev "seed"
- - recreate core jenkins jobs in jenkins-job-builder to be able to
merge it as soon as possible. That actually means all jobs on
jenkins.plone.org Core tab, see [6] for how to create jenkins jobs
with jenkins-job-builder
- - define a strategy on how to handle passwords and access to jenkins
without having to put all credentials in a public repository (Timo
said that will ask Sven, but anyone is welcome to help!)[8]
- - a GitHub app (see second point in [8]) needs to be created, as
currently plone.jenkins_server is using a dummy app on my (gforcada) user
- - understand how ansible galaxy works so that we publish our playbooks
and fetch them from there (also the ansible-jenkins base playbook that
we reuse), by now we added them on jenkins.plone.org repo as submodules
- - check the status of plone.jenkins_node, the ansible playbook for
jenkins nodes and update it according to how plone.jenkins_server is
currently working

- - add QA scripts on esteele.manager (or somewhere where it makes sense
and esteele.manager (or that package) to buildout.coredev so that
everyone can use them
- - add a way (jenkins-checkouts.cfg?) that buildout.coredev can use all
packages from checkouts so that we can run the above QA tests with
everything being in development mode (to spot circular dependencies,
track usage of a distribution, etc etc)
- - these scripts are actually meant to run on jenkins mostly (though
being able to run them locally without too much magic is key also) so
we need to define a common way to report what the QA scripts are
outputting to be able to parse it in a generic(?) way from jenkins itself

- - once a few parts of the above are ready, create/update the jenkins
job that creates jenkins jobs to also be able to create a jenkins job
for each QA script defined


About this QA scripts:
While talking with Timo it was clear that if we don't have a way to
get all distributions from checkout it will be impossible to actually
know if we are getting rid of certain dependency or to track the usage
of something (say plone.api), etc etc.

Is not silver bullet though, buildout can get really slower due to
having to checkout so many repositories, etc etc, so some
experimentation and measurements need to be done, see also[9]




So if anyone fancies working on anything of the above, say it![10]

Have nice Christmas holidays everyone!
Gil


[1] http://ci.openstack.org/jenkins-job-builder
[2] http://78.46.245.68
[3] https://github.com/plone/plone.jenkins_server
[4] https://github.com/ICTO/ansible-jenkins
[5] Like the credentials needed to access from the command-line,
jenkins CLI or from within a jenkins job for example, needs to be
discussed and solved somehow
[6] https://github.com/plone/jenkins.plone.org/tree/jenkins-job-builder
[7] https://github.com/jenkinsci/workflow-plugin
[8] things that shouldn't be on a public repo but somehow need to be
scripted:
  - access to jenkins while the github oauth is active. One manually
working solution is to create a jenkins local user and use its access
token. This is needed to be able to create a jenkins job that creates
jenkins jobs and also for the ansible playbook to be able to
administrate jenkins
  - the github oauth plugin itself needs the Client ID and the Client
Secret of an existing GitHub app (which should not be on everyone's
laptop...), this is needed on the ansible playbook, to be able to
automatically configure Jenkins properly
[9] https://github.com/plone/jenkins.plone.org/issues/57
[10] Just kidding, wanted to add 10 references :D
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJUi4cGAAoJEM2P68YKpRQlGBEP/ihM7Unjwi34nX6zI/2nnYSg
7vIm5tvtNBA0VUS5O2M//PyoLGC8NeKWOJyLDPN4wNMmD2y+Zfe5PEfqieyW3H88
3hsPvGhA/uCIehGBb2dO2VUJiHs6wIC7lNF4N0R01kpyEM6Rd6fRo5y0fvyVGkSC
wL0TnvOyDL1FzB6CRbv9wnD4WHc4FgMWmHSHmGSDBMi1S0dYlcoY/USa8ub11+If
eiA6EuUUeWd25cAoZXcq9XKS2BsCunydeo84PRtCs+Khx1qJVVeFuIZ1zevP58DJ
q85X5Aq9oZpDAUIdkpOqQpEpD5stK8f8NSxxHhgu4RmFrbt0rg1lwV1+VJdjKKcd
Q99m9UnYBJp0dagTaJqGtVm0pQsAFDt/5Vspf3+2GVg3CbTWRn+rG6khUU1Z4Uxd
0mHR+n0/CHSQZYRiiZZH4wwCL6zrWXoAUSsGv6iHfv+Vq7/a1hazzYLtWD5cjyZy
cNbH6M6AR4enfC7vzdvgaDQjSDUnveaYxaYm3y71dGHjKnvEnVoXHqiftHo6x+Yc
7z+R1qpk2rBoGmfah9FIUepiX42PxCW4ZQQrXNklnRxJSZ9GGYx6KCIwLVokOEfP
CGWZ6P+wy8GZvCx8Vkr3QqiSXFEnOdgM2hwOqil2yiJjFcR5Y2nE/c+HQMaKCTTl
/2T4Tj3qIDIBd6H2y/+I
=kDuT
-----END PGP SIGNATURE-----


den Freitag gibt es ab sofort auch digital! Jetzt stöbern auf: https://digital.freitag.de


Diese Nachricht (inklusive aller Anhänge) ist vertraulich. Sie darf ausschließlich durch den vorgesehenen Empfänger und Adressaten gelesen, kopiert oder genutzt werden. Sollten Sie diese Nachricht versehentlich erhalten haben, bitten wir, den Absender (durch Antwort-E-Mail) hiervon unverzüglich zu informieren und die Nachricht zu löschen. Jede unerlaubte Nutzung oder Weitergabe des Inhalts dieser Nachricht, sei es vollständig oder teilweise, ist unzulässig.

This message (including any attachments) is confidential and may be privileged. It may be read, copied and used only by the intended recipient. If you have received it in error please contact the sender (by return E-Mail) immediately and delete this message. Any unauthorised use or dissemination of this message in whole or in part is strictly prohibited.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.plone.org/pipermail/plone-testing-team/attachments/20141213/40116bad/attachment.html>


More information about the Plone-testing-team mailing list