Project maintenance

[anusch-athari]

Hello everybody, @fujin, @fnichol, @patcon, @realityforge, @atomic-penguin

as you all know there are a couple of chef-jenkins projects and forks on github.

Seeing that this repo is the one listed under
http://community.opscode.com/cookbooks/jenkins
I was wondering if there was any way to combine at least some of the forces here. So please feel free to add anyone who might be able to help with this.

Would be more then happy to help fix some issues myself.

[patcon]

👍 I'm in.
/cc @dols you seem to be active lately too :)
@gosquared

[chrisroberts]

I will start carving out some time for this cookbook to get pull requests merged in. Getting some test-kitchen support in would also be a huge help to getting requests merged faster. I'll see if I can't throw in at least a basic set of tests to ensure new bits don't cause major breakage as things progress.

[atomic-penguin]

I have been too busy to do any further work on this cookbook at the moment. I am interested in a deploy from war file style recipe. On the other hand the branch I think which was most considered for release makes use of a process monitor, rather than an init-style system. Which as far as I know, is kind of a deal-breaker at the moment on RHEL-like systems.

The existing logic around service management is quite convoluted, that is a given. I think the process-monitor + war file deployment is a good idea for platforms where that actually works. Here are some thoughts I had, but haven't got around to testing yet, considering I haven't had a chance to touch this in almost 6 months. Would refactoring this into a tomcat_application wrapped cookbook make the task of deploying Jenkins easier for all supported platforms across the board? Are there any issues with deploying Jenkins on tomcat? Because, frankly that wasn't an option 6 months ago (application cookbooks were being refactored), and it seems like an easier path to make this more maintainable in the long run. A lot of the Apache/Nginx recipes would remain the same, or could make use of AJP proxy modules with such a Tomcat/War file refactor. I wouldn't imagine it would heavily affect the Windows slave recipe going that direction.

No offense, meant to anyone that took this cookbook in a different direction. Just what I was thinking about, and wanted to get off my chest. I will help where I can, with whatever is decided.

[realityforge]

I would like to help out. I think the LWRPs are of particular value to be shared.

Initially we used the deploy as war approach (initially with tomcat but then with glassfish) and the deploy was easy but found all the little gotchas around upgrading plugins and config from within the web containers reduced the value and went back to a simpler run it from the command line approach.

[bdols]

I can help. Just let me know if you need something.

[fnichol]

I've been far too busy lately to help give this cookbook the love that it clearly needs 😿

@chrisroberts, I totally understand where you're coming from. I'm finding it increasingly difficult to take patches quickly without a lot of manual testing time. Hoping I can change that soon, but until then, I so completely sympathize

@atomic-penguin, Every time I revisit Jenkins deployments that's my first choice too: just deploy it as a war and use the (now usable) application cookbook. Part of what's making this an increasingly gnarly codebase to maintain is the platform conditional hell with packages/init systems that's crept up over the years (no really, years). Then you're back to long testing sessions (see above).

While I hesitate to ever push a "reinventing the wheel" approach, do anyone see the value in giving this cookbook a fresh take? Like I said, thinking aloud and maybe there's more good in here than I can remember…

In the meantime, it would be nice to point people to one development mainline (here perhaps?). Would it be helpful to give @patcon, @fujin or @chrisroberts (or others) push bits on my repo to help unify things? I know this version was downstream at a certain vintage and I'd argue this one is more up to date than mine.

Go team! 💌

[pwelch]

Wanted to jump in and also offer my time on this project. Let me know what I can do to help.

[patcon]

@fnichol Wahoo! I think a fresh start is a GREAT idea. And perhaps we could lay the ground rules that everything gets added through pull requests, and set the expectation that tests should accompany new features as we build out?

And I don't think it would it be irresponsible to be generous with maintainer status? Thinking that giving it to a good few people might be a great way to make sure we all have buy-in, the lack of which is probably a big reason we're so fragmented right now :) Really kills momentum when a single-point-of-failure maintainer gets busy with important real-life stuff

[schisamo]

Hey everyone just wanted to chime in here. I did a refactor on this cookbook earlier this week based on some changes we have made internally here at Opscode. I would love to get some feedback and also offer to step up as a maintainer.

Some of the changes include:

• massive simplification to main server recipe which now stands Jenkins up using a war file (vs os packages)
• fixed numerous bugs in Groovy scripts
• monolithic attributes file has been broken based on server/node.
• Chef 11 compat fixes
• mostly Foodcritic compliant now, only warnings are because Foodcritic doesn't properly recognize the recommends and suggests keywords for dependencies in metadata.rb files.
• repo has been fully Berkshelf-ized for easy development

Next steps include:

• actual test coverage including test-kitchen/jamie integration tests
• re-examine existing LWRPs and see if Jenkins REST API can be better leveraged.
• LWRPs for defining pipelines
• complete rewrite of README based on recent changes
• ??

More info on the refactor-all-the-things branch of my fork:
https://github.com/schisamo/chef-jenkins

[fujin]

We don't really use Jenkins very much anymore. I'd be happy for @opscode
@schisamo to take over maintenance, especially if it was pulled into the
rest of the COOK project.

I spent a short amount of time trying to start a jenkins v2 version of
this, but didn't really find the inspiration to take it all of the way. As
mentioned, we've removed the last remnants of Jenkins from our stack(s).

Cheers,

AJ

On 26 January 2013 14:38, Seth Chisamore [email protected] wrote:

Hey everyone just wanted to chime in here. I did a refactor on this
cookbook earlier this week based on some changes we have made internally
here at Opscode. I would love to get some feedback and also offer to step
up as a maintainer.

Some of the changes include:

• massive simplification to main server recipe which now stands
  Jenkins up using a war file (vs os packages)
• fixed numerous bugs in Groovy scripts
• monolithic attributes file has been broken based on server/node.
• Chef 11 compat fixes
• mostly Foodcritic compliant now, only warnings are because
  Foodcritic doesn't properly recognize the recommends and suggestskeywords for dependencies in
  metadata.rb files.
• repo has been fully Berkshelf-ized for easy development

Next steps include:

• actual test coverage including test-kitchen/jamie integration tests
• re-examine existing LWRPs and see if Jenkins REST API can be better
  leveraged.
• LWRPs for defining pipelines
• complete rewrite of README based on recent changes
• ??

More info on the refactor-all-the-things branch of my fork:
https://github.com/schisamo/chef-jenkins

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/23#issuecomment-12728884.

[schisamo]

@fujin I talked with @jtimberman about moving my fork of this cookbook into the opscode-cookbooks org (and to our supported list). If you guys are still cool with that do you mind adding the opscode user as a collaborator on the community site? Current plan is to prep the cookbook for a 1.0.0 release next sprint.

[fujin]

I'll action this momentarily.

--AJ
On 1/02/2013 11:04 AM, "Seth Chisamore" [email protected] wrote:

@fujin https://github.com/fujin I talked with @jtimbermanhttps://github.com/jtimbermanabout moving my
fork https://github.com/schisamo/chef-jenkins of this cookbook into the
opscode-cookbooks org (and to our supported list). If you guys are still
cool with that do you mind adding the opscode user as a collaborator on
the community site http://community.opscode.com/cookbooks/jenkins?
Current plan is to prep the cookbook for a 1.0.0 release next sprint.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/23#issuecomment-12968523.

[fujin]

Sorry for the delay, been busy: opscode is now the maintainer of cookbook
jenkins

Cheers,

AJ

On 1 February 2013 11:04, AJ Christensen [email protected] wrote:

I'll action this momentarily.

--AJ
On 1/02/2013 11:04 AM, "Seth Chisamore" [email protected] wrote:

@fujin https://github.com/fujin I talked with @jtimbermanhttps://github.com/jtimbermanabout moving my
fork https://github.com/schisamo/chef-jenkins of this cookbook into
the opscode-cookbooks org (and to our supported list). If you guys are
still cool with that do you mind adding the opscode user as a
collaborator on the community sitehttp://community.opscode.com/cookbooks/jenkins?
Current plan is to prep the cookbook for a 1.0.0 release next sprint.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/23#issuecomment-12968523.

[anusch-athari]

👍

[jtimberman]

FYI: Repo waiting for @schisamo to push: https://github.com/opscode-cookbooks/jenkins

[schisamo]

Just an update. Some initial refactor code has been pushed to https://github.com/opscode-cookbooks/jenkins. I want to add some chefspec and test kitchen coverage before the 1.0.0 release. Watch the projects TODO.md for more details.

[patcon]

LWRP for defining pipelines

This. Is. Awesome.

Already using pipelines via an inelegant method in another project, but hadn't even thought to make it into a LWRP...!

If you're indeed talking about using the build-pipeline plugin though, I'm thinking maybe it would be more fitting to add that to a new cookbook that depends on the jenkins one? Suggesting that from the same angle that @fujin clarified at one point -- the jenkins cookbook could easily be a behemoth with all the different ways people use it, so best to set the precedent of splitting out opinionated things like this, imho :)

[schisamo]

@patcon I'm thinking build pipelines belong in a downstream cookbook from this one. We use that pattern internally at Opscode with our opscode-ci cookbook. There is currently an opscode_ci_server_pipeline LWRP which is used to deploy build/test/release pipelines for chef-server (and private-chef) Omnibus packages. For instance this defines the full pipeline for the chef-server builds:

opscode_ci_server_pipeline "chef-server" do
  repository "git://github.com/opscode/omnibus-chef.git"
  branches ["master"]
  platforms ["ubuntu-10.04", "ubuntu-12.04", "centos-5.8", "centos-6.3" ]
  architectures "x86_64"
  triggers [:git_poll, :nightly, :ad_hoc]
  notifications(
    :hipchat => {:token => "XXXXXXXXXXXX", :room => "000000"}
  )
  action :create
end

I would like to open source that LWRP because it really could be called omnibus_pipeline and any omnibus based project could use it for build/test/release. I know @adamhjk has some continues delivery stuff he would like to share also.

[patcon]

Really excited for this. Thanks for the snippets. But it sounds like you don't use the build-pipeline plugin per se? Or perhaps just no feature for manual gates yet? If not using the plugin, I suppose that's my own extension cookbook to write. Whoop whoop!

Also, quasi-related, but came across OpenStack's python tool that might be interesting:
http://ci.openstack.org/jenkins_jobs.html

Seems it already has a cookbook in the works with a definition:
https://github.com/needle-cookbooks/chef-jenkins-job-builder

cc: @cwjohnston @timmytool

[cwjohnston]

I've previously mentioned to @schisamo that we wouldn't mind folding our jenkins-job-builder cookbook into this one if there were any interest. However, if Opscode already has their own pipeline LWRP in the wings, I don't expect it will be worth their while to support two different approaches to generating pipelines.

One unfortunate aspect of our current implementation is that we invoke jenkins-job-builder on every chef run, depending on it's own cached system state to ensure idempotency. Not sure how Opscode's LWRP works, but I found this trade off to be acceptable since it has helped me to avoid writing XML template files. :)

[realityforge]

We have a similar job building library at [1] that we use to build up pipelines. Feel free to yoink that if interested

[1] https://github.com/realityforge/chef-jenkins/blob/master/libraries/jenkins_xml_config.rb

[fujin]

Thanks @patcon appreciate it.