Maven architecture help


  • mod

    So my team is too junior to have much experience with this, and my peers are not developing in Java. Maybe ya'll can help?

    Background:
    I have a series of Maven-based projects, and a shiny new Archiva server to deploy the JARs to once they build. I'm trying to put together my bamboo plan to build, test, and archive the jars. Bamboo has two entity types: a build plan, and a deployment plan. When a build passes in a build plan, and there is a linked deployment plan, you can create a "release" (which has a name) that can then be deployed to the various deployment environments.

    Email 1

    So looking over this it looks like, when we merge into trunk, the build plan should be:

    • Checkout code
    • Run any local tests (not the functional tests, but like, unit tests)
    • Run the release:prepare step unattended
    • Run the release:perform step

    It looks like I should put that all into the build plan as a sequence of tasks, and not use a deploy plan?

    Email 2

    Ugh, I cannot figure out how to do this :(
    So if I do the release plugin stuff as a deployment job, I can use the “release name” to figure out what release to make it. That’s all well and good, but [the release:prepare step] assumes you have a copy of the working directory checked out already, and it won’t, it would only have the “build artifacts”. I could do a checkout, but if you’re releasing a build that isn’t the latest one, it’ll make a mistake and check out the latest commit. Ideally I’d do the “prepare” step during the build, and the “perform” step in the deployment, but it wants to know what version this will be during the “prepare” step, so that has to be after the release is made.

    How the hell does anyone do this? I will accept jenkins-based answers if you have them, because I can maybe translate them into Bamboo.


  • mod

    Ideally it would see what the latest version was and increment it automatically, but it doesn't seem to do that?

    I suppose what I want is for the build plan to, after running the tests, create a -SNAPSHOT version. Then, when I choose to create a release from a good build and release that, I should be able to put in a version number as the release name and it would create a proper version. Bamboo stores that release name in an environment variable, so I can pass it on the command line, but why does it need a working copy? Am I going to have to put a command-line script that does an svn checkout of the commit number that prompted the build (assuming I can even access that in a deploy job) just to give it a working copy to work with? That seems so wasteful...


  • Winner of the 2016 Presidential Election

    @Yamikuronue We have a similar setup in Jenkins (build and nightly release are separate). What we are currently doing is publishing both a SNAPSHOT version and a file containing the commit id of the latest successful build when the last build step completed successfully. The nightly release process then checks out that commit, runs a few more expensive tests that take too long to run them on every push, and then creates a release.



  • Maven releases are TR :wtf:



  • @Yamikuronue said in Maven architecture help:

    Ideally it would see what the latest version was and increment it automatically, but it doesn't seem to do that?

    Yep. release:prepare asks you for a version for the release and the next version to be appended with -SNAPSHOT

    I suppose what I want is for the build plan to, after running the tests, create a -SNAPSHOT version. Then, when I choose to create a release from a good build and release that, I should be able to put in a version number as the release name and it would create a proper version. Bamboo stores that release name in an environment variable, so I can pass it on the command line, but why does it need a working copy?

    Because release:prepare commits a POM with the updated version and a tag pointing to that commit.. As well, release:perform is going to run a maven build in a forked process, so it needs the source code.

    Why aren't those unit tests part of the maven build?

    Am I going to have to put a command-line script that does an svn checkout of the commit number that prompted the build (assuming I can even access that in a deploy job) just to give it a working copy to work with? That seems so wasteful...

    Didn't you have step 1 as checkout code? Aren't you already doing this or am I completely missing something here.


  • mod

    @JazzyJosh said in Maven architecture help:

    Why aren't those unit tests part of the maven build?

    They are. Why am I doing them again? It already built and passed testing.

    @JazzyJosh said in Maven architecture help:

    Didn't you have step 1 as checkout code?

    There's two phases. But apparently to run phase 2, the deployment job that creates a release and pushes it to Archiva, I have to re-run phase 1, which is build and test.


  • mod

    This just isn't very CI friendly I think. I want to build and test on every commit, but only release when I feel like I've made actual changes and like how it's working. It wants to do all of it in one step.


  • :belt_onion:

    My way is to just don't bother with automating maven-release.


    I have three things for cutting a release:

    • A checkout of trunk which I update before each release.
    • A batch file which runs the release:prepare step:
    @echo off
    IF %MVN_CMD%.==. SET MVN_CMD="mvn"
    echo ##############################################################
    echo ################## Dry run ###################################
    echo ##############################################################
    call %MVN_CMD% release:clean release:prepare -DdryRun
    pause
    echo ##############################################################
    echo ################## For real ##################################
    echo ##############################################################
    call %MVN_CMD% release:clean release:prepare && call mvn release:clean
    
    • A Jenkins server which runs a job with the clean source:jar javadoc:jar install goals and accepts either a parameter VERSION or a Subversion repo path (deploying to a repo is done using a plugin rather than Maven's deploy goal).

    To make a release, I then do an svn update and run the batch file. Maven will ask the version(s), run a local build (in my case twice - this might be overkill) and commit to a pre-defined location in Subversion because I've put the tagBase parameter in the maven-release-plugin config block.

    Once the local verify step has run and everything is commited, I go to Jenkins and either fill the VERSION field on the build job or pass it a path in the SVN repo.

    Bam, if the build succeeded then everything is now in the Maven repo.

    Main problems is that some of the build outputs still need to be sent to a fileshare, something which I still do manually now.


  • :belt_onion:

    @Yamikuronue said in Maven architecture help:

    So looking over this it looks like, when we merge into trunk, the build plan should be:
    ...

    • Run the release:prepare step unattended
    • Run the release:perform step

    Do you really need to cut a release on each merge? I have something like this in my parent POM to make each SNAPSHOT build unique by putting extra fields in the MANIFEST.MF file (Note: those environment variables are set by Jenkins plugins):

    <plugins>
    	<plugin>
    		<groupId>org.apache.maven.plugins</groupId>
    		<artifactId>maven-jar-plugin</artifactId>
    		<version>2.3.1</version>
    		<configuration>
    			<archive>
    				<manifestEntries>
    					<Build-Id>${BUILD_ID}</Build-Id>
    					<Build-Number>${BUILD_NUMBER}</Build-Number>
    					<Build-Version>${project.version}</Build-Version>
    					<Build-Revision>${SVN_REVISION}</Build-Revision>
    				</manifestEntries>
    			</archive>
    		</configuration>
    	</plugin>
    	<plugin>
    		<groupId>org.apache.maven.plugins</groupId>
    		<artifactId>maven-ear-plugin</artifactId>
    		<version>2.4.2</version>
    		<configuration>
    			<archive>
    				<manifestEntries>
    					<Build-Id>${BUILD_ID}</Build-Id>
    					<Build-Number>${BUILD_NUMBER}</Build-Number>
    					<Build-Version>${project.version}</Build-Version>
    					<Build-Revision>${SVN_REVISION}</Build-Revision>
    				</manifestEntries>
    			</archive>
    		</configuration>
    	</plugin>
    	<plugin>
    		<groupId>org.apache.maven.plugins</groupId>
    		<artifactId>maven-ejb-plugin</artifactId>
    		<version>2.3</version>
    		<configuration>
    			<archive>
    				<manifestEntries>
    					<Build-Id>${BUILD_ID}</Build-Id>
    					<Build-Number>${BUILD_NUMBER}</Build-Number>
    					<Build-Version>${project.version}</Build-Version>
    					<Build-Revision>${SVN_REVISION}</Build-Revision>
    				</manifestEntries>
    			</archive>
    		</configuration>
    	</plugin>
    	<plugin>
    		<groupId>org.apache.maven.plugins</groupId>
    		<artifactId>maven-war-plugin</artifactId>
    		<version>2.1</version>
    		<configuration>
    			<archive>
    				<manifestEntries>
    					<Build-Id>${BUILD_ID}</Build-Id>
    					<Build-Number>${BUILD_NUMBER}</Build-Number>
    					<Build-Version>${project.version}</Build-Version>
    					<Build-Revision>${SVN_REVISION}</Build-Revision>
    				</manifestEntries>
    			</archive>
    		</configuration>
    	</plugin>
    </plugins>
    

  • mod

    @JBert said in Maven architecture help:

    Do you really need to cut a release on each merge?

    No, which is why I wanted to set up the deploy job to do the release: it's only triggered manually, while the build job is triggered on every merge.



  • Maybe, I am confused, but don't you just want two build plans? One for snapshots and one for releases? You don't want to build a release build on commit.


  • Discourse touched me in a no-no place

    @jaming said in Maven architecture help:

    Maybe, I am confused, but don't you just want two build plans?

    Maybe more. I could see different plans being used for on-commit builds, nightly builds and release builds; the differentiators would be things like the amount of testing and packaging applied. (The rules for what to do during the different types of building and testing would be encoded in Maven, probably using different profiles.)

    Back when I was doing a lot like this (as opposed to being more devops focused) releases were done mostly manually; advancing things to the release state would be done manually, and special build/test jobs would be then run on a stabilisation branch for just that release (which would also generate the Release Candidates). If we were properly on top of things, there would be very few commits on the branch. Sometimes that was even true. :)

    But then we weren't doing many releases a year. If you're doing a release a week, you'll need more automation.


  • mod

    @dkf said in Maven architecture help:

    releases were done mostly manually;

    Honestly, the biggest issue with doing them manually is training. I want to be able to hand off to my most junior team member some of the smaller repos and be like "To cut a release, click this button" rather than "To cut a release, download and run these batch scripts and pray they work". You know? I'm 100% in charge of this project, so I want to do it right



  • @Yamikuronue said in Maven architecture help:

    I want to be able to hand off to my most junior team member some of the smaller repos and be like "To cut a release, click this button" rather than "To cut a release, download and run these batch scripts and pray they work".

    I am not talking about manually doing the release builds. You just need to manually trigger the release builds. That is why you need more than one build plan.


  • mod

    @jaming Right. Bamboo has the concept of a deploy plan that is manually triggered using the results of a successful build, which is designed for things like this... I think.



  • @Yamikuronue said in Maven architecture help:

    @jaming Right. Bamboo has the concept of a deploy plan that is manually triggered

    I am not familiar with Bamboo, but won't you need a build plain that is manually trigged as well?

    Do you really want a release build built with every commit? I must be missing something in your process. What is your definition of a release build? To me it is a build using the maven release plugin (linky) that is named and tagged.


  • mod

    @jaming said in Maven architecture help:

    but won't you need a build plain that is manually trigged as well?

    The build plan triggers on every commit. After a successful build, I have the option to trigger a second plan manually, which will package the release. That's what I'm trying to do. Only naming is a problem because bamboo plans can't have inputs. Deploy plans can have one: the name of the release. (well, two, in that you can tag artifacts from the build to be deployed)


Log in to reply
 

Looks like your connection to What the Daily WTF? was lost, please wait while we try to reconnect.