Setting up a windows AMI for use with Elastic Bamboo

This is a revision of a post I originally posted at the http://consultingblogs.emc.com/gracemollison/ but it’s quite useful until Atlassian get round to supporting windows as part of Jira Studio.

Jira Studio uses elastic bamboo as its build controller which in turn uses elastic agents (AWS ec2 instances that are spun up by the build controller to run the build) rather than the standard remote agents.

The Diagram from Atlassian  below illustrates the Elastic Bamboo configuration:


You will need to have an Amazon AWS account and will have to create a custom windows AMI to be used to create your elastic-agents.. This post outlines what I did to get this working for a recent project. Please note that this is unsupported by Atlassian but hopefully they’ll be supporting windows soon.

Please note that this post assumes you have some basic knowledge of using AWS.

Start a Windows 2008  instance from an  Amazon base windows 2008 AMI in the us-east-1  region. It needs to be in the US-East region as this is where Jira Studio expects to find the build agent.

All actions below are carried out from the instance you have started.

Turn off windows firewall

Install the latest JDK. (You will need tools.jar in your JAVA_HOME path hence why the JDK is required)

Install whatever components you need to be able to undertake a build e.g visual studio, msbuild, SDK’s  etc and other bits

Use the link here as a  guide : http://confluence.atlassian.com/display/BAMBOO/Creating+a+Custom+Elastic+Image

Set up the Amazon ec2 API tools as outlined in the Atlassian guide ( section 5.4). Specific guidance for windows can be obtained from the AWS documentation.

Check the version of Bamboo by  clicking on administration then expand under system \ system information  :


Scroll down to see the Bamboo version

Download the bamboo-elastic-agent that matches the version of Bamboo that is being used in Jira Studio: http://www.atlassian.com/software/bamboo/BambooDownloadCenter.jspa Make sure you click show all.

Create the   folder c:\bamboo-elastic-agent . Unzip the bamboo-elastic-agent-2.6.2.zip to this folder.

Download the latest zip of ant  from http://www.apache.org/dist/ant/binaries/ and unzip into c:\ant\

Download the latest zip of maven and unzip e.g  into c:\ apache-maven-2.0.11

Set up environment variables and paths

Example Summary of relevant Environment variables ( depends on versions you have downloaded):

ANT_HOME  C:/ant

EC2_CERT  c:\ec2-api-tools\YOUR-cert.pem

EC2_HOME  c:\ec2-api-tools

EC2_PRIVATE_KEY c:\ec2-api-tools\YOUR-pk.pem

JAVA_HOME  C:\Program Files\Java\jre6

MAVEN_HOME c:\apache-maven-2.0.11

Example path variables:

…….  c:/ant\bin;C:\Program Files\Java\jdk1.6.0_23;c:\apache-maven-2.0.11\bin;C:\ec2-api-toolsbin;c:\bamboo-elastic-agent\bin

Create a  batch file that  consists of two lines:

Line 1: the Java classpath (This was obtained by using a simple Powershell script to scrape the lib folder under bamboo-elastic-agent  and setting the result as  a CLASSPATH  variable)

Line2: This will run the actual elastic-agent

Note there are over a  hundred jar files but to date Atlassian have been unable to let me know which ones are actually needed hence the snippet of the batch file I used  rather than the full list.

SET CLASSPATH=acegi-security-1.0.4.jar;activation-1.1.1.jar;activeio-core-3.1.0.jar;activemq-core-5.2.0.jar;activemq-ra-5.2.0………

java -server -Xms32m -Xmx512m -XX:MaxPermSize=256m  -classpath %CLASSPATH% com.atlassian.bamboo.agent.elastic.client.ElasticAgentBootstrap 2>&1 > c:\bamboo-elastic-agent\bamboo-elastic-agent.log

Test that everything is set up okay by running the batch file interactively. You should see output similar to:

………  Java.io.FileNotFoundException: http://169.254.169.254/2008-02-01/user-data
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
at java.net.URL.openStream(Unknown Source)

……….

oo.agent.elastic.client.ElasticAgentBootstrap

0 [main] INFO com.atlassian.bamboo.agent.elastic.client.ElasticAgentBootstrap  –

Using tunnnelling. Registering ‘httpt’ and ‘httpst’ protocols.

577 [com.sun.sungrid.service.tunnel.server.TunnelServer] INFO com.sun.sungrid.service.tunnel.server.TunnelServer  – Waiting for tunnel connection.

The key points are it trying to get userdata and attempting to create the tunnel. The agent needs to be started by the bamboo controller hence the errors

The agent needs to start automatically on starting the instance so the batch file  needs to be wrapped as a service . The nssm works well https://iain.cx/src/nssm/

Set the service to start automatically:

Create an AMI based on this instance

Log onto Jira Studio and  register the AMI, Set up the  capabilities , test that it will spin up the instance  and  test a basic build .

Jira Studio recognises both Visual Studio and msbuild so you just need to add the paths in when setting up the capabilities:

 

 

Advertisements

2 comments

  1. Cyril Monpellier · July 8, 2011

    This is a great post and was able to follow it. Unfortunately when I run the batch file to test the bamboo agent, I get the following :
    Exception in thread “main” java.lang.NoClassDefFoundError: com/atlassian/bamboo/agent/elastic/client/ElasticAgentBootstrap
    Caused by: java.lang.ClassNotFoundException: com.atlassian.bamboo.agent.elastic.client.ElasticAgentBootstrap
    at java.net.URLClassLoader$1.run(Unknown Source)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    Could not find the main class: com.atlassian.bamboo.agent.elastic.client.ElasticAgentBootstrap. Program will exit.

    my studio is at 3.0.4-studio-3 and there is a bamboo build for that so I downloaded the 3.0.3 elastic agent… Do you know what could cause the error ?

    • Grace Mollison · July 9, 2011

      Hi Cryil,
      Thanks
      You may have missed out including one of the jar files .
      Feedback I’ve received about Studio 3.0.4 is that there may be some sort of mismatch with the elastic agent and bamboo with this version of Jira. If you mange to get a success with the tunnel test but get further errors such as “FATAL [Thread-2] [AgentBootstrap] Exiting due to fatal exception.
      org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘eventManager’ defined in class path resource [applicationContextRemoteAgentStartupCore.xml]: Cannot create inner bean ” I would encourage you to log a call with Atlassian to get them to devote some effort to sorting the mismatch out f it’s not already sorted out.
      Good luck

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s