Wednesday, October 23, 2013

Jenkins Jobs to Deploy Artefact to Amazon Cloud Server hosting JBoss Instance

Problem background:

We develop IT solutions for clients. Mainly Java applications running on a JEE application server and connecting to a database. Development and support is done at Head office. The clients are based throughout the world. We have a Continuous Integration process hosted and running at Head office.

Our Continuous Integration Process contains the following:

  • GIT Repository
  • Jenkins Build Server
  • Maven Build Tool (Maven projects)
  • Nexus Artefact Repository

Our  clients host the production solutions on their own infrastructure. We configure the environments at project initiation time and then a typical deployment consists of only deploying a new java artefact(ear, war or jar).

We traditionally connect to our Client's server environments via a VPN connection but this time our client went for a Cloud Hosting Solution(Amazon).


My usual advice to my colleagues that are attempting to build any Jenkins job is, to first get the process working manually. If you are able to execute the process manually, then building a Jenkins job is very focused and you are able to break down your specific problem area. It is also a form of decoupling the problem.

When I tried to execute my process manually for this particular scenario, I have a few issues. I was provided with a security key file, a hostname and a username. I tried to use the : ssh -i <path_to_key> <user>@<Cloud_server> command but it kept prompting me for a password. I initially thought that it was looking for a passphrase for the security key but I was assured that there was no passphrase. The problem turned out to be that the security keys were in .ppk format. Someone converted the .PEM file to .ppk format so that it could be loaded into Putty(Telnet, ssh, ... ,keygen utility). My problem was that I use a MAC and use the native ssh client which requires a .PEM file and not a .ppk file.

I tried to convert the .ppk file back into .PEM with no success. It was easier asking the server administrator for the .PEM file.

The Solutions available:

Open Up Ports to the Cloud server 

The first option available is getting ports opened on the Cloud server to the internet. The advantage of this approach is that we can create our deployment job on Jenkins using the JBoss CLI utility to deploy the artefact onto the application server. I usually just create a "free-style software project", use the "wget" command to download the artefact from Nexus and run the JBoss command line utility to deploy the application. The other option is to use the Maven jboss-as plugin to do the deployment. This would work well if you are deploying a Snapshot version built from a fresh maven install. The only other snag that you might encounter is, to configure Proxy settings if connecting to the internet using a Proxy Server.

The disadvantages are far too many in my opinion. Our Internet connection is not that great. This means that the JBoss CLI utility can timeout. The next problem is that we need to secure the JBoss server. Remember now, everyone on the internet has access to the administration ports on JBoss, which is a huge security risk. One way of countering this issue, is to only open up access to these ports to specific public IP addresses. Not to forget, someone needs to keep account of which ports have been opened up and to which IP addresses are they accessible. Migrating your Jenkins Server to an alternative IP address can easily turn into a nightmare

The simplicity to this mechanism is that, you can build your Job in the same way as you would connect to any other server hosted in your own network.

SSH into the Server

The other option is to use the Publish over SSH Jenkins plugin.
Create a "free-style software project". Use the "wget" command to download the deployment artefact from Artefact Repository(Nexus) into the Job's workspace. Use the ssh plugin to copy over the artefact to the Cloud server. Once the artefact is on the Cloud server,  use the same plugin to execute a script hosted on the Cloud server. This script needs to either copy the artefact into the JBoss server's deployment directory or executes the JBoss CLI utility to do the deployment. The Jenkins Publish over SSH plugin is capable of both connecting via ssh and securely copying files or scp.

One disadvantage of this mechanism is that the script that is executed on the Cloud server should be able to return both positive and negative results. In this way the job can report successful or unsuccessful execution of the project. Another disadvantage is loading and maintaining the SSL keys.


I am pretty sure that if we had a perfect internet connection I would have had a much simpler life. Unfortunately this is IT, always have a plan and a backup plan!

This is my first Blog Posting and I would love to hear your opinions or questions.

No comments:

Post a Comment