Parameterised Scheduler Plugin

“VJ, I want you to run the automation job on all our three environments on a nightly basis. So that we can compare the results & find out any discrepancies across environments.”

My team lead said this to me one day out of the blue.A small background on this – we have automation scripts used to test Rest APis on our cloud. This normally runs on a single environment every night, mainly pre-prod.Now the requirement was to run the same set of scripts on all environments i.e staging/dev, pre-prod & prod.

This was a new and exciting task for me and I got on with it right away!

Till now, I only knew how to schedule a job periodically as I had earlier written a blog post regarding it. But I did not know how to schedule the same job, multiple times with different parameters.

I found exactly what I needed in the Parameterised Scheduler Plugin!!

So once you install this plugin,restart the jenkins application for it to take effect. Very important. I tried scheduling it without restart and the plugin did not work.

Here’s how you can schedule the same job multiple times throughout the day with different parameters:-

Build_periodically_with_parameters

The % symbol separates the cron notation from the parameters(if any) that you want to give for your run. ‘env’ is my parameter which my scripts pick up.

Another example of scheduling the same job with multiple parameters :-

Scheduling_With_Multiple_Params

The parameters have to be separated by a ; for it to take effect. Here ‘env’ and ‘param’ are 2 params that my automation scripts accept.

So there’s the solution for it.Hope this post was helpful to some.

I love Jenkins for it’s plug-ability & wide range of solutions to commonly faced problems/requirements. Simply  superb!

Alright guys.. Have fun… Enjoy 🙂

This is VJ signing off!

 

 

 

sudo /etc/init.d/jenkins start not working

I had installed jenkins and builds were working fine.

Suddenly it stopped working. There is nothing in logs

I tried starting it, with the following command :-

sudo /etc/init.d/jenkins start

but still it’s not running

root@localhost:$# service jenkins restart
 * Restarting Jenkins Continuous Integration Server jenkins [ OK ] 

root@localhost:$# service jenkins status
Jenkins Continuous Integration Server is not running

After some googling I was able to get to run jenkins with the foll command :-

-Djava.awt.headless=true -jar /usr/share/jenkins/jenkins.war –webroot=/var/cache/jenkins/war –httpPort=8080

Still not able to get why the conventional Jenkins start command is not working. I’m working on ubuntu 14.04 version. So maybe there is some issue with Jenkins-Linux combination. To get it working is enough for the time being!

Jenkins ERROR: Failed to parse POMs : java.io.IOException: remote file operation failed: Caused by: java.lang.NoClassDefFoundError:

Hi all,

I have recently faced a peculiar error with my Jenkins+Selenium Integration.The scripts which were running fine for many weeks suddenly started failing.The scripts would not even get invoked.

Initially before invocation of the scripts I got a failed to parse POM error. How could that happen when I have’nt changed the setup at all myself??

This was the error I saw in the Jenkins console logs :-

parsing pom error-Jenkins

Anyhow after reading a few Stack overflow posts, I got a few clues that it must be somewhere related to the default Maven installations done by Jenkins in the respective slaves.

In the Jenkins slave default path I saw there are few jars/ packages that Jenkins configures/copied on to the slave :-

Temp Maven Files copied on to Jenkins Slave

The tools folder contains the following Maven Package :-

Temp Maven Files copied on to Jenkins Slave- inside Tools Folder

I deleted the 3 jars(maven3-interceptor-commons.jar, maven31-agent.jar, maven31-interceptor.jar) and the complete ‘tools’ folder and once again re-run my automation job.

And… Voila!! The scripts were invoked fine once again! 🙂 All those files were again copied fresh to the Jenkins Slave once again.

So it looks like maybe one of the Maven jars got corrupted for some reason and were breaking the execution. The reason this could have happened is still hazy to me. But atleast we have a solution for it :).

Let me know if you come across such an error and if you were able to resolve it with this post.

Regards,

VJ

Useful Jenkins Plugins: JobConfigHistory Plugin

This plugin comes in handy while creating/modifying Jenkins jobs. Once you install this plugin, there is small ‘spanner’ symbol next to the individual builds run

Screen Shot 2015-04-27 at 5.45.05 pm

Clicking on this spanner icon will show you an xml comparison between the configurations before and after the change on the respective jenkins job.

Screen Shot 2015-04-27 at 5.41.19 pm

Also there is a button called ‘Restore this configuration’ which will revert the configuration to the previous one. This comes in very handy while designing CI pipelines where in frequent changes/additions are made to the build process.

This is a must have plugin if you are a Jenkins user. 🙂

Hope this is helpful 🙂

Regards,

VJ

Adding Coverity Reports to Continuous Integration pipeline using Jenkins

Hi all,

A recent interesting activity assigned to me at work, once again brought out the mechanic in me 🙂 I’m the sort of guy who likes to get his hands dirty with a various tools to get my tasks done and I was excited and rearing to go!

Task : Query the coverity server after every coverity build and display a table of outstanding issues (consisting of high, medium & low impact defect count in various modules) within the Jenkins email notifications.

Hmmm… This is a simple one I felt.. The overconfident me 😉

I quickly jotted down my plan!!

  • Query Coverity Server to get the list of outstanding issues.
  • Export queries to excel sheets
  • Use some simple logic to get total impact count of issues
  • Append table & send mail!

Though the task on the outset looked like a fairly simple one, on closer evaluation it seemed that there were some challenges in it. For a start the coverity server did not have a good query mechanism provided by means of %installation_folder%/bin/cov-manage-im.exe. The query gave us a list of issues with 8 fields i.e  stream name, cid, checker, status, classification,  owner, severity & action.

There was no field which stated the ‘Impact’ of the issue! How could this happen I thought. I tried querying the server in many different ways. No good! To make things worse I was not able to get the filenames in the same query along with 8 other fields.

The overconfidence melted completely after spending almost a day experimenting with various queries and not finding the optimum one… The best I could get was 2 queries  :-

  • Query 1 : Gave me a csv with list of files with the above mentioned 8 fields

“C:\Program Files\Coverity\Coverity Static Analysis\bin\cov-manage-im.exe” –mode defects –stream %stream_name% –host %hostname% –port 8080 –user%username%  –password %password%  –show –status New –status Triaged –action “Undecided” –action “Fix Required” –action “Fix Submitted” –action “Modeling Required” >%path to csv%\detailed_query.csv

8fields_query

  • Query 2 : Gave me a list of filenames one below the other.

 “C:\Program Files\Coverity\Coverity Static Analysis\bin\cov-manage-im.exe” –mode defects –stream %stream_name% –host %hostname% –port 8080 –user %username% –password %password% –show –fields file –status New –status Triaged –action “Undecided” –action “Fix Required” –action “Fix Submitted” –action “Modeling Required” > %path to csv%\issues_filenames.csv

Filenames

From csv1(Query 1) we get a Column named “Checker” with various fields in it. There is a reference document provided in C:\Coverity\Coverity Static Analysis\doc\en\cov_checker_ref.pdf. This gives a mapping of the impact for the given Checker Field.

So with the help of these 3 files I was able to create a summary report something like this. I filled in the Legacy defects part and this will serve as a benchmark for further builds.

Summary_Report1 (1)

Suspense!!  – I will not reveal all my secrets of how I proceeded on my job from this point onwards. It is quite obvious that I used some sort of scripting language to automate my tasks namely :-

  1. Run queries to get 2 csv files.
  2. Compare the checkers with the checker ref file and assign an impact to it
  3. Use some logic to get the final count of high impact, medium impact and low impact issues.
  4. Populate the summary report which for me was in excel(xls) format.
  5. Embed this report within Jenkins email notification
  6. Send mail 😀 😀

I assure you that the learning curve is higher when you figure out how to do these tasks…Each in one’s own style will be able to innovate and find awesome solutions :).

This task sure was a reminder to me that challenges in IT are not how it seems. It might be a simple thing that you might be asked off. Never be overconfident or underconfident … Just wear a smile on your face and say “Yes.. I will do it…After an initial evaluation of the task” 🙂

You’re generous feedback and comments are always welcome. If you run into any roadblocks do let me know.

Regards,

VJ

Jenkins Environment Variables

The following variables are available to shell scripts

BUILD_NUMBER
The current build number, such as “153”
BUILD_ID
The current build id, such as “2005-08-22_23-59-59” (YYYY-MM-DD_hh-mm-ss)
BUILD_DISPLAY_NAME
The display name of the current build, which is something like “#153” by default.
JOB_NAME
Name of the project of this build, such as “foo” or “foo/bar”. (To strip off folder paths from a Bourne shell script, try: ${JOB_NAME##*/})
BUILD_TAG
String of “jenkins-${JOB_NAME}${BUILD_NUMBER}“. Convenient to put into a resource file, a jar file, etc for easier identification.
EXECUTOR_NUMBER
The unique number that identifies the current executor (among executors of the same machine) that’s carrying out this build. This is the number you see in the “build executor status”, except that the number starts from 0, not 1.
NODE_NAME
Name of the slave if the build is on a slave, or “master” if run on master
NODE_LABELS
Whitespace-separated list of labels that the node is assigned.
WORKSPACE
The absolute path of the directory assigned to the build as a workspace.
JENKINS_HOME
The absolute path of the directory assigned on the master node for Jenkins to store data.
JENKINS_URL
Full URL of Jenkins, like http://server:port/jenkins/ (note: only available if Jenkins URL set in system configuration)
BUILD_URL
Full URL of this build, like http://server:port/jenkins/job/foo/15/ (Jenkins URL must be set)
JOB_URL
Full URL of this job, like http://server:port/jenkins/job/foo/ (Jenkins URL must be set)
SVN_REVISION
Subversion revision number that’s currently checked out to the workspace, such as “12345”
SVN_URL
Subversion URL that’s currently checked out to the workspace.

Jenkins : Passing a parameter from one job to another

Hey all,

I had this interesting thing to do :-

Requirement: To pass a parameter from one Jenkins job to another(downstream job) & to display this parameter as part of the second job report.

 Solution:After reading few threads on the solution, I inferred that the plugin that would do the job for me was this “Parameterized Trigger Plugin”. –> https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin

Once this plugin is installed , go to Job 1(upstream job) > Post-Build Actions > Select ”Trigger parameterized build on other projects”

The option I chose is given below :-

Screen Shot 2015-02-09 at 4.04.49 pm

I had to send the ‘Build_Number’ of this job to the next. So assigned to a variable ‘Temp’.

In the Downstream Job: To access this parameter use $Temp where you need it. In my case I used in the Editable Email Notification.

There are many other ways of assigning parameters from one job to another using this plugin. You can explore them. This one worked for me like a charm so thought I’d share my 2 cents :).

Regards,

VJ