Redirecting port 80 on Linux

I recently installed a JEE application and wanted it to run on port 80 to simplify the URL. For security reasons I didn't want the process to run as root. When I tried to change the application servers config to attempt to bind port 80 I would receive this error:
Non root user cannot bind to port 80

After doing some research, I finally found the solution I decided to move forward with on Server Fault.

This allowed me to use iptables to redirect port 80 requests to port 8080:
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

Now I'm a happy admin as I get the security of not running the process as root and the pretty URL that doesn't require specifying the port!

Automated Merges with Rational Team Concert (RTC)

When building software there is almost always a need to maintain multiple versions of the code base. This provides the ability to do bug fixes on the production code while still moving ahead with new development for the next release. In RTC this is done using Streams. Streams are similar to the branching concept that many other version control systems use, it just adds some additional capabilities.

I'm going to describe a quick use case that will hopefully better explain the problem that the auto merge is trying to solve. Let's say we have a production release, version 1.0. The next release will be 2.0. A bug is found in release 1.0 that we want to fix and get out into production asap. When the 1.0 release was built we only had 1 stream setup in RTC and were creating new snapshots with each build. However since the 1.0 build was done some 2.0 features had already be delivered into the stream. Because of that we cannot just fix the bug and cut a new build as it will also include the 2.0 features. So we create a new stream, 1.0 Maintenance Stream, using the snapshot that was created for the 1.0 release build. Now we have two streams that will allow us to fix 1.0 and add new features to 2.0. The problem we have now is we want all bug fixes added to 1.0 to also be added to 2.0. We can do this manually by delivering the change sets to both streams, but could be potentially tedious and time intensive if we have a lot of changes in 1.0.

This is where the auto merge comes in. This tool can be used to automatically attempt to deliver change sets into the 2.0 stream that were delivered into the 1.0 stream. The tool does attempt to auto resolve conflicts, but is very limited and will fail if there are any conflicts that will need to be merged manually.

Here is a basic overview of how the auto merge works. We use the Jazz Build Engine (JBE) to do the merge. Two repository workspaces are required. The first repository workspace has its flow target set to the "from" stream or release 1.0 in the example above. The second repository workspace has its flow target set to the "to" stream or release 2.0. The build definition uses the first repo workspace to detect changes in the 1.0 stream that need to be merged. The second repo workspace is then used to accept the change sets from the 1.0 stream into a 2.0 workspace and then deliver these change sets into the 2.0 stream.

Setting up the auto-merge.sh shell script
The first steps will be to copy the auto merge shell script onto your JBE server somewhere. Don't forget to chmod the script to be executable by the Linux user used to start the JBE.

Here is the source for the shell script:

#!/bin/bash
#######################################################
#
# auto-merge
#
# Author: Brian Freeman (www.befreeman.com)
#
# Merges from the fromStream to the toStream
#
# We have a fromStream a toStream and a repoWorkspace.  The repoWorkspace
# should be flowing from the toStream as this is where we want to deliver the changes
#
# The build definition should be using a different repository workspace that is flowing
# from the "From" stream.  As it needs to detect changes from that stream in order
# to determine if a merge is needed
#
# The script does the following:
# 1) login to RTC
# 2) change-target to the toStream
# 3) accepts changes in the repoWorkspace from the toStream
# 4) loads the repoWorkspace into the fileWorkspace
# 5) change-target on the repoWorkspace to the fromStream
# 6) accept changes from the fromStream to pull in the changeSets that need to be merged
# 7) runs the scm conflicts command to print out any possible conflicts as this script is not
#        smart enough to auto resolve any conflicts
# 8) delivers the change sets to the to stream
# 9) change-target on the repoWorkspace back to the toStream
# 10) logout
#
#######################################################

if [ $# != 9 ]
then
echo "Error in $0 - Invalid Argument Count"
echo "Syntax: $0 [scmPath] [repository] [username] [passwordFile] [nickname] [fileWorkspace] [repoWorkspace] [fromStream] [toStream]"
echo ""
echo "scmPath       - The full path to the scm command"
echo "repository    - The url to the repository \(https://rtc.lmig.com/jazz\)"
echo "username      - The RTC username"
echo "passwordFile  - The file containing the encrypted password"
echo "nickname      - The nickname to use for login and follow on cmds"
echo "fileWorkspace - The location on the file system where the repositoryWorkspace should be loaded to"
echo "repoWorkspace - The Repository Workspace to laod"
echo "fromStream    - The stream to grab the latest change sets from in order to merge them to the to stream"
echo "toStream      - The stream to deliver the newly loaded change sets"

exit 1
fi

scm=$1
repository=$2
username=$3
passwordFile=$4
nickname=$5
fileWorkspace=$6
repoWorkspace=$7
fromStream=$8
toStream=$9

echo "scm               = $scm"
echo "repository       = $repository"
echo "username        = $username"
echo "passwordFile    = $passwordFile"
echo "nickname         = $nickname"
echo "fileWorkspace   = $fileWorkspace"
echo "repoWorkspace = $repoWorkspace"
echo "fromStream      = $fromStream"
echo "toStream         = $toStream"

# Unfortunately the current RTC command line for RTC 2.0.0.2 does not provide a
# way to encrypt the password. This causes a problem as we don't want to be
# delivering code that contains a password. To avoid this we'll have to setup a
# password file ourselves.
# To create a password file:
# 1) Create a one line file containing the password called secrets.txt
# 2) Run this command:
#        openssl aes-256-cbc -a -salt -in secrets.txt -out .rtcPasswd
# 3) The openssl command will prompt you to specify an encryption password.
#    This password will be he password required to decrypt the password file.
#    The automated merge script will use the nickname parameter passed into the
#    script as the password.  So the nickname provided when launching the script
#    will need to match.
# 4) Delete secrets.txt
#
# decrypt the RTC user password
special=$(openssl aes-256-cbc -d -a -in $passwordFile -pass stdin << END-SESSION
$nickname
END-SESSION)

function failure {
echo $1
$scm logout -r $nickname
exit 1
}

function changeTarget {
echo "##### Changing flow target to: $1 #####"
$scm change-target workspace -r $nickname "$repoWorkspace" "$1"
if [ $? != 0 ]
then
echo "Failed changing flow target"
fi
}

# log into RTC and creates a session using $nickname
echo "##### Logging into RTC as $username #####"
$scm login -r $repository -n $nickname -u $username -P $special
if [ $? != 0 ]
then
echo "Failed to Login to RTC, please check the username and passwordFile"
exit 1
fi

# make sure our repo workspace's flow target is at the right place
changeTarget "$toStream"

echo "##### Accepting changes in the repoWorkspace: $repoWorkspace #####"
# accept all the latest changes into the repoWorkspace
$scm accept -r $nickname -t "$repoWorkspace" --flow-components --verbose
if [ $? != 0 ]
then
failure "Failed accepting the latest changes into the repoWorkspace: $repoWorkspace"
fi

echo "##### Loading repoWorkspace: $repoWorkspace #####"
# load the repoWorkspace that should be flowing from the TO stream
$scm load -r $nickname -d "$fileWorkspace" "$repoWorkspace" --force --quiet
if [ $? != 0 ]
then
failure "Failed to load the repoWorkspace the workspace: $repoWorkspace"
fi

# Change the repoWorkspace flow target to the "from" stream so we can accept those incoming changes
changeTarget "$fromStream"

echo "##### Accepting change sets from fromStream($fromStream) into the repoWorkspace: $repoWorkspace #####"
# accept all the latest changes into the repoWorkspace now that we're
$scm accept -r $nickname -t "$repoWorkspace" --flow-components --verbose
if [ $? != 0 ]
then
changeTarget "$toStream"
failure "Failed accepting the latest changes into the repoWorkspace: $repoWorkspace"
fi

$scm conflicts -r $nickname

# Change the repoWorkspace flow target back to the "to" stream before delivering 
changeTarget "$toStream"
if [ $? != 0 ]
then
failure "Failed to set the flow target back to the \"TO\" stream"
fi

echo "##### delivering accepted changes changes #####"
# deliver all the accepted changes
$scm deliver -r $nickname -d "$fileWorkspace" -v -s "$repoWorkspace"
if [ $? != 0 ]
then
failure "Failed to deliver the changes"
fi

echo "##### Logging out of RTC #####"
$scm logout -r $nickname

The script starts off by logging in the provided user into RTC using an encrypted password. You'll need to follow the steps described within the script to create the encrypted password file. Once you have the password file you'll need to place it in a location on the JBE server that will be readable by the Linux user used to start the JBE. It is also recommended that the permissions on this file are setup as read only for just the user used to start the JBE.
Security Note: This technique simply hides the password from being easily viewable in the build definition and in a plain text file on the system.

The sections that follow will describe the steps needed to setup the script in RTC. The instructions were created using RTC 2.0.0.2. Theoretically they should work in RTC 3.0 as well, but I haven't verified it.

Creating the "from" stream Repository Workspace
This is the repo workspace used only to detect changes were made in the "from" stream (release 1.0).

1. Right click the Source Control folder under your project area in the Team Artifacts view in Eclipse
2. Right click the stream you wish to merge the code from and select New -> Repository Workspace
3. Specify the repo workspace name and description. I typically like to have a name similar to the following: Auto Merge My Project Stream: 1.0 to 2.0 - Detect Changes In 1.0 - Workspace. Once you're finished press the "Next" button.
4. As for the Read Access Permission, this is totally up to you. I typically like to scope mine so that only the project area can see it. Make your choice and press the "Next" button.
5. For components, you're typically going to want to merge all the components of the stream so I'd make sure all the components are selected and then press the "Finish" button.
6. Now that the new repo workspace has been created you may need to change the ownership to match the user used by the JBE.

Creating the "to" stream Repository Workspace
This is the repo workspace used to actually do the merge work.

1. Create this new stream by following the same steps described for creating the "from" stream. Except this time right click on the "to" stream instead (release 2.0 stream).
2. The ownership of this repository workspace will need to match the RTC username used for the rtc.username property mentioned later in the build definition instructions.

Setting up the Build Definition
The next steps are to setup the Build Definition.

1. Right click the Builds folder under your project area in the Team Artifacts view and select "New Build Definition..."
2. Select the project area you want the build definition under by pressing the "Browse..." button
3. Select "Create a new build" and press the "Next" button
4. Enter the ID and Description, then select the "Command Line - Jazz Build Engine" build template and press next
5. Check the "Jazz Source Control" checkbox on the "Pre-Build" window and press the "Next" button
6. Press the "Next" button on the Post-Build window
7. Leave all three options checked on the "Additional Configuration" page and press the "Finish" button

You should now see a new tab in eclipse for the Build Definition

1. Select the build engine you want to use to run this build definition
2. Select the "Schedule" tab and setup how often you want to run this automated merge. We have been using every two hours.
3. Switch to the "Properties" tab and add the following properties
   

   build.dir	path on the build engine where the streams should be merged
   fromStream	This is a Repository Workspace property that should direct you to the Stream you wish to merge changes from. In the example above, this was the 1.0 stream.
   rtc.pwdFile	This is the password file the shell script will use to pull the encrypted password from
   rtc.username	This is the RTC username used login to RTC with via the command line
   scm.cmd	The full path location to the scm command installed on the JBE
   toStream	This is a Repository Workspace property that should be linked to the stream you wish to merge the changes to. In the example above, this was the 2.0 stream
   toStreamRepositoryWorkspace	This is a Repository workspace property that should be linked to the repository workspace used to perform the merge

   
   
4. Switch to the "Jazz Source Control" tab
5. Setup the Build Workspace to point to the "from" Stream repository workspace. This is the repo workspace that should have the flow target pointing to the 1.0 stream. You might need to create this repo workspace if you haven't done so already. (Note: This repository workspace will need to be owned by the user who is used to run the JBE)
6. Add ${build.dir} to the "Load directory" field under Load Options
7. Press the "Select..." button after the "Components to exclude" field and in the window that appears select all components to be excluded. Since this repo workspace is only being used to detect changes we do not need to load the workspace into the file system on the JBE.
8. Make sure both check boxes are selected in the "Accept Options" section as we want the build definition to accept the latest changes and build when changes are detected
9. Switch to the "Command Line" tab
10. In the "Command" field enter the path location to where you installed the auto-merge.sh file on the JBE
11. In the "Arguments" field enter the following:
    ${scm.cmd} ${repositoryAddress} ${rtc.username} ${rtc.pwdFile} autoMerge ${build.dir} ${toStreamRepoWorkspace} ${fromStream} ${toStream}
    The repositoryAddress is obtained from RTC, all of the others were defined on the properties tab

That should be it. Now when changes are made to the 1.0 stream they should automatically be merged into the 2.0 stream by having this build definition run the shell script.

RTC Command Line for 2.0.0.2

I've recently been working with the RTC Command Line capabilities in 2.0.0.2. For a long time I was looking at the help for the old 1.0 commands. I was happy to recently discover that I was looking at the old version and found the more update documentation for the release we're using here:
http://publib.boulder.ibm.com/infocenter/rtc/v2r0m0/index.jsp?topic=/com.ibm.team.scm.doc/topics/c_scm_cli.html

They've made some significant improvements to the commands available which have made what I'm trying to do much nicer. I'm trying to setup an automated merge capability that will merge one stream to another stream using an automated build definition. More to come soon...

On a side note, I have run into an issue with case sensitivity using the scm conflicts command. For some reason it was always complaining about not being able to login. We're using an LDAP for RTC authentication and I realized that the LDAP stores my user name in all caps. When I switched the scm login to use my username in all caps the problem went away.