Previously, I extolled the virtues of using Jenkins as both a continuous integration platform and a deployment tool. This time I outline some of the flexibility within the system to support new tools.
Rsync is an open source utility designed to synchronize files in two separate locations whether local or remote. I use the word synchronize for a specific reason- this is not a copy utility. Rsync’s primary benefit is after the initial copy process; the rsync protocol allows for efficient synchronization of the locations after the copy.
It achieves this by way of a delta encoding protocol. Don’t worry, I won’t bore you with the mundane protocol details. The important part is that rsync works to ensure that only the changes are transmitted from source to destination. It is smart enough to only transmit the bits of files that changed; if a small change is made to a 2GB file the entire file would not be retransmitted.
I should add something else here: even though rsync is open source and built for unix, through the miracle of Cygwin it runs perfectly fine in the Windows environment.
Earlier I extolled the virtues of Jenkins as, among other things, a deployment platform (http://blogs.mybridgepoint.com/jenkins-from-ci-to-production-deployments/). At BridgePoint, rsync is a part of this process. We use rsync as the tool for deploying compiled web applications from the build server to the application’s intended environment.
This does require a small amount of setup because our infrastructure is nearly 100% Windows. Rsync is installed as a client application on the build servers, and rsync is running as a service on the hosting servers. The overall process for using rsync in this capacity is:
- Configure the Web Application with a publish profile configured for a directory on the build system
- When Jenkins builds the application, the results are published using the specified profile
- After successful publishing, rsync is used to synchronize the server’s application directory with the build servers locally published update
The key to configuring Jenkins to publish the application after build is to pass the appropriate parameters to the MSBuild plugin in the Build section of the Jenkins project:
The switches /p:DeployOnBuild=true and /p:PublishProfile=QA are the switches that accomplish this. DeployOnBuild does what it says, and PublishProfile should be set to the name of the profile that specifies the build servers local path.
A brief word about security: rsync has security features built-in, however I would highly recommend using security layers external to rsync such as firewalls or SSH tunnels to further control access. Rsync running as a service has the potential to allow a network client to access and rewrite the content of the application, including critical configuration files.
In the Post-Build actions, we define a Windows batch command that invokes rsync to perform the synchronization from the locally published output to the server.
The command is in the format:
rsync <options> <source> <destination>
Rsync will add files to and change existing files within destination as needed to synchronize the two paths. There is some specific notation in the source and destination that should be noted.
Cygwin is fundamentally a wrapper that provides a unix environment inside windows for the benefit of applications built for unix. One of the defined conventions for this unix environment is that Windows drive letters will be available from the unix path /cygdrive/<drive letter>. The source path is equivalent to c:\inetpub\deployments\qa.somesite.com\*.
The destination path is in the format of server::configuration item. Server is an IP address or resolvable hostname. Configuration item refers to a definition within the server’s rsync config; this definition contains information such as the local path to use as well as security and file permission options. This is done on purpose as a part of rsync’s network security: rsync running as a service can only synchronize from/to paths within the server that are explicitly defined in the configuration.
We recently deployed an update to the QA site I’m using in a scrubbed fashion for this article. This was a functionality-only change. I believe only four lines of C# changed; this was a quick bugfix. Here is the results of rsyncs run, taken from the console output for the build:
E:\workspace\qa.somesite.com>c:\rsync\bin\rsync.exe -rltDvp /cygdrive/c/inetpub/deployments/qa.somesite.com/* qa-server::somesite
sending incremental file list
sent 119,371 bytes received 7,964 bytes 23,151.82 bytes/sec
total size is 43,518,621 speedup is 341.76
Out of the 43mb contained in the entire site, rsync only needed to send 119k in order to deploy the changes to the remote server. Looking in the directory reveals that the DLL is 545kb while the PDB is 602kb. The delta encoding clearly did its job.
While we live in times where bandwidth is plentiful and typically not metered, there are still benefits from efficiently synchronizing files across networks. Less data to transmit means quicker sync times. Quicker sync times mean less time for a transient network issue to halt the process and leave the server’s application in an unknown state. This adds up to quicker deployment times. I think we can all be honest and reveal that during deployments resources need to be at the ready to react quickly in case anything goes wrong; quicker deployments reduce the amount of time spent in this waiting to act phase.
I mentioned in a previous article that Jenkins is very capable; here is a concrete example of that concept. With minimal difficulty Jenkins was taught to use a command line tool that greatly improves the process of deploying web applications to a remote server.