Sherpa’s Blog

Subversion MultiSite Enables Agility For the Distributed Enterprise

The Agile methodology has become a wildly popular approach for software development with close to 50% of development teams using at least some agile techniques based on different surveys. The responsibility of an agile team is to deliver value in the software they are creating and meet the criteria or objectives that determine success for the project and a common set of development “best practices” have arisen that are crossdisciplinary. Adopting these best practices can be especially challenging for collaborating agile teams that are distributed or have distributed members.

The Agile principles state that successive short development cycles, popularly referred to as rapid iterations, be used to develop software quickly in little bits (pun ;) ) at a time. Different methodologies have arisen from the Agile principles like Scrum, XP, TDD, Crystal and others which define activities and process for agile teams. The process gets into the specific details for each discipline but agile teams enjoy a great amount of freedom for self organization as well as self definition which as a result teams may borrow one aspect of another methodology to apply to their own agile approach. Popular examples of this are the XP User Story or and the Scrum Backlog.

The common principles mixed together with the common practice of borrowing the best aspects of each discipline has helped popularize a set of software development best practices. These are practices which are good for any software development team but the popularity and nature of agile software development has led these practices to be commonly associated with agile specifically. Here is a list of the more common agile best practices:

  • Rapid Iteration
  • Continuous Integration
  • Parallel Development
  • Frequent Commits
  • Automated Testing
  • Code Review
  • Refactoring

If you are an enterprise considering a large scale adoption of agile software development practices then there are a number of challenges that you may be facing. Scaling Scrum is well documented in Ken Schwabers book “The Enterprise and Scrum” and he has special sections dedicated to the problems of distributed teams, distributed members and the scarce skills problem (eg: one DBA shared by many teams). Ken handles the issues associated with the organization and daily routine for Agile Scrum adopters, but what about the technical challenges especially those associated with the best practices I listed? This is an area where the value added services offered by WANdisco enter the conversation.

Using Subversion Multisite and Apache Subversion will enable teams to fully adopt an Agile practice like Scrum on an enterprise scale, especially where distributed teams or members are concerned. Here is how.

  1. Subversion Multisite instantly synchronizes development changes between locations. This means that distributed teams always work against a local source code repository, without which frequent commits, continuous integration, code reviews and parallel development might not be possible at all or at best will be severely hindered if users are required to connect across the internet to access the source code repository (for read or write activities)
  2. Multisite also supports local clustering option, an enterprise that sets up two or three load balanced Subversion servers will better handle the increased traffic from frequent commits and especially from continuous integration where automated tests and builds are being performed continuously and using up a lot of server resources.
  3. Multisite leads to improved team communication, imagine a tester working in Los Angeles getting the latest build locally almost immediately after a task is committed in New York (again locally). When each site has access to a complete replicated copy of the source code repository many things are possible and the usual downtime related to downloading a new build off a remote server or uploading a set of new changes to a remote server can be entirely eliminated.

An enterprise that is considering or has already adopted agile practices will also want to adopt the software best practices. Given the nature of agile development and rapid iterations adopting these best practices are critical and with Subversion Multisite many of the technical challenges facing distributed teams can be easily solved. There are of course other benefits to Multisite such as built in disaster recovery, guaranteed uptime that will also interest any enterprise and the value offered to Agile teams will be indispensable especially as the Agile usage scales upwards.

Intro to Subversion Access Control 4.1

Big plans are brewing at WANdisco for 2012, and one project that has just been completed, is Subversion Access Control 4.1.

Access Control 4.1 includes a revamped user interface. The new look makes it easier for users to navigate through the security features offered by Access Control and setup new rules and permissions for Subversion repositories and paths.

The ‘Group’ concept has been replaced with “Teams’ and a new feature has been added to allow the designation of “Team Leader” who is then granted some administrator privileges. Specifically, the Team Leader is able to setup user access rules to a pre-defined repository path (or paths).

We are calling this new feature Delegated Admin since it allows granting of some administrator privileges to users. The context of how this feature could be used might be best described in terms of business units for an enterprise corporation. Consider the following scenario.

A company has 100 Apache Subversion users split into 10 teams, each associated with a separate BU, each team has its own Subversion repository. In the past, Access Control admin can go in and setup repository security rules that applies to all 100 users and repositories. With delegated admin the Access Control administration can create 10 new Teams and assign a Team Leader to each one, specifying the Repository that team should have access to. From there, the Team Leader can log into the system, add/remove the users for the team, and setup multiple rules needed to properly manage those Subversion assets.

Effectively what delegated admin does is allow Team Leaders, people in charge of their respective Business Units, to define and configure their own team rules. Access Control supports the concept of nested-teams which is called Sub-Teams (see screenshot) and this would even allow you to further organize teams from larger sections into multiple sub-sections (eg: West Coast Branch –> Plugins Team; West Coast Branch –> Application Team) and designate a team leader at each level to manage users and rules.

The role of the Access Control administrator now is to simply define Team Leaders and the resources which they can direct access over. Once defined the Team Leader can only create rules that affect their resources meaning the West Coast Branch won’t be able to setup access rules for the East Coast Branch’s repositories and vice-versa.

The ability to delegate some administrator functionality to Team Leaders, and in the context of specific Subversion paths and repositories, is a powerful new feature that will empower enterprise organizations and save time and money. Team Leaders are able to set access restrictions based on the need and requirements of their given team or business unit and no longer have to rely on a single administrator to do that for them.

Subversion Tip of the Week

The Oops Command

Often when you are changing code, you will realize that you have meandered down the wrong coding path. You can take the manual approach and try to back out of your changes but Apache Subversion gives us an easier way. It is called the revert command.

If you want to have a file restored to where it was when you last checked it out, or when it was last updated, you can use the revert command to restore a file (or files). This command will appear in the context menu from your working folder if any files have been changed. Be aware that if you select a changed file and then right-click, you will only be reverting that file (or files) you have selected. To make sure you will be affecting all the files that have been modified, either right click on the top of the working folder or make sure you have selected (using the Ctrl/Key) all the files you may want to convert.

Below is an example of the first step of the revert command. Here you can decide which files to revert and which to ignore.

You can check the box of any file you may want to revert to what it looked like when it was checked out or updated. Be aware that both a files’ changes and any changed Subversion properties will be changed by the revert command.

At the bottom of the form there is a button labeled “Delete unversioned items…” This will present you with a form where any files that you have added since doing a checkout (but not added with the Subversion ADD Command) can be selected, and then deleted.

The revert command also works with folder changes. Below you see the effect of running the revert command after adding a new folder (“newfolder”) and renaming a folder (“reports” was renamed to “User_reports_renamed”).

The effect of a rename shows up as the deletion of the old folders and addition of the new (renamed) folder.

Every week WANdisco’s Head of Training Mike Lester will publish a tip to help you get the most out of Apache Subversion. Mike also delivers free, bi-weekly training webinars on topics ranging from branching and merging best practices, to Subversion in the enterprise, and advice on making Subversion agile. Register early, to avoid missing out! Enterprise Training is also available.

Where Did That Branch Go?

When teaching “Subversion User” classes, one of the frequently asked questions is “When do you delete a branch?”

Unfortunately the reason they are asking about deleting branches is because they are concerned with saving space. And as all experienced SVNers know, deleting will have no effect on repository size. Remember, when anything is deleted in Subversion, whether it is a project (folder), a file or a property, it is only deleted in the revision where it is deleted, and subsequent revisions. It still exists in all previous revisions and can be viewed or recovered at any time.

So the question remains. “When, or why would you ever delete a project?”

First let’s look at the cases when you would NEVER delete a branch:

1) Some branches are always needed such as the “trunk” or branches that contain other branches like the folders “tags”, and “branches”. This assumes you are using the TTB layout format.
2) Some branches are referenced frequently. The branches in the “tags” folders that hold your recently released versions are used to create releasable code and compare to current trunk development efforts repeatedly.
3) Branches where concurrent development is taking place. Before the branch is merged back to the trunk (or wherever it was branched from) it would be confusing to delete the branch. It might also lead to someone forgetting to do the merge.

So that leaves us with the following types of branches that MAY be candidates for deletion.

1) Abandoned work efforts. For example, you may start a branch to do concurrent development, and later decide to do the work in the trunk. What do you do with the abandoned branch?
2) Branches that represent old versions of your code. Let’s be realistic, if you are on Release 5.6.004 you may go back to Release 5.6.003 or Release 5.5.012. But how likely is it you are going to need to reference Release 1.0.01 from 4 years ago.
3) Mistakes created when using the COPY command. You meant to create a Release branch from revision 1255 of the trunk and you mistakenly selected the revision 1250.

All three of these are good examples of when you might do a deletion. But if you are going to do a deletion you should have some rules. And these rules should be part of your organization’s “Policies and Procedures”. (We actually spend considerable time in the “WANdisco Subversion Administration Class” talking about the contents and importance of Subversion Policies and procedures.)

Rule 1) Always use the log message to document that a deletion has been done and why. There is a new TortoiseSVN property (tsvn:logtemplatedelete) that provides your users with a template of the information you might want them to fill out. To see a demonstration of this property in use, see this short video. http://www.youtube.com/watch?v=foNQME4rcVQ

Rule 2) (optional) Instead of deleting folders, move them to a folder called “Deleted”. This makes the “deleted” branches/folders easier to find and recover.
Rule 3) Make sure everyone knows how to restore deleted folders/branches.

The best reason to delete any file or folder from a Subversion project is to make the viewing of the rest of the project easier. The less information you have to look at, the more likely you WON’T checkout a branch incorrectly or use the difference command with the wrong branch. Both of these actions might waste your time or create some sort of error that could go unnoticed.

An Introduction to Subversion Hook Scripts on Windows

So you recently started running Subversion on Windows? You’re keen to unlock the power of hook scripts but don’t know where to start? Well dear reader, you can start right here…

Subversion hook script?

In a nutshell, it’s a program that performs an action which is triggered by a specified repository event. You can think of hook scripts as Subversion’s tool chest for setting up tighter control of a repository or integration with third-party applications, such as auto-closing tickets in a defect tracker when someone commits a change with “Fixes #123″ in the log, or triggering builds in a continuous integration system. The most common and powerful hook scripts are server side, although it’s also possible to run client side hook scripts.

Server side hook scripts usually have the permissions of the web server, which gives them power and flexibility, being able to affect other repositories on the server – of course the flip side to this power is the risk that a badly designed hook could slow the server to a crawl or in extreme cases, corrupt the repository.

What can hook scripts do?

  • Report on events, such as emailing alerts when a commit occurs, sending a report on the commit to a third-party.
  • Test for a pre-condition before allowing a commit to occur. For example, checking that a commit has been commented, or follows coding guidelines.
  • Block certain actions. Maybe prevent a commit that contains multi-gigabyte files that could block other SVN traffic for hours, or prevent locks from being stolen.

What hook scripts can’t (shouldn’t) do

  • Hook scripts shouldn’t modify files being committed. Right now, if a hook script actually changed a commit transaction, there’s no way of communicating the change back to the client. So while it might be tempting to write a script that automatically corrects errors or applies positive changes to a committed file, it will result in two different versions of the file in the same revision one version written to the repository and another in the user’s cache. The result of this loss of consistency can be as horrible as it is unpredictable.

Where are the hook scripts located?

When a new repository is created, a hooks subdirectory is included, e.g. if you create a repository called chancode, hooks will live in the /repositories/chancode/hooks/ directory. By default a non executable template (.tmpl) for each type of hook script is automatically created with the directory.

Hooks

Hook types

  • Start-commit Invoked before a transaction is created, pre-pre-commit, if you will. Commonly used to check if a user has sufficient privileges to perform a commit.
  • pre-commit / post-commit Invoked before or after a commit is completed.
  • pre-revprop-change / post-revprop-change Invoked before or after a revision propertly is added, modified or deleted.
  • pre-lock / post-lock Invoked when a user attempts to lock a path or after a lock has been created.
  • pre-unlock / post-unlock Invoked when a user attempts to destroy an exclusive lock, or immediately after the lock’s destruction.

There’s a good source of pre-written scripts available online, although the vast majority have been written for Unix/Linux, so aren’t immediately usable on Windows. Hooks can be written in anything, although cross-platform languages like Perl, Python and Ruby are quite popular. For simple scripts, it’s possible to use Windows’ plain old batch scripting.

Testing the water

We’ll do a simple test to show a hook script in action. We’ll introduce a pre-commit hook that checks for a log message that is n characters long – in this case, at least 6 characters.

  1. On your Windows Subversion/uberSVN server navigate your way to the hooks directory in your repository.

    i.e. C:\Program Files (x86)\WANdisco\uberSVN\repositories\chancode\hooks
  2. Open the pre-commit.tmpl file – you’ll recall that this is the script that gets triggered before a commit is completed.
  3. The template file contains lots of useful information, as well as an example script that does exactly what we’re going to do, unfortunately it’s aimed at the Unix/Linux /bin/sh interpreter, so we can’t use it for Windows without a rewrite.
  4. Delete all the contents of the file and copy in the following Windows batch scripting:
    @echo off
     :: Stops commits that don't include a log message of at least 6 characters.
     @echo off  
    
     setlocal  
    
     rem Subversion sends through the repository path and transaction id
     set REPOS=%1
     set TXN=%2           
    
     svnlook log %REPOS% -t %TXN% | findstr ...... > nul
     if %errorlevel% gtr 0 (goto err) else exit 0  
    
     :err
     echo --------------------------------------------------------------------------- 1>&2
     echo Your commit has been blocked because it didn't include a log message. 1>&2
     echo Do the commit again, this time with a log message that describes your changes. 1>&2
     echo --------------------------------------------------------------------------- 1>&2
     exit 1
    

    The script uses Svnlook, the incredibly useful command line tool to examine the commit transaction’s log message, then calls upon the findstr command to look for a string of at least 6 characters – the length of the minimum message length is set by the number of “.” characters after the findstr command.

  5. Save the file so that it can be executable. Windows supports .exe or .bat files.
  6. Testing time. Make an arbitrary change in a file, commit the change with a log message that is either blank or contains fewer than 6 characters. You’ll get the following error message:

    Tortoise 01

  7. Repeat the commit with the necessary number of characters, the commit will then succeed.
  8. Of course, this is a super-simple example. Lazy developers soon learn to leave comments along the lines of ‘update update update’ or ‘blah blah blah.’ You can use more complex scripts for better checking.

Parting Tips

Debugging and troubleshooting hook problems is notoriously difficult as there’s no formal error logging. These points are worth keeping in mind when setting up hook scripts.

  • If you’re completely stuck when trying to convert a hook script for use with Windows, remember there’s always Cygwin, a set of tools for running Linux tools on Windows, which will let you run many Linux scripts on Windows without the need to rewrite them.
  • Empty environment problem – Subversion executes hook programs with no environment variables. That’s a good security precaution but often leaves administrators pulling their hair out with frustration when a manually tested hook script absolutely refuses to work with Subversion. So ensure that any necessary environment variables are set in your hook script and/or use absolute paths.

Software Designed to Facilitate Collaboration

There are lots of examples of groupware. Project management systems, bug tracking systems and version control systems. Yes, Subversion is a groupware application.

The training and standards needs for applications that are classified as groupware are different, and more necessary than other applications.

Typically with groupware applications, one person’s output is another person’s input. This means that as soon as enough users start distrusting the completeness or quality of a group’s input, they will start to devalue the use of that information.

What this means in Subversion, is they start not using the tools on certain projects, or they maintain alternate copies of the projects, or they start only inputting certain project information. All of which make the use of Subversion less valuable and results in software configuration management problems.

How Do You Avoid This?

  • Training – Make sure all the people who need to use, or make decisions, on how Subversion should be used know about the tool.
  • Policies and Procedures – Have rules for the usage of Subversion. This will include naming conventions, project standards and log message usage. Hook scripts can make following standards easier, but someone has to decide which are needed, and then implement their use.
  • Management – If someone violates the polices, show them the correct way to do something. Evaluate your workers on how well they follow standards. Do not allow exceptions. “This is a small/quick/unimportant project so I will do it outside of Subversion”.

Hopefully the idea of training and standards in a highly technical environment is not a new idea. The manner in which you train your developers on the capabilities and usage standards for Subversion is less important than the fact that you do some training. The training could be standup, online interactive, or one-on-one mentoring. But don’t kid yourself, training is necessary.

It would be nice to think that your workers will take home the Subversion/TortoiseSVN manual and read it over the weekend. But ask yourself this question: have you done that?

WANdisco has spent years helping companies use Subversion in an effective manner. We’ve seen all the mistakes and we can help you avoid them.

Michael Lester
Director of Training, WANdisco

How to Install Subversion on Windows 7

A beginner’s guide to installing Subversion on Windows 7 using WANdisco’s easy-to-use uberSVN, which for the first time opens up the power of Subversion version control to people who are not computer boffins.

1) Check that your computer is up to the task.

  • Pentium 4 3.0 GHz or higher.
  • 1.5GB of RAM (2GB+ recommended) RAM should be split: 256MB to Apache and 1GB to Tomcat.
  • If running additional apps, such as Jenkins, 2GB of system memory should be considered a minimum, with Tomcat given no less than 1GB.
  • 200MB of disk space for uberSVN (not including space for your Subversion repository data).
  • TCP/IP networking must be configured and working.
  • Subversion itself isn’t particularly demanding, although uberSVN lets you quickly add additional applications and services, which can quickly gobble up a basic system’s resources.
  • Your computer looks up to the task? Good stuff, then it’s time to Download uberSVN.

2) Run the installer, making sure that you have admin permissions.

3) If an Open File prompt appears, click the Run button.

4) The Welcome to uberSVN Setup Wizard starts. Just click Next>.

5) Confirm where you want uberSVN to be installed. The default location works for most people.

6) Wait until all the files have been extracted. Click Next >.

7) The installation’s completed. There’s a start option; do you want uberSVN to automatically start up whenever your computer boots up? It’s a good idea to leave this ticked if you want Subversion to be available to users without you needing to start it up every time the system restarts. However, you can easily change your mind either way, later. To move on, click on the big link to go through uberSVN’s settings.

8 ) The first uberSVN setup screen (Installation Settings) will open in your browser. This screen shows you where uberSVN has been installed (Installation Directory), where uberSVN will store your repository data (Repository Storage Location) and the uberSVN key. If you don’t have a key, follow the link to get one (don’t worry, it’s free). Click the Next button to continue.

9) Time to create the Admin User’s account. Enter your full name, your email address and a username which you’ll use to login to uberSVN. Provide a password, then re-enter it. To continue, click the Next button.

10) Some technical stuff next, but don’t worry, the default values will work fine. You can come back later to change these values, especially the BASE URL. See the Tips section about how and why you should change ‘localhost’ for your computer’s actual IP number. Click the Next button to continue.

11) Final step. Woohoo! You can set uberSVN up to send out emails to users. You need an email server that will accept mail using SMTP (Simple Mail Transfer Protocol). Again, you can select ‘No’ and come back to these settings later after uberSVN is setup. Click the Next button.

12) uberSVN will now take your settings and apply them. This will take a minute or two. Time enough for a coffee, maybe? When the setup is complete you’ll be able to click on the Start using uberSVN button.

13) Starting uberSVN will open a browser window that shows a login prompt. Enter the Admin username and password that you provided in step 10, then click Login.

14) Congratulations, you just installed uberSVN. You are now running Subversion.

Tips

During installation you’ll be asked to confirm the BASE URL of your computer. By default, the installer will use the short-hand address ‘localhost’ which on a network simply points back at the local computer. However, this won’t work for anyone looking to use uberSVN from a different computer – it will point to their own computer instead of yours. The solution is to get your computer’s IP address. Here’s how:

  • Click Start
  • In the search window, enter cmd, then press the Enter key.
  • A trusty DOS style command window will open up. ‘Old School!’
  • Type in ipconfig then press the Enter key.
  • Pick out the IPv4 Address for your network card, if you have multiple network adaptors, it’s likely to be the top one. The address will have the format xx.xx.xx.xx, four numbers separated by full stops.

How to Install TortoiseSVN and Make Your First Repository Change

This is an introduction to Subversion, using WANdisco’s uberSVN and the trusty Windows client, TortoiseSVN. In fact, you can think of this tutorial as a set of TortoiseSVN water wings, aimed at developers who have been thrown into Subversion at the deep end. As well as getting TortoiseSVN installed, it’ll show you how to get your working copy in place and your first changes made to the repository.

1) Download the latest version of TortoiseSVN

http://www.wandisco.com/subversion/download#tortoise

2) Double click the installer file.

3) Click the Run button.

4) You’ll see the welcome screen, which will confirm the version of Tortoise that you’re about to install. Click Next >

5) Read the End-User License Agreement. If you’re happy to proceed, click the ‘I accept the terms…’ radio button and then click Next >.

6) The Custom Setup screen lets you deselect various elements of the install. Unless you are really low on disk space, it isn’t worth worrying about this. When you’re ready to continue, click Next >

7) Click Install.

8 ) Hey presto! Tortoise is now installed. Click Finish.

9) To work with a Subversion repository we first need to download a local copy (called a working copy) of either the full or part of the repository. We start by creating a directory in which we’ll store the working copy.

10) Open Windows Explorer, go to the new directory and right click within the folder space. The Explorer context menu appears, along with Subversion commands that are added with TortoiseSVN. Click on the option ‘SVN Checkout.’

  • URL Repository: This is the URL of the repository on which you want to work. This is the address of the repository, or portion of a repository for which you want to make a working copy. You can get the repository URL from the repository’s screen in uberSVN.
  • Checkout directory: This is the location on your local file system where the checkout will occur. By default it will be the location that you’re currently focused on in Explorer, however you can change it here.
  • Checkout Depth: The Checkout Depth drop-down allows you to limit what you checkout. Useful if you only want to make a change in one directory of a very large repository.
  • Revision: You can get the latest version of the repository, called ‘HEAD’ (most up-to-date version), or select a specific revision to go back to a version before particular changes.
  • When you’ve finished with these options and you’re ready to do the checkout, click OK.

11) You’ll now see the progress of the checkout. All files and folders that are included in the checkout will be logged.

12) Next, it’s time to make a change. The working copy is now in place on your computer, ready to edit and change to your heart’s desire. One of the benefits of Tortoise as a shell replacement is the fact that it can overlay repository files with status icons that tell you the current state of each file.

13) We’ll now make a change to a file. In this example the file ‘system.txt’ will be modified and saved.

14) You’ll notice that once saved, ‘system.txt’ now has a different icon overlain, showing an exclamation mark in a red circle to denote that it has been changed.

15) Having tested the change, it’s time to add it to the repository. To apply changes to the repository that you’ve made in your local copy you need to use the SVN Commit command. You can select individual files or their directories, then right click to get the context menu. On the menu, click SVN Commit….

16) You’ll be prompted for a message to associate with the changes you’ve made. It’s human nature to try to leave this blank because you’re in a hurry and can’t be bothered, but simple log messages can really help with troubleshooting problems at a later date.

17) The outcome of the commit will appear in a dialog window. If the commit was successful the repository’s revision number will increase. Click OK. That’s it, you’re working with version control.

18) Next we’ll add something new to the repository. You see, it’s not possible to commit something unless Subversion already has a record of the file you are adding. So first, right click the file and choose Add… from the Subversion commands.

19) The added elements should now appear in Explorer with the ‘added’ overlay icon. Although Subversion now knows about the file, you still need to perform a commit. So again, right click the file and choose SVN Commit… from the menu.

20) It’s always good practise to leave a note, whenever you do a commit. While Subversion gives you lots of tools for scrutinizing file changes, it’s nice to have a quick summary to look back on. To complete the commit, click OK.

21) You’ll get a log of the commit, confirming that your new files are now part of the new repository revision.

22) That’s all there is to using Subversion with the TortoiseSVN client. Probably 90% of the work you need to do is covered here. If you get stuck though, don’t forget that you can read the user guide by clicking on the Help link in the TortoiseSVN commands menu.

Tips:

  • Checkout only what you need. If you include a subdirectory in the address, the checkout will ignore anything in the repository that is above the directory. This is very useful as it lets you quickly checkout a small part of a repository that might take a really long time to copy. e.g:
  • Checkout the whole repository: http://10.2.5.2:9880/zelle/
  • Checkout just a directory: http://10.2.5.2:9880/zelle/trunk/genshla01/
  • Checkout just a file: Alas, you can’t. Subversion needs to organize working copy data using directories as its own ‘housekeeping’ data needs to live in a hidden subdirectory. There would be no opportunity to create a subdirectory if you checked out only a file. The workaround is to use the “Checkout Depth” option to checkout only directories, then perform selective updates on the things you need.

Subversion Apache tuning for MS Windows

The use of Apache on Windows amongst our enterprise customers is rare, with the majority opting to host the Subversion Apache servers on one of the flavours of Linux.

In the cases where Windows is the Subversion Apache server’s operating system, it is common that Administrators are plagued with overly high memory consumption by the Apache httpd process, leading in extreme cases to the Apache server shutting down.

All subversion binaries for windows only provide 32-bit versions of Apache. Apache on Windows is run as a single process, with child threads that handle the http connections. This means that the total amount of memory available to Apache on Windows is always limited to 2 Gb. Couple this with known memory leaks for Apache modules, and it becomes easy to see why an Apache Subversion server on Windows is likely to die frequently.

Thankfully there are ways to minimise Apache’s memory hemorrhaging and achieve long term uptime for the Subversion server. What follows are some key Apache configuration directives which are optimal for an Apache Subversion server on Windows:

KeepAlive On

KeepAliveTimeout 15

MaxKeepAliveRequests 500

MaxRequestsPerChild 400

MaxMemFree 8000

Looking at each of the directives in turn:

KeepAlive On – Allows a connection to be re-used for multiple requests. This results in better performance as there is an overhead associated with establishing new connections.

KeepAliveTimeout 15 – close the connection if no new request occurs after 15 seconds. We can increase this value to improve performance for clients – but doing so will keep child threads tied up for longer.

MaxKeepAliveRequests 500 – each connection is allowed to service up to 500 requests before it is closed.

MaxRequestsPerChild 400 – each child thread is allowed to service 400 connections before it is killed.

MaxMemFree 8000 – each thread can keep 8Mb of free memory before it is forced to release it.

The strategy behind the above settings is to limit the life of each thread, so that if there are memory leaks, the memory gets released when the threads are killed.

Depending on the exact distribution of Apache Subversion installed, it is often the case that the default Apache configuration allows each thread to live forever and can keep an unlimited amount of free memory – “not the best”.

After applying the above settings, each thread will service MaxRequestsPerChild * MaxKeepAliveRequests = 200000 requests before being killed, and each thread is forced to relinquish any free memory once they have accumulated 8Mb.

MaxKeepAliveRequests and MaxRequestsPerChild may need to be fine tuned, depending on how users report performance and how much memory is consumed by Apache, the aim being to find an acceptable compromise between data transfer speed and Apache memory size.

Jonathan Paul
WANdisco Sherpa