Monthly Archive for November, 2011

Where Did That Branch Go?

When teaching “Subversion User” classes, one of the frequently asked questions is “When do you delete a branch?”

Unfortunately the reason they are asking about deleting branches is because they are concerned with saving space. And as all experienced SVNers know, deleting will have no effect on repository size. Remember, when anything is deleted in Subversion, whether it is a project (folder), a file or a property, it is only deleted in the revision where it is deleted, and subsequent revisions. It still exists in all previous revisions and can be viewed or recovered at any time.

So the question remains. “When, or why would you ever delete a project?”

First let’s look at the cases when you would NEVER delete a branch:

1) Some branches are always needed such as the “trunk” or branches that contain other branches like the folders “tags”, and “branches”. This assumes you are using the TTB layout format.
2) Some branches are referenced frequently. The branches in the “tags” folders that hold your recently released versions are used to create releasable code and compare to current trunk development efforts repeatedly.
3) Branches where concurrent development is taking place. Before the branch is merged back to the trunk (or wherever it was branched from) it would be confusing to delete the branch. It might also lead to someone forgetting to do the merge.

So that leaves us with the following types of branches that MAY be candidates for deletion.

1) Abandoned work efforts. For example, you may start a branch to do concurrent development, and later decide to do the work in the trunk. What do you do with the abandoned branch?
2) Branches that represent old versions of your code. Let’s be realistic, if you are on Release 5.6.004 you may go back to Release 5.6.003 or Release 5.5.012. But how likely is it you are going to need to reference Release 1.0.01 from 4 years ago.
3) Mistakes created when using the COPY command. You meant to create a Release branch from revision 1255 of the trunk and you mistakenly selected the revision 1250.

All three of these are good examples of when you might do a deletion. But if you are going to do a deletion you should have some rules. And these rules should be part of your organization’s “Policies and Procedures”. (We actually spend considerable time in the “WANdisco Subversion Administration Class” talking about the contents and importance of Subversion Policies and procedures.)

Rule 1) Always use the log message to document that a deletion has been done and why. There is a new TortoiseSVN property (tsvn:logtemplatedelete) that provides your users with a template of the information you might want them to fill out. To see a demonstration of this property in use, see this short video. http://www.youtube.com/watch?v=foNQME4rcVQ

Rule 2) (optional) Instead of deleting folders, move them to a folder called “Deleted”. This makes the “deleted” branches/folders easier to find and recover.
Rule 3) Make sure everyone knows how to restore deleted folders/branches.

The best reason to delete any file or folder from a Subversion project is to make the viewing of the rest of the project easier. The less information you have to look at, the more likely you WON’T checkout a branch incorrectly or use the difference command with the wrong branch. Both of these actions might waste your time or create some sort of error that could go unnoticed.

March of the uber

Admin Console - LDAP

So, we thought it would be good to give you a ‘broad stroke’ update on what’s going on in the world of uberSVN. Firstly, you’ll probably have noticed that another update recently appeared; a modest interim fix that sorts out some niggles encountered by Subversion 1.7’s early adopters. If you didn’t catch the update, here’s what changed:

uberSVN – Release 11.11

What’s New

[Portal] It’s no longer possible to browse to cached screens without re-authorization. (ESVNB-2157)

[Portal] We’ve cleaned up the layout of the uberSVN login screen. (ESVNB-2265)

[uberAPPS] We’ve changed our back-end server to use the C3P0 connection pooling library to improve the handling of long running connections to the database. (ESVNB-2267)

What’s Fixed

[svnSWITCH] We’ve fixed an issue where running with Subversion 1.7 stopped the repository browser from working. Also fixed is the problem with repository browsing that occurred if you switched to Subversion 1.7 when using SSL. (ESVNB-2468) (ESVNB-2473)

[svnSWITCH] Windows XP users are now able use the SVNswitch tool. (ESVNB-2471)

[svnSWITCH] Activating Subversion 1.7 no longer stops commit activity from being reported on the dashboard or repository activity stream. (ESVNB-2472)

The bigger picture

This is the 7th update in the 7 months since our first release – from that you can probably work out the release cycle that we’re aiming for, and so far hitting. This release tempo should also tell you that there’s been no slow-down here at WANdisco’s Sheffield-based software forge – indeed, if cities had middle names, Sheffield’s might be ‘Industrious’, and it’s hard for a developer of software tools to not be inspired by a city that unexpectedly became the world’s supplier of industrial tools.

If that allusion to Sheffield’s heritage seems like big talk from a small “upstart”, maybe it’s because we know we’re on to something and we’ve got the potential to change the game, if we can keep on course, and keep on delivering regular improvements. To this end we’re very happy with the results we’ve got from an Agile approach. Keeping your releases bite-sized but regular forces you to keep the focus on stuff that works rather than trying to build Rome in a day. Of course, in an office with a giant slide, who needs to ride the waterfall?

Now we’d concede that this might give the impression that the big changes are never coming, in fact you can be sure that they are. Our Suggestion Website has given us an invaluable insight into what many of you like about uberSVN, but more importantly it tells us what holes we need to fill. Full LDAP integration, Defect tracker, anyone? We are hard at work on plugging both into uberSVN, as well as a few other things-that-you-want. Of course, there are lots of features and capabilities that fall outside the scope of what we can build ourselves, we’ll still be able to deliver these through uberAPPS, which will soon benefit from another round of enhancements, and some cool new APPS for you to buy

It’s your turn

For us, uberSVN’s social coding element is important because software development, at its best, should be a conversation – sometimes, an argument. So we’ll ask again, keep asking us for stuff, most definitely keep telling us what we’re doing wrong. Most of all, keep talking.

Not given uberSVN a try? It’s here, it’s FREE, and so easy to install your kids could do it – in fact, ours did. http://www.ubersvn.com/

Total Economic Impact of Subversion MultiSite

It’s been ten years since the Agile Manifesto was published, and Agile is still a buzzword for the software industry, but Subversion’s single point of failure can be a problem for distributed teams looking to implement agile practices. Subversion MultiSite combines Subversion with WANdisco’s value-added features, which are particularly useful for keeping distributed teams agile. Crucially, Subversion MultiSite removes the potential bottleneck of Subversion’s single point of failure. With Subversion MultiSite, all repositories and fully readable and writeable, and remote developers aren’t dependent on a centralized build team.

Other key benefits of Subversion MultiSite include:

  • Automatic failover – individual servers or an entire site can be taken offline for planned outages, without interrupting user access. Of course, this automatic failover also applies to unplanned outages.
  • Built-in, automatic and continuous hot backup – as each server is a mirror of every other, and they are kept continuously in sync, recovery is automatic. As soon as the failed server comes back online, Subversion MultiSite retrieves all the transactions committed to the other servers in the Subversion cluster, during the outage. Whenever a server, or an entire site fails, users won’t notice and administrators won’t have to do anything – failover is truly transparent.
  • Ability to run builds at every site immediately – eliminates time wasted while waiting for a build to happen in a different time zone.
  • LAN speed performance – users at every location experience LAN-speed performance for both read and write operations.
  • Synchronization of all servers in a cluster – whether they’re located over a LAN or a WAN. This is achieved by replicating changes made against one server, to all the other servers, in real-time.
  • No additional hardware required – Subversion MultiSite is typically installed on the same server as Subversion or Apache, at each site.
  • Eliminates the need for disk mirroring solutions – no administrator intervention is required to achieve recovery.
  • Allows distributed systems to scale to support new sites/users – new nodes can be added to a distributed system, or existing nodes can be removed without interrupting the operation of the remaining nodes.

But, what is the actual economic impact and potential return on investment, for enterprises deploying Subversion MultiSite? Our latest report, illustrates the financial impact on a company employing Subversion MultiSite to replicate distributed Subversion servers. The report looks at the idle time saved, changes in bug fix rate, and the general benefits the company experienced from deploying Subversion MultiSite. The report employs the four fundamental elements of TEI in evaluating Subversion MultiSite:

1) Costs and cost reduction.
2) Benefits to the entire organization.
3) Flexibility.
4) Risk.

GIST Meetup!

This week we attended a meetup with the guys from the GIST Foundation. The GIST Foundation is a knowledge sharing organisation for the digital / tech community. This meetup was focused on apps developed for the Sheffield Appathon – a competition for students in Sheffield.

The winning App Solar Helper was beautifully designed with an impressive commercial strategy. The guys behind it – Ste Prescott, Rumning Du, Jonathan Barker and Michael Wain did a great job, and we can’t believe they’re only in their second year of university! We know they’ll go far in the future. Thanks GIST, for giving exciting young talent a platform to showcase their work! After the presentations we all went for a curry and a pint. Can’t wait to see everyone at the next meetup!

An Introduction to Subversion Hook Scripts on Windows

So you recently started running Subversion on Windows? You’re keen to unlock the power of hook scripts but don’t know where to start? Well dear reader, you can start right here…

Subversion hook script?

In a nutshell, it’s a program that performs an action which is triggered by a specified repository event. You can think of hook scripts as Subversion’s tool chest for setting up tighter control of a repository or integration with third-party applications, such as auto-closing tickets in a defect tracker when someone commits a change with “Fixes #123” in the log, or triggering builds in a continuous integration system. The most common and powerful hook scripts are server side, although it’s also possible to run client side hook scripts.

Server side hook scripts usually have the permissions of the web server, which gives them power and flexibility, being able to affect other repositories on the server – of course the flip side to this power is the risk that a badly designed hook could slow the server to a crawl or in extreme cases, corrupt the repository.

What can hook scripts do?

  • Report on events, such as emailing alerts when a commit occurs, sending a report on the commit to a third-party.
  • Test for a pre-condition before allowing a commit to occur. For example, checking that a commit has been commented, or follows coding guidelines.
  • Block certain actions. Maybe prevent a commit that contains multi-gigabyte files that could block other SVN traffic for hours, or prevent locks from being stolen.

What hook scripts can’t (shouldn’t) do

  • Hook scripts shouldn’t modify files being committed. Right now, if a hook script actually changed a commit transaction, there’s no way of communicating the change back to the client. So while it might be tempting to write a script that automatically corrects errors or applies positive changes to a committed file, it will result in two different versions of the file in the same revision one version written to the repository and another in the user’s cache. The result of this loss of consistency can be as horrible as it is unpredictable.

Where are the hook scripts located?

When a new repository is created, a hooks subdirectory is included, e.g. if you create a repository called chancode, hooks will live in the /repositories/chancode/hooks/ directory. By default a non executable template (.tmpl) for each type of hook script is automatically created with the directory.

Hooks

Hook types

  • Start-commit Invoked before a transaction is created, pre-pre-commit, if you will. Commonly used to check if a user has sufficient privileges to perform a commit.
  • pre-commit / post-commit Invoked before or after a commit is completed.
  • pre-revprop-change / post-revprop-change Invoked before or after a revision propertly is added, modified or deleted.
  • pre-lock / post-lock Invoked when a user attempts to lock a path or after a lock has been created.
  • pre-unlock / post-unlock Invoked when a user attempts to destroy an exclusive lock, or immediately after the lock’s destruction.

There’s a good source of pre-written scripts available online, although the vast majority have been written for Unix/Linux, so aren’t immediately usable on Windows. Hooks can be written in anything, although cross-platform languages like Perl, Python and Ruby are quite popular. For simple scripts, it’s possible to use Windows’ plain old batch scripting.

Testing the water

We’ll do a simple test to show a hook script in action. We’ll introduce a pre-commit hook that checks for a log message that is n characters long – in this case, at least 6 characters.

  1. On your Windows Subversion/uberSVN server navigate your way to the hooks directory in your repository.
    i.e. C:\Program Files (x86)\WANdisco\uberSVN\repositories\chancode\hooks
  2. Open the pre-commit.tmpl file – you’ll recall that this is the script that gets triggered before a commit is completed.
  3. The template file contains lots of useful information, as well as an example script that does exactly what we’re going to do, unfortunately it’s aimed at the Unix/Linux /bin/sh interpreter, so we can’t use it for Windows without a rewrite.
  4. Delete all the contents of the file and copy in the following Windows batch scripting:
    @echo off  
     :: Stops commits that don't include a log message of at least 6 characters.        
     @echo off  
    
     setlocal  
    
     rem Subversion sends through the repository path and transaction id  
     set REPOS=%1  
     set TXN=%2           
    
     svnlook log %REPOS% -t %TXN% | findstr ...... > nul  
     if %errorlevel% gtr 0 (goto err) else exit 0  
    
     :err  
     echo --------------------------------------------------------------------------- 1>&2   
     echo Your commit has been blocked because it didn't include a log message. 1>&2  
     echo Do the commit again, this time with a log message that describes your changes. 1>&2
     echo --------------------------------------------------------------------------- 1>&2  
     exit 1    
    

    The script uses Svnlook, the incredibly useful command line tool to examine the commit transaction’s log message, then calls upon the findstr command to look for a string of at least 6 characters – the length of the minimum message length is set by the number of “.” characters after the findstr command.

  5. Save the file so that it can be executable. Windows supports .exe or .bat files.
  6. Testing time. Make an arbitrary change in a file, commit the change with a log message that is either blank or contains fewer than 6 characters. You’ll get the following error message:

    Tortoise 01

  7. Repeat the commit with the necessary number of characters, the commit will then succeed.
  8. Of course, this is a super-simple example. Lazy developers soon learn to leave comments along the lines of ‘update update update’ or ‘blah blah blah.’ You can use more complex scripts for better checking.

Parting Tips

Debugging and troubleshooting hook problems is notoriously difficult as there’s no formal error logging. These points are worth keeping in mind when setting up hook scripts.

  • If you’re completely stuck when trying to convert a hook script for use with Windows, remember there’s always Cygwin, a set of tools for running Linux tools on Windows, which will let you run many Linux scripts on Windows without the need to rewrite them.
  • Empty environment problem – Subversion executes hook programs with no environment variables. That’s a good security precaution but often leaves administrators pulling their hair out with frustration when a manually tested hook script absolutely refuses to work with Subversion. So ensure that any necessary environment variables are set in your hook script and/or use absolute paths.

Best Practices for Merging in Subversion

A quick Google search will uncover countless tales of merge hell in Subversion but, when used correctly, branching and merging can be an invaluable tool for the Subversion developer. Essentially, a merge takes the differences between two separate revisions and applies them to the working copy path, using various invocations of the svn merge command. A merge never creates a new version, it always modifies a workfile, which is created by the merge and is then edited before the changes are committed back to the repository. Things do get a bit more complicated when conflicts occur during merging. Whenever two or more people have modified the same code, Subversion will flag up any conflicts when you attempt to merge.

While you’re not forced to resolve these conflicts immediately, you will be unable to commit your changes to the repository until all the conflicts have been resolved.

Merging is a crucial function for getting the most out of Subversion. Here are our ten tips, to help you avoid living through your own merge horror story:

1) Project partitioning – consider assigning bugs to team members based on the location of those bugs. This reduces the chances of different people modifying the same files, at the same time.

2) If confused, start over – if you make any errors (e.g you’ve merged the wrong working folder) you can always start over. The ‘Revert’ command will discard all changes.

3) Always start with a checkout or an update – always have the latest code from the trunk before you start resolving any conflicts, so make sure you have just performed a checkout or an update. If you’ve made changes to the working folder before you start the merge, it makes sense to commit that back to the repository, so you have a new revision of what the code looked like before you started your merge.

4) Merge on logical checkpoints – in most cases, it makes sense to merge when the ‘from’ part of the merge (i.e the branch you’ve been working on) has reached the level of quality where changes are testable and compilable. Do not merge anytime an experimental new change has been made to the branch.

5) Use appropriate log messages – make it easy for people to go in later and find out what merges happened, what changes were made, etc. The more information you provide, the easier it will be two months later when someone asks whether the changes from a particular branch were ever incorporated into trunk development.

6) Merge soon – fix conflicts as soon as possible. The quicker you resolve conflicts and commit those changes back to the repository, the sooner the rest of your team can take your changes into account when working on their own part of the development effort.

7) Isolate the merge – sometimes, it’s tempting to merge some changes and then add additional planned development before performing a commit. Try to avoid this, as it makes it difficult to separate which changes came from the merge, and which changes were made as part of the rest of the development.

8 ) Don’t become too isolated – even if it seems your changes have little to do with the rest of the team, by the time you’ve finished perfecting your branch, the trunk may have evolved to the point that it’s a nightmare to merge your changes. To avoid this, you should regularly replicate changes being made within the trunk, into your own branch, just to ensure they fit with your own changes.

9) Use commands – Subversion has some commands to help take the hassle out of merging. These include:

a) svn:mergeinfo – displays the history of merges made into a particular file or directory.

b) –dry-run – previews status codes that would be printed in a real merge, without actually applying any local changes to the working copy.

c) svn log – a svn log command on the /branches URL will display the entire history of a branch – even if a successfully merged branch is no longer visible in the /branches directory, it’s still a part of the repository’s history.

10) ….And don’t panic! – conflicts happen much less frequently than you would think. Whenever different team members can access the same files, there’s going to be occasional conflicts, but Subversion is a great tool for rectifying these conflicts when they do occur.

For more tips on merging, check out our free-to-download ‘Introduction to Branching and Merging’ and ‘Advanced Branching and Merging’ cheat sheets.

Happy (belated!) Halloween from WANdisco

This year, we celebrated Halloween with a party at our UK office in the Electric Works, and enjoyed some spooky cocktails, Halloween sweets, and even had a go on the helter skelter and Skull Coconut Shy! Thanks to Spice Yorkshire and Themed Evenings, for all the Blood Punch and Black Velvet cocktails, and the great decorations!

Software Designed to Facilitate Collaboration

There are lots of examples of groupware. Project management systems, bug tracking systems and version control systems. Yes, Subversion is a groupware application.

The training and standards needs for applications that are classified as groupware are different, and more necessary than other applications.

Typically with groupware applications, one person’s output is another person’s input. This means that as soon as enough users start distrusting the completeness or quality of a group’s input, they will start to devalue the use of that information.

What this means in Subversion, is they start not using the tools on certain projects, or they maintain alternate copies of the projects, or they start only inputting certain project information. All of which make the use of Subversion less valuable and results in software configuration management problems.

How Do You Avoid This?

  • Training – Make sure all the people who need to use, or make decisions, on how Subversion should be used know about the tool.
  • Policies and Procedures – Have rules for the usage of Subversion. This will include naming conventions, project standards and log message usage. Hook scripts can make following standards easier, but someone has to decide which are needed, and then implement their use.
  • Management – If someone violates the polices, show them the correct way to do something. Evaluate your workers on how well they follow standards. Do not allow exceptions. “This is a small/quick/unimportant project so I will do it outside of Subversion”.

Hopefully the idea of training and standards in a highly technical environment is not a new idea. The manner in which you train your developers on the capabilities and usage standards for Subversion is less important than the fact that you do some training. The training could be standup, online interactive, or one-on-one mentoring. But don’t kid yourself, training is necessary.

It would be nice to think that your workers will take home the Subversion/TortoiseSVN manual and read it over the weekend. But ask yourself this question: have you done that?

WANdisco has spent years helping companies use Subversion in an effective manner. We’ve seen all the mistakes and we can help you avoid them.

Michael Lester
Director of Training, WANdisco

WANdisco Become Team MANdisco for Movember!

Welcome to Movember – the month formerly known as November! It’s a month long celebration of the mustache. WANdisco have formed team MANdisco. We’re being sponsored to grow mustaches to support male focused charities like the Prostate Cancer Charity and the Institute of Cancer Research.

Movember has really made a positive impact on the way men think about their health and we’re proud to be Mo-Bros and help out.

Today the men of WANdisco are clean shaven, but by the end of the month they will be sporting full on mustaches. We’ll be recording our efforts on Twitter and we’ll blog our final Movember photo on the 1st of December.

If you’d like to join the MANdisco team, start your own Movember team, or donate you can visit the Movember Website; http://uk.movember.com/mospace/1843078/

Open Source: Avoiding The Legal Risks

Over the last couple of years, open source solutions have been gaining popularity within the enterprise, and with all the benefits of open source, it’s easy to see why! Cost is the most compelling reason (with closed source, there’s the initial outlay, but there can also be hidden extras; support charges, the expense of ongoing upgrades, training costs, etc) but that’s just the start of what open source has to offer! There’s the vibrant ecosystem of free client tools, GUIs and plugins that grows around successful open source projects; the tutorials and guides that can be accessed for free online; and the community support available through mailing lists and forums. But, there are some downsides too, and enterprise organizations in particular often have some common concerns when adopting open source. The major concern is often the legal risks associated with open source solutions, particularly the potential liability for intellectual property (IP) and copyright infringement. Here are the major legal risks organizations should consider, before they opt for open source:

  • Your typical open source project is a collaborative effort between potentially thousands of developers. Theoretically, any one of those developers could add code that infringes on IP or copyright.
  • Open source software does not come with the protection typically given for commercial products. If you employ open source in your organization, any consequent exposure to copyright and IP infringement, rests solely on you.
  • It’s not uncommon for open source projects to be treated as hobbies, and consequently it’s unlikely they’ll be subjected to the same testing processes as commercial software.
  • Without a contractual agreement, it’s your problem if open source code contains bugs – and your responsibility to foot the bill for any financial consequences of these bugs.
  • Some open source licenses (particularly the GPL) use copyright law to allow the distribution of copies and modified versions of a project, while stipulating that the same rights must be preserved in modified versions of the project. Theoretically, an organization using open source could be forced to release the code of their “value-added” version, for the whole world to use.
  • Even if you are aware of the origins of all the code used by your company, an acquisition could change all that! Adding a component from a freshly-acquired company, could mean you are unwittingly integrating open source code into your commercial offering.
  • In the event of copyright or IP infringement, injunctive relief can be sought in addition to monetary damages. This could effectively force a company to stop using or distributing their software.

Open source has a lot to offer organizations – it’s cheaper than your average proprietary offering, and is usually backed by a vibrant ecosystem, plenty of readily-available online tutorials and guidelines, and a helpful user community – but there are some very real legal risks to take into consideration before you adopt open source. With this in mind, there are some steps organizations should take to protect themselves against potential IP and copyright infringement from their use of open source:

Do your research – not all open source licenses are equal! Different licenses can have different consequences, and everyone in your team should fully understand all the potential financial and legal consequences of your chosen open source license. Even if employees are familiar with open source and have even used it in their own personal projects, they may not be aware of the potential legal risks.

Consider appointing a dedicated open source compliance officer – this individual will be charged with regularly checking your organization’s compliance with the relevant open source licenses. Alternatively, you could create a review group.

Keep records – fully documenting your use of open source code will help you avoid unwitting IP or copyright infringement.

Take out professional support – WANdisco’s indemnification coverage protects against a third party filing intellectual property claims against your company, based on the use of Subversion open-source software obtained through us. We cover four types of intellectual property infringement: patents, trademarks, trade secrets, and copyright, as part of our Platinum and Platinum Plus support contracts. More information on indemnification coverage, is available at our support page.