SVN MultiSite Plus: The Fastest Solution for Game Developers

Handling large binary files, such as the media assets used during game development or design files used by hardware and firmware engineers, can be challenging for SCM systems.  Apache Subversion has accelerated development of enterprise features such as class-leading performance with large binary assets.  Subversion now shows a significant advantage when committing and transferring files over a WAN compared to Perforce, a commercial SCM system with a reputation for being the performance leader in binary asset management.

Test Setup

In order to test the performance of large binary file handling over a WAN, I set up a test configuration as follows.

 

SVN MultiSite Plus test configuration

SVN MultiSite Plus test configuration

Perforce test configuration

Perforce test configuration

 

 

During the test, a 1.7 GB ISO file is committed to a local node three times, and each iteration is timed.  Two measurements are taken:

  • Time to run the commit command. This is the user’s experience of system speed.

  • Time for the file to transfer to the remote node over a link with 256 ms simulated latency. This is the time after which the file would be available to users at other sites.

Test Results

As the chart below shows, Subversion is significantly faster to commit and transfer the data.

Test results

Test results

Analysis

Several conclusions can be drawn.

1.       Perforce is very slow to transfer a large binary file over a WAN. No compression was used between client and server, and the link was unencrypted. As the chart indicates, the bulk of the commit time was simply waiting for file transfer. Subversion is significantly faster. [1]

2.       SVN MultiSite Plus offers the benefit of using a second local node to satisfy data integrity before accepting the commit. By default commit data must reach at least one other node to ensure data availability, but this can be a local node. The user does not need to wait for the file to transfer over the WAN. This result is clearly shown in the results for the first iteration, when the commit completed in 5 minutes and the file appeared on the remote node 9 minutes later.

3.       Subversion does not retransfer or store duplicate objects in the repository storage area. Thus, there is no time for transferring the commit content in the second and subsequent runs.

The departments working on game development or hardware/firmware design are often separated by function (e.g. game artists versus game developers or hardware engineers versus application developers), and often each team will be in a separate location. For these teams, SVN MultiSite Plus offers a significant performance advantage when handling large binary files.

[1] The ‘svn’ protocol used by svnserve is relatively unaffected by increased latency as it does not wait for responses from the server while transmitting data.  See https://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_ra_svn/protocol for more information.  Perforce, however, is very sensitive to TCP send and receive buffer sizes over a high latency network.  Increasing the operating system network tunables does improve Perforce performance.  In a simpler test of a regular commit from client to server over a 256ms latency link, the transfer time dropped from 2 hours to 20 minutes.  However that still compares poorly with Subversion’s out of the box, non-replicated transfer time of 7 minutes, and again SVN MultiSite Plus completes a replicated commit in even less time.

 

 

 

1 Response to “SVN MultiSite Plus: The Fastest Solution for Game Developers”


  • Very interesting performance test!

    Just an FYI for those who would like to convert from Perforce to SVN, there’s a tool called p42svn that is available at http://p42svn.tigris.org/.

    p42svn extracts versions and history from Perforce repository and generates a SVN dump file. You can then import the dump file into SVN repository.

    Martin

Leave a Reply