WebSVN To Monitor Subversion Repositories

Subversion Subversion (SVN) is a version control system initiated in 2000 by CollabNet Inc. It is used to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly-compatible successor to the widely used Concurrent Versions System (CVS). Subversion is well-known in the open source community and is used on many open source projects.

Subversion was started in 2000 as an effort to write a free version control system which operated much like CVS but with fixed bugs and misfeatures in CVS. By 2001, Subversion was sufficiently developed to be capable of hosting its own source code. More information, including this above paragraphs, is here.


Getting started with SVN

Please refer to my earlier post.


Managing websites with SVN

Please refer to my previous post.


Getting ready for WebSVN

Let us assume that /var/www/mydomain_repos will serve as the DocumentRoot for viewing our subversion repositories/projects via web and that the corresponding URL will be http://repos.mydomain.com/. Let us also assume that this location is already in existence. The Apache configuration file, /etc/httpd/conf/httpd.conf, needs to have the following in it [one can add more stuff]:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# WebSVN - password protected
<VirtualHost *:80>
  UseCanonicalName Off
  DocumentRoot /var/www/mydomain_repos
  ServerName repos.mydomain.com
  ServerSignature On
  DirectoryIndex index.html index.php index.cgi index.pl
  AccessFileName .htaccess
 
# Main Directory
  <Directory "/var/www/mydomain_repos">
    AuthName "Reserved for developers only!"
    AuthType Basic
    AuthUserFile /var/www/mydomain_repos/.htpasswd
    AuthGroupFile /dev/null
    require valid-user
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Once these changes are saved, Apache needs to be restarted.


1
/etc/init.d/httpd restart

Once that’s done, one can download the latest and greatest stable version from websvn.info and suppose that it’s downloaded to $HOME/Desktop.


1
2
3
4
5
6
7
8
9
10
11
12
13
cd /tmp
mkdir WEBSVN
cd WEBSVN
tar -zxvpf $HOME/Desktop/websvn-x.y.z.tar.gz
 
cd /var/www/mydomain_repos/
cp -rf /tmp/WEBSVN/websvn-x.y.z/* .
 
# Needed for RSS feed to work properly
chmod 777 cache
 
cd include
cp distconfig.php config.php


Customizing WebSVN – /var/www/mydomain_repos/include/config.php

Included below are portions of my copy of config.php that I have edited to make it work for this set up:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// {{{ PLATFORM CONFIGURATION ---
//
// Configure these lines if your commands aren't on your path.
$config->setSVNCommandPath('/usr/bin');
$config->setDiffPath('/usr/bin');
 
// For syntax colouring, if option enabled...
$config->setEnscriptPath('/usr/bin');
$config->setSedPath('/bin');
 
// For delivered tarballs, if option enabled...
$config->setTarPath('/bin');
 
// For delivered GZIP'd files and tarballs, if option enabled...
$config->setGZipPath('/bin');
 
// }}}
 
// {{{ REPOSITORY SETUP ---
//
// Remote repositories (without and with optional group):
$config->addRepository('mydomain', 'http://svn.mydomain.com/', NULL, 'svn-userid', 'svn-password');
 
// }}}

If the system does not have enscript, it may be installed using the command:


1
2
# Run as root
yum install enscript


Does it work?

When the browser is pointed to http://repos.mydomain.com/, it prompts for UserID/Password and then displays a following looking screen, it should be working.



WebSVN


Troubleshooting

I followed every step as mentioned. http://repos.mydomain.com/ opens fine and shows only the project name. When I click on it, I don’t see anything else. What’s going on?

I had that problem as well. It took me a while and I thought there was something wrong with the way I had configured Subversion and/or WebSVN. As it turned out, the current [latest, stable] release of WebSVN (2.1.0) requires Subversion 1.2 (or greater), and my instance was 1.1.4. The following command may be used to check the version of Subversion:


1
svn --version --quiet

CentOS 4.x didn’t have any further updates available for subversion but Dag Wieers repository had the necessary packages. I downloaded subversion-1.4.6-0.1.el4.rf.i386.rpm, mod_dav_svn-1.4.6-0.1.el4.rf.i386.rpm and installed them using the command:


1
2
# Run as root
rpm -Uvh subversion-1.4.6-0.1.el4.rf.i386.rpm mod_dav_svn-1.4.6-0.1.el4.rf.i386.rpm

That resolved the issue – WebSVN elegantly displayed the project(s) in the repository and all relevant information.

Website Management With Subversion

Subversion Subversion (SVN) is a version control system initiated in 2000 by CollabNet Inc. It is used to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly-compatible successor to the widely used Concurrent Versions System (CVS). Subversion is well-known in the open source community and is used on many open source projects.

Subversion was started in 2000 as an effort to write a free version control system which operated much like CVS but with fixed bugs and misfeatures in CVS. By 2001, Subversion was sufficiently developed to be capable of hosting its own source code. More information, including this above paragraphs, is here.


Why manage websites with Subversion?

As I mentioned in my previous post, I often find myself modifying the web pages I (have) design(ed). It’s only fair [to me] that I keep a better track of changes – rather just relying on taking screenshots [which I mostly forget] and/or adding comments to the files and keeping just one copy. Keeping track of revisions with Subversion also makes it [relatively] easier & less time consuming to revert back to a specific version of the website.


Getting started with SVN

Please refer to my previous post.


BETA and LIVE versions

The proposed concept/work-flow is to check out a working copy from the repository, make the necessary edits, check it back into the repository, update the BETA version [to see if everything works the way it should] and then [if everything in BETA version is working fine] update the LIVE version. A schematic representation looks like this:



Subversion Website



To that effect, let us assume that /var/www/mydomain serves the live version (http://mydomain.com/) while /var/www/mydomain_beta serves the beta version (http://beta.mydomain.com/), and that these locations are already in existence. The Apache configuration file, /etc/httpd/conf/httpd.conf, needs to have the following in it [one can add more stuff]:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# Live version
<VirtualHost *:80>
  UseCanonicalName Off
  DocumentRoot /var/www/mydomain
  ServerName mydomain.com
  ServerSignature On
  DirectoryIndex index.html index.php index.cgi index.pl
  AccessFileName .htaccess
 
# Main Directory
  <Directory "/var/www/mydomain">
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>
 
# Beta version - password protected
<VirtualHost *:80>
  UseCanonicalName Off
  DocumentRoot /var/www/mydomain_beta
  ServerName beta.mydomain.com
  ServerSignature On
  DirectoryIndex index.html index.php index.cgi index.pl
  AccessFileName .htaccess
 
# Main Directory
  <Directory "/var/www/mydomain_beta">
    AuthName "Reserved for developers only!"
    AuthType Basic
    AuthUserFile /var/www/mydomain_beta/.htpasswd
    AuthGroupFile /dev/null
    require valid-user
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Once these changes are saved, Apache needs to be restarted.


1
/etc/init.d/httpd restart

Add the following lines to /var/www/mydomain/.htaccess, so that any request to view .svn folders will be directed to an error code/page.


1
2
# Do not display .svn (Subversion) stuff
RedirectMatch 403 /\.svn.*$


Adding website to the repository

If your case is like mine, then you would already have a live website running – and its contents need to be imported into the repository. In that case, it’s a good idea to have a full back up of the website – so that there is something to revert back to, if things were to go wrong in the process. The following was my work-flow:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
cd $HOME
 
# Backing up the website - trailing slashes in 'rsync' are important
mkdir -p backup/mydomain
rsync -avH /var/www/mydomain/ $HOME/backup/mydomain/
 
mkdir svn_projects
cd svn_projects
 
mkdir mydomain
cd mydomain
 
mkdir branches tags trunk
rsync -avH /var/www/mydomain/ $HOME/svn_projects/mydomain/trunk/
 
cd trunk
# Decide which files and folders should be kept in the repository.
# Third party software [for e.g. PixelPost, WordPress] can be removed.
 
# Keep a snapshot of current version as v1.0
cd ../tags
mkdir 1.0
rsync -avH $HOME/svn_projects/mydomain/trunk/ $HOME/svn_projects/mydomain/tags/1.0/
 
# Importing into the repository
cd $HOME/svn_projects/
svn import mydomain http://svn.mydomain.com/mydomain -m "First import of mydomain into SVN"
#

Depending on the size & type of files, it will take a while to import the website into the repository. When all said and done, one should see something like:



Subversion Import


Checking out BETA and LIVE versions

Since /var/www/mydomain.beta location is [most likely] empty, it doesn’t pose any problem and checking out a [trunk] version is pretty straightforward.


1
svn co http://svn.mydomain.com/mydomain/trunk /var/www/mydomain_beta

Depending again on the size and type of files, it will take a while to check everything out. However, since there is [most likely] some stuff already in the live location (/var/www/mydomain), check out process will complain:


svn: Failed to add file ‘foo.bar’: object of the same name already exists

One way to deal with this problem is remove everything from /var/www/mydomain, check out a copy from the repository and restore the files [not in the repository] from the full backup created earlier. If the website gets a lot of frequent visitors, it might be a good idea to schedule a specific time slot for this process and let the visitors know about it well in advance.


1
2
3
4
5
6
7
8
9
10
11
12
# Stop any cron jobs that might be writing stuff into this location
# Delete everything in live website
cd /var/www/mydomain
chmod -R 777
\rm -rf *
 
# Check out a copy from the repository
svn co http://svn.mydomain.com/mydomain/trunk /var/www/mydomain
 
# Restore the files, not in repository, from the full backup
rsync -avH $HOME/backup/mydomain/ /var/www/mydomain/
#

Once that’s done, the beta and the live versions both contain the trunk [a.k.a. bleeding edge] version of the website.


Checking Out, Editing/Updating and Checking In Local/Working Copy

Personally, I prefer to keep the local/working copy on a different machine [Apple MacBook, which has subversion installed]. The command to check out is still the same.


1
2
3
cd $HOME/Sites
mkdir mydomain
svn co http://svn.mydomain.com/mydomain mydomain

The files $HOME/Sites/mydomain/trunk/ may be edited/updated with a text editor of choice [I use vi but one can also use Coda or other such application].


1
2
3
4
5
6
7
8
9
10
11
12
# Editing
cd $HOME/Sites/mydomain/trunk
 
# This 'svn info' and 'svn up'are very useful - lets us know the revision number, etc.
svn info
svn up
#
# Edit/update files using preferred editor; I use vi
#
svn stat
svn diff
# This tells what's different between local copy and the one on the server.

Once the edits are completed, it needs to be checked back into the repository.


1
2
3
# Checking in
cd $HOME/Sites/mydomain
svn ci -m "Checking in first set of revisions: edited /index.html and /css/mystyle.css"

One can refer to excellent online resources for commands that help perform adding/deleting files/folders, tagging and branching projects, and so on. Personally, I tag the main versions as YYYY.MM and not-so main yet important versions as YYYY.MM.DD. One of the advantages of such a tagging scheme is that – in the absence of a web/graphical viewing interface of the repository/logs – I can still learn, just by looking in tags, about different tagged versions.


1
2
3
4
5
6
7
# Tagging
cd $HOME/Sites/mydomain
svn cp trunk/ tags/2009.03
svn ci -m "Tagging version 2009.03 - a routine monthly revision"
 
svn cp trunk/ tags/2009.03.23
svn ci -m "Tagging version 2009.03.23 - significantly changed the layout and approach"


BETA Testing & LIVE Deployment

Once the edits are completed and the changes have been properly checked back into the repository, Beta testing is relatively easier.


1
2
3
# These commands are to be executed on the web server
cd /var/www/mydomain_beta
svn up

This will promptly update /var/www/mydomain_beta with files/folders that were changed/edited. If things looks good in the browser, then the same command can be run for LIVE version as well.


1
2
3
# These commands are to be executed on the web server
cd /var/www/mydomain
svn up

If BETA testing didn’t yield the desired results, one can go back to Checking Out, Editing/Updating and Checking In Local/Working Copy section, fix those mistakes, check the code back into the repository and redo the BETA testing.

Subversion With Apache

Subversion Subversion (SVN) is a version control system initiated in 2000 by CollabNet Inc. It is used to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly-compatible successor to the widely used Concurrent Versions System (CVS). Subversion is well-known in the open source community and is used on many open source projects.

Subversion was started in 2000 as an effort to write a free version control system which operated much like CVS but with fixed bugs and misfeatures in CVS. By 2001, Subversion was sufficiently developed to be capable of hosting its own source code. More information, including this above paragraphs, is here.


What’s in it for ME?

As often as I change the scripts I use and modify the web pages I (have) design(ed), I think I have finally realized that I need to keep a better track of changes – rather just relying on taking screenshots [which I mostly forget] and/or adding comments to the files and keeping just one copy. If only I knew how to set this up little over a year ago, I would have certainly used it to keep track of my dissertation writing process [and not forgotten, by complete accident, to thank a bunch of my beloved seniors from M.Sc. days in Bangalore University].


Getting Started

It’s my practice to do a full/maximum installation of the OS and as such, that will ensure that most packages – though not everything needed – will be present, whether I use them or not. As such, I had Apache & Subversion up and running but was missing the mod_dav_svn module. One way to figure out its presence/absence is to look for /etc/httpd/conf.d/subversion.conf. If the file is there, then there is a good chance that the module is already installed. If the file is not there, then there is a very good chance that the module is missing. Either way, running the following command as root will ensure that the module (as well as subversion itself) will be installed, if not already installed.


1
yum -y install subversion mod_dav_svn


Updating httpd.conf and subversion.conf

Before updating these config files, one needs to create some directories. As root, I did:


1
2
3
4
5
mkdir /var/www/mydomain.svn
mkdir /var/www/svn
cd /var/www/svn
svnadmin create repo
chmod -R apache:apache repo

In my particular case, I have several VirtualHosts configured in httpd.conf and as such, I did not edit /etc/httpd/conf.d/subversion.conf. I took the information from there and added it appropriately in /etc/httpd/conf/httpd.conf.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
<VirtualHost *:80>
  DocumentRoot /var/www/mydomain_svn
  ServerName svn.mydomain.com
  ServerSignature On
  DirectoryIndex index.php index.html
  AccessFileName .htaccess
 
## Main Directory
  <Directory "/var/www/mydomain_svn">
    Options Indexes FollowSymLinks Includes
 
    AuthName "Reserved for SVN Admin Only!"
    AuthType Basic
    AuthUserFile /var/www/mydomain_svn/.htpasswd
    AuthGroupFile /dev/null
    require valid-user
    require user svnuserid
 
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
 
## Value of SVNPath must not be under DocumentRoot
## defined above. Else, 'moved permanently' kind of errors
## are to be expected.
#
## Also, creating .htpasswd file with an appropriate entry
## can be accomplished by referring to one of the many good
## online resources.
  <Location />
    DAV svn
    SVNPath /var/www/svn
  </Location>
</VirtualHost>

Once these changes are saved, Apache needs to be restarted.


1
/etc/init.d/httpd restart


Is it working alright?

Point the browser to http://svn.mydomain.com/ and if the browser presents a screen that looks like:



Browser Authentication

it should be working.


Additional Notes

These can be found in Wikipedia as well as other websites, but I am including them here for my sake and for completeness purposes (Image courtesy: Wikipedia).

Subversion Tree The Subversion filesystem can be described as “three dimensional”. In addition to the two dimensions of a standard directory tree (e.g., tree view), the Subversion filesystem’s third dimension is revisions. Each revision in a Subversion filesystem has its own root, which is used to access contents at that revision. Files are stored as links to the most recent change; thus a Subversion repository is quite compact. The storage space used is proportional to the number of changes made, not to the number of revisions.

The Subversion filesystem uses transactions to keep changes atomic. A transaction is begun from a specified revision of the filesystem, not necessarily the latest. The transaction has its own root, on which changes are made. It is then either committed and becomes the latest revision, or is aborted. The transaction is actually a long-lived filesystem object; a client does not need to commit or abort a transaction itself, rather it can also begin a transaction, exit, and then can re-open the transaction and continue using it. Multiple clients can access the same transaction and work together on an atomic change.

Subversion offers two types of repository storage — FSFS and Berkeley DB. FSFS works faster on directories with a large number of files and takes less disk space, due to less logging. Subversion has some limitations with Berkeley DB usage leading to repository corruption and data loss when a program that accesses the database crashes or was terminated forcibly. When using Berkeley DB repository, the only way to use it safely is on the dedicated server and by a single server process running as one user, according to Version Control with Subversion. Existing tools for Berkeley DB repository recovery aren’t completely reliable, so frequent repository backups are needed. The FAQ Section says that FSFS file system is default for Subversion v1.1.x [my instance runs 1.1.4].


Adding a project to the repository

Personally, I prefer to create the folder with appropriately named sub-folders:



Subversion Project Tree

Suppose that the folder my-big-project has the above structure (located in $HOME) and once the trunk folder has all the required files/folders of a project, then the following command may be used to import them into the subversion repository:


1
2
cd $HOME
svn import my-big-project http://svn.mydomain.com/my-big-project -m "Adding my-big-project to Subversion Repository"

The process should prompt for a password [remember entry in .htpasswd file]. Once successfully completed, one may delete the $HOME/my-big-project folder.


Checking Out, Editing/Updating and Checking In

Now that the subversion repository contains my-big-project, one [an authenticated user] can check it out, edit/update and check it back in. The commands are as follows:


1
2
3
4
# Checking out
cd $HOME
mkdir my-big-project
svn co http://svn.mydomain.com/my-big-project my-big-project
1
2
3
4
5
6
7
8
9
10
11
# Editing
cd $HOME
cd my-big-project
svn up
#
# Edit/update files using preferred editor; I use vi
#
svn stat
svn diff
# This tells what's different between local copy and the one
# on the server.
1
2
3
4
# Checking in
cd $HOME
cd my-big-project
svn ci -m "Checking in first set of revisions: 2 cool features added"

One can refer to excellent online resources for commands that help perform adding/deleting files/folders, tagging and branching projects, and so on.


Is Subversion it or is there something better?

There probably is one – dear friend, Jon (@nuxi), tells me that GIT is better than SVN. For now, this seems to have got me a good start and as I find more time, I will look into GIT. If you find this useful and/or find bugs/mistakes and/or have other elegant methods of accomplishing the same thing, please do share them – readers, including me, will greatly appreciate it.

Apache, Virtual Hosts, HTTPS, Self-Signed Certificates

Disclaimer

The aforementioned instructions/steps worked for me running CentOS. It may very well work for you on Red Hat-like distributions or otherwise. Please note that if you decide to use these instructions to update PHP/MySQL (or other packages) on your machine, you are doing so entirely at your very own discretion and that neither this site, sgowtham.com, nor its author is responsible for any/all damage – intellectual or otherwise.



It has been a while and many things (Apache and Virtual Hosts) have been working smoothly for few years now but I could never quite figure out how to set up the secure portion. Had tried it several times with very little success and a bit more (extensive) search later, I think I have it working properly now. For the sake of completeness, let me just go through with Apache as well as Virtual Hosts.


Apache, PHP, MySQL

It is my practice that I do a full/maximal installation of any linux distribution and that takes care of installing Apache (with all the required modules), PHP, MySQL, etc. I bet there are tons of documents online that you can refer and install them if you don’t already have them.


Virtual Hosts

Let us suppose that ISP does not block port 80 (and 443) and let us further suppose that ISP assigns a static IP address – something like 24.205.120.89. For whatever reason, let us also suppose that we need to serve websites from this address, namely http://mydomain.com/ and http://yourdomain.com/. Since I am using CentOS distribution, the ServerRoot is set to /etc/httpd and the configuration file is located in /etc/httpd/conf/httpd.conf. To accommodate hosting multiple domains, my configuration was made to look as follows (towards the very end):


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
NameVirtualHost *:80
 
<VirtualHost *:80>
  DocumentRoot /var/www/html
  ServerName 24.205.120.89
  ServerSignature On
  DirectoryIndex index.html index.php index.cgi index.pl
  AccessFileName .htaccess
</VirtualHost>
 
<VirtualHost *:80>
  DocumentRoot /var/www/mydomain
  ServerName mydomain.com
  ServerSignature On
  ServerAdmin admin@mydomain.com
  DirectoryIndex index.html index.php index.cgi index.pl
  AccessFileName .htaccess
</VirtualHost>
 
<VirtualHost *:80>
  DocumentRoot /var/www/yourdomain
  ServerName yourdomain.com
  ServerSignature On
  ServerAdmin admin@yourdomain.com
  DirectoryIndex index.html index.php index.cgi index.pl
  AccessFileName .htaccess
</VirtualHost>



It is to be noted, needless to say, that the folders /var/www/html, /var/www/mydomain and /var/www/yourdomain must exist and must contain appropriate content. Also, A Record in the corresponding registrars must point to 24.205.120.89 as the destination. Upon restarting apache (/etc/init.d/httpd restart), these domains should be serving out contents just fine.


HTTPS & Self-Signed Certificates

Now that the non-secure (HTTP) portions are working fine, let us suppose that one of the domains, http://yourdomain.com/ has some contents that need to be served securely (HTTPS) as https://yourdomain.com/secure-content/. It is to be noted that only one of the three (or however many you may have) domain contents can be served in a secure way. First step is to edit /etc/httpd/conf.d/ssl.conf and make (parts of it) look like:


1
2
3
4
5
6
DocumentRoot "/var/www/yourdomain"
ServerName yourdomain.com:443
ServerSignature On
ServerAdmin admin@yourdomain.com
DirectoryIndex index.html index.php index.cgi index.pl
AccessFileName .htaccess



Next step is get Security Certificates but getting them commercially, although more trusted, can be expensive. Especially if the purpose is for testing and such, generating self-signed certificates using built in tools is the most economical way. Again, there are plethora of documents online that provide this information but is included here for completeness purposes:

  1. First step in this process is to generate the private keys for the server:
    openssl genrsa -des3 -out server.key 1024
  2. Next step is to create a Certificate Signing Request (CSR) to get signed by the CA. Enter appropriate information when prompted.
    openssl req -new -key server.key -out server.csr
  3. Next, CSR needs to be signed with the key generated in Step #1.
    openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
  4. Optionally, one can remove the password from key generated in Step #1.
    cp server.key server.key.withpass
    openssl rsa -in server.key.withpass -out server.key
  5. Penultimate step is to place these files appropriately:
    cp server.key /etc/httpd/conf/ssl.key/
    cp server.crt /etc/httpd/conf/ssl.crt/
    cp server.csr /etc/httpd/conf/ssl.csr/
  6. Restart Apache
    /etc/init.d/httpd restart

That’s it. If nothing went wrong in the process, https://yourdomain.com/ should now be serving secure content. The browser should display some warning and the security certificate can be accepted temporarily or permanently before the secure contents can be seen.


A Working Example

I followed these steps to make my website secure – not that there are too many things that need to be served in a secure way. You can check it out if you wish.