S3FS on CentOS

So, we’re using CentOS 5 for some of our servers; the ones we need cPanel for.  These are our shared setups with people running blogs, Joomla, Drupal, and such.

I’ve never liked FTP for anything due to its insecurity, slowness, and its inability to recover from even the simplest  of errors.

So I finally got our server provider to build a kernel with FUSE support built in so that I could use s3fs to mount an Amazon S3 bucket as a normal mount.

It was a little annoying to set up the first time but, when I had do it a second time, and had to go all the way back to the beginning, I figured I’d better write it down this time.

Install Subversion

First step is to get s3fs from its site at: http://code.google.com/p/s3fs/wiki/FuseOverAmazon.

I prefer to check out from Subversion but Subversion was not installed on my server.

A simple:

	# yum install subversion

 

gave me an error about a missing dependency:

Error: Missing Dependency: perl(URI) >= 1.17 is needed by package subversion-1.4.2-4.el5_3.1.x86_64 (base)

 

To make a long story short, I ended up downloading and installing the RPM directly with:

# wget http://mirror.centos.org/centos/5/os/i386/CentOS/perl-URI-1.35-3.noarch.rpm
# rpm install perl-URI*

 

Download and Install s3fs

Once I had subversion installed, I checked out and built s3fs:

# svn checkout http://s3fs.googlecode.com/svn/trunk/ s3fs-read-only
# cd s3fs-read-only/s3fs
# make install

 

There are a handful of warnings from the compiler, but I ignored them since I wasn’t particularly interested in working on the code.

Setting up Keys

You can invoke s3fs with your Amazon credentials on the command line, in the environment, or in a configuration file. Since command lines and environments are too easy for bad guys to find, I opted for the configuration file approach.

Create a file /etc/passwd-s3fs with a line containing a accessKeyId:secretAccessKey pair.

You can have more than one set of credentials (i.e., credentials for more than one amazon s3 account) in /etc/passwd-s3fs in which case you’ll have to specify -o accessKeyId=aaa on the command line.

Once that’s all set up, you can mount the S3 bucket mybucket at the mountpoint /mnt/mybucket, the command line is:

	# /usr/bin/s3fs mybucket /mnt/mybucket

Now, you can treat /mnt/mybucket as a regular copy destination including using it for rsync!

If you ever want to get rid of the mount, the normal unix umount command does the trick:

	# umount /mnt/mybucket

Enjoy!

Stupid Subversion on Stupid CentOS

After using Ubuntu for my day to day server for a while, I’m really realizing how much CentOS and yum suck.

oot@tequila [~]# yum -y install subversion
Loading “fastestmirror” plugin
Loading mirror speeds from cached hostfile
* base: pubmirrors.reflected.net
* updates: mirrors.gigenet.com
* addons: yum.singlehop.com
* extras: mirror.steadfast.net
Excluding Packages in global exclude list
Finished
Setting up Install Process
Parsing package install arguments
Resolving Dependencies

[...]

–> Finished Dependency Resolution
Error: Missing Dependency: perl(URI) >= 1.17 is needed by package subversion

Stupid miserable POS!

Quick Solution:

wget http://rpm.evopanel.net/rpms/perl-URI-1.35-3.noarch.rpm

rpm -ihv perl-URI-1.35-3.noarch.rpm

yum -y install subversion

Done.

Why git is Better Than Whatever You’re Using Now

Just found this interesting link:

Why Git is Better than X

Answers a lot of questions about why Git is taking over the world.

Git, SSH, How To Serve Git Securely

I’ve been using Git for a while now on my local projects. It’s quick to set up, fast, and I’ve always got my full revision history with me.

I’ve been using it locally, and backing up my projects with my normal methods but now I’ve got to share a repository.

I set up Subversion as per my previous couple of posts but, after doing so, realized that I really didn’t want to deal with Subversion any more.

In my opinion, Subversion’s conflict resolution absolutely sucks. You get three files, yours, the revision you started from, and the newest revision in files named ‘filename.r26′, ‘filename.r35′, and ‘filename.mine’ which is what the file you edited was before it spews conflicts right into your original source file to be cleaned up by hand. No thanks.

I haven’t found a tool that would make it easy to merge them and, because of that, I was always afraid to branch; way too much effort when trying to merge things back again.

We used Perforce on a project and its conflict handling was much better but at $900/seat, it’s a little hard to justify.

Git is supposedly much better than Subversion at conflict resolution and such and, as a leap of faith (and because I’ve found Subversion’s merging virtually unusable) I’m just going to set up the new project under Git and see how things go.

Into the breach!

I’m looking at the following references as I begin to figure out how to serve Git securely.

Subversion Main Page

Git Main Page

Gitosis, scripts to help with admin

Gitosis how-to

Upgrade Subversion from 1.4 to 1.5

We’re hiring some new contractors who are wanting to use Subversion (we’ve mostly switched to git for in-house use).

Figured this would be a good time to upgrade from the old 1.4 to the new 1.5 branch.

Working on one of our older CentOS 4.x boxes with Subversion 1.4.3 installed.

Download the new 1.5.4 tgz, untarred, ./configured (using the apache path info gleaned fron config.nice in the 1.4.3 source dir), installed and attempted to restart the Apache server that’s used to serve subversion.

Got an error from Apache:

httpd: Syntax error on line 60 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_dav_svn.so into server: libsvn_fs_util-1.so.0: cannot open shared object file: No such file or directory

Did a quick Google search on:

“mod_dav_svn.so into server: libsvn_fs_util-1.so.0: cannot open shared object file: No such file or directory”

And came up with:

http://www.nabble.com/Library-load-problem-with-Subversion-upgrade-td18348966.html

which ended with the advice to:

export
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib and try the ldd again.

So, I ran the aforementioned “export…” command from the command line.

NOTE: if you cut and paste from this post, make sure that the “export” and the LD_LIBRARY… are all done in one command or the environment will not be updated and you’ll still get the same error when running ldd.

I did the export, re-ran `ldd` and, sure as shootin’, there was no more error from `ldd.`

Next piece from that post:

“If it looks good that time, you need to add /usr/local/lib to ldconfig”

Hmmmm, WTF does that mean to have me do, exactly?

Turns out there’s a file, /etc/ld.so.conf that contains one line:

include ld.so.conf.d/*.conf

Inside the ld.so.conf.d directory are files specifying which paths to add to ‘ldconfig’

One of them is usrlocal.conf which contains:

/usr/local/lib

which would, you’d think, make sure that ldd would know about /usr/local/lib.

Well, you’d be right!  <strong>Except</strong> nobody told anyone to run ldconfig which you have to do by running:

ldconfig

If you’ve followed the steps above, should now include two copies of /usr/local/lib in the LD_LIBRARY_PATH environment variable which you can confirm at the command line by running:

set | grep “LD_LIBRARY_PATH”

Now, running:

/usr/local/apache2/bin/apachectl restart

works as expected and all is well with the SVN world.

Whew.  That was fun.

Other things to see:

http://blog.andrewbeacock.com/2007/10/how-to-add-shared-libraries-to-linuxs.html

http://linux.die.net/man/8/ldconfig