Archive for July, 2011

Setting Up a TeamDrive Personal Server

Friday, July 15th, 2011

TeamDrive Overview

If you’ve heard of DropBox, then you understand the paradigm of TeamDrive. It provides a central repository for your files that can be shared across multiple computers and platforms. TeamDrive offers a couple of advantages over DropBox: 1) encryption before uploading to the server, and 2) a personal server that you can have “in-house” on your own intranet. The major disadvantages thus far is the limited storage (10 GB) even on your own server and the lack of support for mobile devices with iOS and Android operating systems.

Why was I looking for this type of file sharing solution? See this blog entry.

TeamDrive Personal Server
TeamDrive provides a personal server (TDPS) that you can run on your own network that will do essentially all things that their servers will do for your TeamDrive clients. They provide it for all three of the major OS variants: Windows, Mac OS X, and Linux.

I could have run the server on my Mac desktop, but I really wanted to be able to have it all on my Synology DiskStation. That is when I ran into my roadblock.

The Problem
The DiskStation’s OS is based on a Debian Linux, which on the surface seemed to be an easy target for the TDPS software. The problem is the CPU in the DS1511+ is an Intel Atom and the OS leverages it with an amd64 (64-bit) build. The TDPS is 32-bit based and needs i386 (32-bit) libraries.

After weeks of searching for the right libraries, I finally gave up. TeamDrive support was not much help either. I gave up looking and was going to abandon TeamDrive all together. I came back after almost as much frustration trying an open-source project, SparkleShare. SparkleShare has promise, but is in its infancy and I needed a more stable solution.

I finally found the solution after exploring all types of avenues. The solution lies in Linux’s ability to host an OS within an OS via the chroot command. Using a somewhat dated and slightly incorrect guide, I was able to be create a 32-bit environment inside my DiskStation’s main 64-bit OS. After a bit of ‘tweaking’ and searching for a Debian package repository for ‘aptitude’, I was able to get TDPS running on the DiskStation.

TeamDrive Client-Server Model
Whether you use TeamDrive’s server or your own server, the model is essentially the same. The client on your computer creates “Spaces” on your hard (or solid state) disk drives that are mirrored on the server. Each Space is represented on your computer as a folder that you can manipulate like any other folder. The client software does all the work syncing the Space with the TeamDrive server’s repository.

Initially, when I was connected via my dated aDSL ISP service to the TeamDrive server, the syncing process was intolerably slow. Even after I upgraded to a cable connection with almost 10 times the throughput, I was still underwhelmed.

However, since I moved my central repository to my own server, the sync time, even over wireless, is negligible. I’m quite happy with the results so far.

The client has crashed a couple of times without obvious consequences (at least no yet). I’m continuing to test this and will update this blog entry with comments.

Setting Up a Network Attached Storage System (NAS)

Thursday, July 14th, 2011

Introduction

I’ve been looking for a way that allows me to access my Adobe Photoshop Lightroom 3 catalogs and pictures via my network whether at home or away.

Lightroom is not very friendly in this regard. It won’t allow sharing of catalogs across a network because it uses SQLite. If you could access your catalogs across the network and two (or more users) tried to open the same catalog there would be all kinds of problems.

One solution that has been suggested on the internet is using DropBox (or similar “cloud” based file-sharing services to share the catalog. This solution has two potential problems: 1) storage unit size / throughput issues and 2) again, concurrency and collisions if two or more users attempt to access the catalog at the same time.

The file-sharing service I use (TeamDrive) has a 5 GB storage limitation unless you pay more(most of my catalogs would push me well over the limit). Now that I’ve upgraded my ISP to cable, my upload speed has improved almost 1000%, so throughput probably may not be too much of a problem.

Concurrency would be an issue if I had another person trying to access a catalog at the same I wanted to use it. We’d each make changes and neither of us would be aware of the changes the other was making. And when the catalog was closed, the last one to close it would get his changes saved over the other’s. Not good. However, since I’m a sole proprietor, it does not effect me now.

The storage size limit, though, impacts me, so that is why I’ve opted for a NAS.

Equipment

I did a fair amount of research and chose the Synology Diskstation DS1511+ for a reliable, albeit, expensive solution. It’s a 5-bay, scalable box, i.e., you can add at least five more bays via a second box called the DX510. I opted to fill each bay with a Western Digital 2 TB Green hard drive and set them up as one Synology Hybrid RAID (SHR). Since all the drives are the same size, using SHR was probably not needed, but if I decide to expand the NAS either with bigger drives or with the DX510, I figured SHR would make that transition easier.

So, after the RAID volume is setup via SHR (with one disk redundancy) I have a total of about 7.15 TB of storage. Details…

I use Gigabit routers and switches to maximize throughput to and from my main workstation and the Diskstation.

However, the Synology NAS operating system does not natively support a “cloud” emulating service like TeamDrive Personal Server

Server Software

TeamDrive provides software application called the Personal Server (TDPS). You can run it on a Windows, Mac OS X, or Linux box. I didn’t want to have it running on my main Mac desktop workstation, so that is one reason why I opted for a NAS. The Diskstation is just a Linux box. However, installing the TDPS was far from straightforward.

First, a command-line interpreter needs to be enabled on the Diskstation. Pretty easy using the embedded DSM software. Once enabled you can access the DiskStations’s shell commands (based on BusyBox) via telnet or SSH. Details…

Despite the fact that TeamDrive provides an Intel distribution of the TDPS, it relies on 32-bit libraries. The Diskstation’s Linux OS is 64-bit without the necessary supporting 32-bit libraries, so those need to be downloaded and installed. TeamDrive has a general NAS solution, but, it doesn’t work for the Synology DiskStations in general, since the OS doesn’t support apt-get to download and install the libraries.

The Synology OS uses ipkg to download and install packages compiled and built for the Diskstation Linux OS, but that has to be enabled, too, via a bootstrap. Details…

Unfortunately, I couldn’t find the libraries to download and install…Email to TeamDrive support to see if they could direct me to the right package…They don’t know how to do this and were very little help. Possibly since they are German company, our communications were not always very clear.

But hey, TeamDrive clients also support (kind of) WebDAV servers and the DiskStation can be a WebDAV server. Eureka!!! But that, too, didn’t seem to work.

Off to do more research on the web…

SparkleShare to the rescue (I was hoping). This is an open source project that allows you to create shareable/auto-updating folders on your own server (like Synology DiskStations) with similar features as TeamDrive. I like open source software.

However, SparkleShare is still in early developmental stages and shows promise, but from my testing it was too unreliable and immature in its development to suit my needs. The another problem right now is the complexity of setting up the software on the server as well as on the client side.

Linux chroot command

My search came full circle back to trying the TeamDrive Personal Server, but this time I came across the Linux ‘chroot’ command. It allow you to “host” another OS within your running OS. It’s almost like a virtual machine.

It was a long process creating the ‘chroot’ 32-bit environment, but it is working and the Personal Server is storing all my catalogs on the NAS with copies on each of the clients.

Client Software

Other than obviously having Lightroom installed, the only other software required is the TeamDrive client. With the client you set up “Spaces” on your local system where your data are stored.

For my Lightroom catalogs, I’ve set up a Space named “GN Photo” with a sub-folder “Catalogs”. The client creates a folder on your local hard drive and the server stores the hosted files within its own repository system. Each catalog that is placed in the local “Space” gets copied to the server and each client that has been invited to share that “Space” gets a copy of the catalog.

The concurrency issue is still present, but the throughput to/from the NAS is much faster than the “cloud” based TeamDrive server. I’m sure I will end up hitting the free storage ceiling that the Personal Server supports (10 GB) and will have to pay the “rent” to increase my storage allowance, but I think that will be just another business expense that will be worth having.

Remote Access

Now, I have to figure out how to access the server from the Internet, i.e., through my router’s firewall. With access to my server via an internet address from www.dyndns.com I should be able to keep my catalogs up to date even when away from my home office.

I’ve set my router to open the TeamDrive server’s port on the NAS and setup a URL via DynDNS since my ISP dynamically allocates my IP address.

This setup allows the client software to contact my NAS when away from my home office/studio. The only issue then is that when at home the client is still accessing the server via a loopback from my home network to the DynDNS nameserver and back through my ISP. The loopback time seems short, but still noticable when compared with just accessing it via the server’s local IP address.

I’m currently researching setting up my own nameserver on my network to directing local requests to my DynDNS URL to the TeamDrive server without going outside the network.

More to follow…

Synology DiskStation DS1511+

Tuesday, July 12th, 2011

Synology DiskStation DS1511+ Network Attached Storage (NAS)

I picked the DS1511+ for based on Synology’s reputation in the NAS world, the scalability of the DS 1511+, and its Linux based operating system (OS).

I populated the 5 bays with Seagate Barracuda Green 2TB SATA 6Gb/s 64MB Cache 3.5-Inch Internal Bare Drive ST2000DL003 drives and I added 2 GB of RAM with a Kingston ValueRAM 2 GB 800MHz PC2-6400 DDR2 CL5 SODIMM Notebook Memory (KVR800D2S5/2G) DRAM module.

I must say I was originally very disappointed with the DS1511+, since it was DOA, but Amazon was very good and cross shipped another box to me without incident. The new DiskStation booted without problem, but didn’t initially recognize the extra RAM. After a couple of restarts it finally showed that it had 3 GB of RAM.

Soon, I was in “storage Nirvana” with over 7 GB of centralized storage for my network and the ability to run Time Machine backups to the DiskStation on all our Macs. I also moved my main image library to the NAS. This is great having them centrally located and easily accessible from all of my computers.

It’s also been fun exploring the 3rd party apps and other open source projects that will run on the DiskStation. It was a bit of a learning curve to get more familiar with Linux and the DiskStation’s flavor of Linux.