Lab overview

In this sequence of labs, you'll build a multi-server file system called Yet Another File System (yfs) in the spirit of Frangipani. At the end of all the labs, your file server architecture will look like this:

You'll write a file server process, labeled yfs above, using the FUSE toolkit. Each client host will run a copy of yfs. Each yfs will create a file system visible to applications on the same machine, and FUSE will forward application file system operations to yfs. All the yfs instances will store file system data in a single shared "extent" server, so that all client machines will see a single shared file system.

This architecture is appealing because (in principle) it shouldn't slow down very much as you add client hosts. Most of the complexity is in the per-client yfs program, so new clients make use of their own CPUs. The extent server is shared, but hopefully it's simple and fast enough to handle a large number of clients. In contrast, a conventional NFS server is pretty complex (it has a complete file system implementation) so it's more likely to be a bottleneck when shared by many NFS clients.

Lab assignments

Lab 1 - Lock Server
Lab 2 - Basic File Server
Lab 3 - MKDIR, UNLINK, and Locking
Lab 4 - Caching Lock Server
Lab 5 - Caching Extent Server + Consistency
For the rest of the assignments, you may pick one of the following two options:
Option 1:
Lab 6 - Paxos
Lab 7 - Replicated lock server
Option 2:
Team project

Collaboration Policy

You must write all the code you hand in for the lab assignments, except for code that we give you as part of the assignment. You are not allowed to look at anyone else's solution, and you are not allowed to look at code from previous years. You may discuss the assignments with other students, but you may not look at or copy each others' code. Violations of academic integrity will result in serious disciplinary actions according to NYU GSAS policy.

Programming Environment

You are required to do all labs on the class virtual machine (based on Ubuntu Linux). To get the virtual machine running on your personal desktop or laptop, take the following steps.

If you already have a Linux desktop or laptop, you may directly run programming labs there by setting up FUSE. To install FUSE, type "sudo aptitude install libfuse2 fuse-utils libfuse-dev ". Then add yourself to the FUSE group by typing "sudo adduser {your_user_name} fuse". Note that the official environment for the labs is still the virtual machine, and that's where we will be testing your code. If you are running on your own Linux machine, note that we don't have the energy or expertise to debug any problems you might have.

There has been at least one report that running the code in virtual machine is much slower than running it directly on hardware, and as a result, RPCs timeout more often even when RPC_LOSSY is unset. Be on the lookout for errors related to this.

Aids for working on labs

There are a number of resources available to help you with the lab portion of this course: