Projects‎ > ‎

Project04

xv6c: Adding Containers to xv6

Due Date #1 Tuesday November 28th at 11:59pm. Turn into your group Project04 git repo.
Due Date #2 Tuesday December 5th at 11:59pm. Turn into your group Project04 git repo.

You can work in groups of up to two students. Let me know your group by Tuesday November 14th at 11:59pm. If you do not form group I assume you will be working alone.

Containers provide an form of isolation that is in general lighter weight than virtual machines. The goal is to create protected environments in which a process or group of processes appear to be running on a dedicated machine, but in fact they are running in a container on a machine that is possibly running other containers as well. Unlike virtual machines, all the containers running on a machine share the same kernel. However, the the kernel supports the container abstraction and isolation among containers.

 There are three main types of container isolation:
  1. Process group isolation
  2. File system isolation
  3. Resource isolation 
We are going to focus on (1) and (2). The third (3) type of isolation can be explored in extra credit.

For process group isolation we need to create separate "namespaces" for each container. That is, processes in one container will have different PIDs than processes in another container. Said another way, processes in one container should not be able to "see or touch" processes in another container. In terms of xv6 this means that a process in one container cannot kill a process in another container. In addition, processes from two different containers cannot share pipes. 

Processes in a single container have a "virtual" PID space. You can have a process in container 1 with a PID of 1 and a process in container 2 with a PID of 1. This should not be an issue for the kernel.

You will also need to enforce three types of process execution restrictions. You need to enforce a maximum number of simultaneously running process, maximum amount of memory for a container, and fair share scheduling of containers.

In addition to process group isolation, process groups should be scheduled in a fair share manner (https://en.wikipedia.org/wiki/Fair-share_scheduling). This means that each container should receive an equal portion of CPU time, not each process in the system.  Say you have two containers: A and B. Assume container A has 1 process and container B has 2 processes and all the process are running at the same time. The process in container A should receive 50% of the CPU time and the two processes in container B should receive the other 50% of CPU split among them so that process B:1 gets 25% and process B:2 gets 25%. You can imagine other scenarios involving more processes and more containers.

For File System isolation, we want each container to have access to it's own virtual file system on top of the real file system. This can be achieved in part with the chroot() system call (https://en.wikipedia.org/wiki/Chroot) which allows you to change the default root directory for an individual process. In our case we want to perform chroot() for an entire container so that all processes with in a container see the same root directory. You will provide some user level tools (applications) that allow the creation of a container and the ability to populate a subdirectory with specific applications for that container (like a specific shell implementation). When the container is launched only the files in the containers file system will be accessible. You can use a directory as the virtual file system for a container. If you want, you can also put container onto a separate disk or partition.

If any of the resource limits are exceeded you should kill the offending container and all of it's processes.

Finally, resource isolation allows other devices to be shared fairly among containers just like the CPU. For example, the hard disk and the network interfaces should be scheduled in a way that each container gets their fair share. This type of scheduling isolation is important in real container systems, but you will not be required to implement resource isolation for this project. I will make this an extra credit opportunity.

Deliverables

  1. A modified xv6 kernel called xv6c that supports containers.
  2. User level tools that allow you to:
    • (ps) A ps command that lists processes
      • In a container, it should only show the processes running in that container
      • In the root container, it should show all processes and identify the container for each process
    • (free) A free command that show the available and used memory
      • In a container, it should only show the available and used memory within a container
      • In the root container, it should show all available and used memory
    • (df) Disk space usage
      • In a container, it should show only the total disk space a disk space used by the container
      • In the root container it should show the total disk space available and used
    • (cinfo) A cinfo command that can only execute in the root container
      • It will show each container
      • The name of the container
      • The directory associated with the container
      • The max number of processes, the max amount of memory allocated, and the max disk space allocated
      • The amount of used/available processes, memory, and disk space
      • The processes running in the container
      • The execution statistics and percent of CPU consumed by each process and each container
    • (ccreate) Create a container and populate a container FS
    • (cstart) Start a container - give a virtual console, container directory, an initial program to start, max number of processes, max mem, max disk space (you can have default values for the resources)
    • (cpause) Pause a container
    • (cresume) Resume a container
    • (cstop) Stop a container
    • If you want, you can create a single container tool (ctool) that take a subcommand:
      • ctool info
      • ctool create c1 sh ls echo ps
      • ctool start vc0 c1 sh
    • Stress test tools that exceed the resource limits
    • Programs that can show that fair scheduling is achieved
  3. Experiments that demonstrate:
    • You are scheduling container process correctly
    • That containers are process isolated (a process in one container cannot kill a process in another container)
    • That containers are file system isolated (a process running in a container should only be able to access the files in the container subdirectory and lower)
    • Containers do not exceed their resource limits
  4. A Design Document that explains your over all design and implementation of containers in xv6
  5. Top 2 projects can present their xv6c implementation at CS Night

Preliminary Work

Virtual Console Support

Larger File System

Extra Credit (1 point each)

  • Fair resource scheduling (e.g., the disk) code and demonstration
  • Ability to suspend a container state to disk, then load and resume the state later (say after shutting down the machine).
  • Propose other extra credit ideas to me.

Comments