Homework 6 (Extra Credit. Due 4/9/2018)

Please check back regularly for updates!

Implement a Distributed Key-Value Store


For this assignment, you will implement an in-memory distributed key-value (KV) store using GO programming language. The KV store should be able to handle data larger than any one node's memory capacity. That is, at any given time, a single node might not have all the data. The deliverables will include:
  • A server program that accepts HTTP get/post/put requests from the clients and returns a valid response. The server will communicate with it's peer processes (spread across the network) to maintain a consistent view of the key-value database. All communication between the HTTP client and this server should be in JSON format.
  • A proxy/coordinator process keeps track of available servers and data stored in those servers. A client connects to the proxy/coordinator process to learn the address of a server that it should connect for performing set/get operations. The proxy server also acts as a load-balancer and ensures a uniform workload distribution among various servers.
The class as a whole will decide the API for client to proxy/coordinator communication as well as the API for client to server communication. Feel free to start a piazza thread to discuss the API. Ideally, it should be possible to swap out any of your client/proxy/server processes for a different one that implements the same API.

Conventions

  • Ensure you use a layered approach to design all your programs. For example, the socket communication layer should be independent of the replication layer and so on.