Homework 6 (Extra Credit. Due 4/9/2018)
Please check back regularly for updates!
Implement a Distributed Key-Value Store
For this assignment, you will implement an in-memory distributed key-value (KV)
store using GO programming language. The KV store should be able to handle
data larger than any one
node's memory capacity. That is, at any given time, a single node might
not have all the data.
The deliverables will include:
- A server program that accepts HTTP get/post/put requests from the clients and
returns a valid response. The server will communicate with it's peer
processes (spread across the network) to maintain a consistent view of
the key-value database. All communication between the HTTP client and
this server should be in JSON format.
- A proxy/coordinator process keeps track of available servers and
data stored in those servers. A
client connects to the proxy/coordinator process to learn the address
of a server that it should connect for performing set/get operations.
The proxy server also acts as a load-balancer and ensures a uniform
workload distribution among various servers.
The class as a whole will decide the API for client to proxy/coordinator
communication as well as the API for client to server communication.
Feel free to start a piazza thread to discuss the API.
Ideally, it should be possible to swap out any of your client/proxy/server
processes for a different one that implements the same API.
Conventions
- Ensure you use a layered approach to design all your programs. For
example, the socket communication layer should be independent of the
replication layer and so on.