Friday, May 3, 2013

Understanding Zookeeper protocols

Credit : zookeeper.apache.org
Zookeeper is the open sourced library of cluster membership. It is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services used by company like twitter.

Though Zookeeper is most used library, but it has no concrete documentation of low-level tcp, wire protocol. The code is itself is the documentation, so to know insight code has to read. Result, there are only client bindings is available for a number of languages, but very few with pure implementation in theirs native code.

After reading code of zookeeper client in Python implementation by hannosch, (as I find java a bit verbose. But for java people see  the jute file used for RPC). Also with the help of wireshark, to inspect zookeeper packets, able to understand protocols quite well.

For Scala, I have written simple client over twitter's finagle codec to demonstrate the working of zookeeper connection.

Protocol

Connection

For establishing connection, zookeeper client send the following data as byte array over TCP connection. Client can negotiate with server over timeOutvalue. For security password is create for session id, for first time client has to send password as "\x00"*16 with session id as 0.
 
Object ConnectRequest {
  val protocolVersion : Int
  val lastZxidSeen : Long
  val timeOut : Int
  val sessionId : Long
  val passwd : Array[Byte]
}
On connection establishment, server send sessionId, password and timeOut value, which client has to follow. The response data with will be as follow.
Object ConnectRequest {
  val protocolVersion : Int
  val timeOut : Int
  val sessionId : Long
  val passwd : Array[Byte]
}

Heart-beat / Ping

If the session is idle for a period of time that would timeout the session, the client will send a PING request to keep the session alive.
Object Ping {
  val type : Int = 11
}

Close

To close connection gracefully, client send SMPP to server. Then Server send FIN to client for closing
Object Close {
  val type : Int = -11
}

CreateRequest

Creating a node, on server require acl access control along with path and data. ACL sequrity class has to be implemented.
Object CreateRequest {
  val path:String
  val data:Array[Bytes]
  val acl:Array[ACL] //ToDo: To implement ACL security layer
  val flags:Int
}

No comments:

Post a Comment