The master and tablet servers do different things. This is confusing when you are getting started with YugabyteDB.
As seen from a YSQL database client perspective, so when using the PostgreSQL psql utility or the Yugabyte ysqlsh, using a JDBC client, or using DBeaver, the only processes to be connecting to is the endpoint of the tablet servers on the cluster nodes.
In fact, for YSQL, the PostgreSQL compatible API, a client is not actually dealing with the tablet server itself, but with a specialised PostgreSQL instance that can talk with YugabyteDB's DocDB that is started by the tablet server process. The YSQL API is available at port 5433 (default).
For YCQL, the Cassandra compatible API, a client connects to port 9042 (default) which does connect to a thread of the tablet server, but this still is a DocDB client, albeit embedded in the tablet server.
So, database client access is via a port that connects to a process or thread that acts as the database.
The Master
So how about the master? The master servers store the central cluster metadata, the PostgreSQL catalogs, and organise cluster operations, such as load balancing tablets and tablet leader placement. Please mind this is all strictly internal YugabyteDB. A database client never has to connect to or deal with the master.
The Tablet server
The tablet server provides the API/network endpoint with each tablet server, as has been described above. The database client connects to the database backend (YSQL) or threads (YCQL).
The process (YSQL) or thread (YCQL) that a client connection created on the tablet server would have access to the (local) data directly in the case of native PostgreSQL or Cassandra. This is not the case with YugabyteDB, for good reasons: the APIs act as a client to the actual distributed database (which we call DocDB) which is served by the tablet server process too, alongside the API. This architecture allows YugabyteDB to place chunks/shards of data on any tablet server that is in the cluster.
How does that work?
The way this works is that for both APIs, you connect to a random tablet server with an existing PostgreSQL or Cassandra database client. This user side client connects to the database, creating a server task, acting in a client-server fashion.
When the server task needs data, it creates a connection to the local tablet server, and requests the data. That connection performs a client-server connection between the server task and the tablet server DocDB server.
The DocDB server keeps a constantly updated map of the cluster, which is refreshed by heartbeat packets from the master, to allow to send the request to the correct DocDB server hosting the data, and receiving the result of it, which it then will pass to the server task, which will send it to the client.
More about the Tablet server
The tablet server's function is to act as the API and to serve tablets. The tablets store and contain the data, and the metadata required to function is stored in the masters. That means that the tablet servers itself without the tablets do not store any data, and as a result are not persistent cluster entities.
Let me explain what that means: the masters store the cluster metadata, and therefore protects this metadata using a RAFT group. Each tablet stores data, and therefore protects this data using a RAFT group too. Both need a single point of truth and therefore elect a leader for the RAFT group. The tablet servers do not form some sort of RAFT group, because it doesn't need that: there is nothing to protect.
Of course once a cluster is up, it needs to be sure a tablet server is available and actively performing its tasks. This status is kept by the master (the master LEADER), and works by letting the tablet servers send an heartbeat to the master leader with a certain interval.
To illustrate the point of the tablet server as entity being stateless: If you got a replication factor 3, 4 node cluster, and blacklist the 4th node, the user tablets as well as the system tablets will move to the other 3 non-blacklisted tablet servers. The 4th tablet server will remain an entity for the master, because it still sends heartbeats. If you stop the tablet server, it will become marked 'DEAD', and remain in the in the master tablet servers view because the entry does not get flushed. If you stop and start the cluster or the masters, or even just switch master leader (yb-admin master_leader_stepdown), the stopped tablet server is not visible anymore. Because there is nothing stateful for the tablet server (not even its blacklist entry, because that is a cluster property, not a tablet server property).
The logical next question is: what if we bring a tablet server down that DOES host tablets? The answer for that is more complicated, but it comes down to that the master does keep the tablet server UUID for each tablet replica, and as such then will still show the tablet server, even if it doesn't exist, and even if it didn't announce its HTTP address. This also is protection for a tablet replica.
Top comments (0)