Berkeley DB Reference Guide:
Berkeley DB Replication


Synchronizing with a master

When a client detects a new replication group master, the client must synchronize with the new master before the client can process new database changes. Synchronizing is a heavyweight operation which can place a burden on both the client and the master. There are several controls an application can use to reduce the synchronization burden.

Delaying client synchronization

When a replication group has a new master, either as specified by the application or as a result of winning an election, all clients in the replication group must synchronize with the new master. This can strain the resources of the new master since a large number of clients may be attempting to communicate with and transfer records from the master. Client applications wanting to delay client synchronization should call the DB_ENV->rep_set_config method with the DB_REP_CONF_DELAYCLIENT flag. This configuration causes the client to return DB_REP_NEWMASTER from the DB_ENV->rep_process_message method as always, but the client will not proceed to synchronize with the new master.

Client applications choosing to delay synchronization in this manner are responsible for synchronizing the client environment at some future time using the DB_ENV->rep_sync method.

Client-to-client synchronization

Clients may accept and service requests from other clients. Clients request records calling the client application's transport callback function. Requests that can be satisfied by other clients will have the DB_REP_ANYWHERE flag set for the transport function. Applications may choose to send these requests to any client or to ignore the flag and send them to the site specified by the message's environment id.

Client applications may use whatever algorithm they choose to load balance requests to other clients. Any client receiving a request it is unable to satisfy will reply to the requesting client, telling the requesting client that it is unable to provide the requested information, and the original client will re-request the information. Additionally, if the original request never reaches the client to which the application sends the request, the original client will again re-request the information. In either of these cases, the re-request will have the DB_REP_REREQUEST flag set for the transport function. Applications may respond to the DB_REP_REREQUEST flag by forwarding the request to the master (as specified by the message's environment id), or by continuing to forward the request to another client.

The delayed synchronization and client-to-client synchronization features allow applications to do load balancing within replication groups. For example, consider a replication group with 5 sites, A, B, C, D and E. Site E just crashed, and site A was elected master. Sites C and D have been configured for delayed synchronization. When site B is notified that site A is a new master, it immediately synchronizes. When B finishes synchronizing with the master, the application calls the DB_ENV->rep_sync method on sites C and D to cause them to synchronize as well. Sites C and D (and E, when it has finished rebooting) can send their requests to site B, and B then bears the brunt of the work and network traffic for synchronization, making master site A available to handle the normal application load and any write requests paused by the election.

Blocked client operations

Clients in the process of synchronizing with the master block access to Berkeley DB operations. By default, most Berkeley DB methods will block until client synchronization is complete, and then the method call proceeds.

Client applications which cannot wait and would prefer an immediate error return instead of blocking, should call the DB_ENV->rep_set_config method with the DB_REP_CONF_NOWAIT flag. This configuration causes DB method calls to immediately return a DB_REP_LOCKOUT error instead of blocking, if the client is currently synchronizing with the master.

Clients too far out-of-date to synchronize

Clients attempting to synchronize with the master may discover that synchronization is not possible because the client and master have been out of contact for too long. By default, the master and client automatically detect this state and perform an internal initialization of the client. Because internal initialization requires transfer of entire databases to the client, it can take a relatively long period of time and may require database handles to be reopened in the client applications.

Client applications which cannot wait and would prefer to postpone internal initialization until a more convenient time, or would prefer to do a hot backup instead of performing internal initialization, should call the DB_ENV->rep_set_config method with the DB_REP_CONF_NOAUTOINIT flag. This configuration flag causes Berkeley DB to return DB_REP_JOIN_FAILURE to the application instead of performing internal initialization.

Client applications choosing to delay synchronization in this manner are responsible for synchronizing the client with the master at some future time. This can be accomplished by turning off the DB_REP_CONF_NOAUTOINIT flag and calling the DB_ENV->rep_sync method, or by performing a hot backup.


Copyright (c) 1996-2005 Sleepycat Software, Inc. - All rights reserved.