How do I use MongoShake to implement delayed synchronization among ApsaraDB for MongoDB instances - ApsaraDB for MongoDB

This topic describes how to use MongoShake to implement delayed synchronization among ApsaraDB for MongoDB instances.

Prerequisites

The MongoShake version must be 2.4.6 or later. For more information, visit the MongoShake releases page.

Background information

If you use MongoShake to synchronize data among multiple instances in real time, MongoShake synchronizes misoperations on the primary instance to secondary instances. In this case, you must restore the data of the secondary instance to undo the misoperations. To resolve this issue, a parameter is added to MongoShake version 2.4.6 to allow delayed synchronization, which provides a buffer period for data synchronization among ApsaraDB for MongoDB instances. If an operation is accidentally performed on the primary instance, you can disable synchronization during the buffer period and migrate your workloads to a secondary instance to which the incorrect data is not synchronized.

Note

This topic describes how to use the incr_sync.target_delay parameter. For more information about how to use MongoShake, see Use MongoShake to perform one-way synchronization between ApsaraDB for MongoDB instances.

Preparations

For best synchronization performance, make sure that the source ApsaraDB for MongoDB replica set instance resides in a virtual private cloud (VPC). If the source instance resides in the classic network, switch the network type to VPC. For more information, see Switch the network type of an ApsaraDB for MongoDB instance from classic network to VPC.
Create an ApsaraDB for MongoDB replica set instance as the synchronization destination. Select the same VPC as that used by the source ApsaraDB for MongoDB replica set instance to minimize network latency. For more information, see Create a replica set instance.
Create an Elastic Compute Service (ECS) instance to run MongoShake. Select the same VPC as that used by the source ApsaraDB for MongoDB instance to minimize network latency. For more information, see Create an ECS instance.
Add the private IP address of the ECS instance to the whitelists of the source and destination ApsaraDB for MongoDB instances. Make sure that the ECS instance can connect to the source and destination ApsaraDB for MongoDB instances. For more information, see Modify the IP address whitelist of an ApsaraDB for MongoDB instance.

Note

If the network type does not meet the preceding requirements, you can apply for public endpoints for the source and destination ApsaraDB for MongoDB instances. Then, add the public IP address of the ECS instance to the whitelists of the source and destination ApsaraDB for MongoDB instances. This way, you can synchronize data over the Internet. For more information, see Apply for a public endpoint and Modify the IP address whitelist of an ApsaraDB for MongoDB instance.

Configure delayed synchronization among ApsaraDB for MongoDB instances

The following example uses an Ubuntu Elastic Compute Service (ECS) instance to describe how to configure delayed synchronization among ApsaraDB for MongoDB instances. For more information, see Use MongoShake to perform one-way synchronization between ApsaraDB for MongoDB instances.

Log on to the ECS instance. For more information, see Connect to a Linux instance by using a username and password from a third-party client.
Run the following command to download the MongoShake package:
```
wget <Download URL of the latest MongoShake package>
```
Sample code:
```
wget https://github.com/alibaba/MongoShake/releases/download/release-v2.0.7-20190817/mongo-shake-2.0.7.tar.gz
```
Note
For more information about the download URL of the latest MongoShake package, visit the MongoShake releases page.
Run the following command to decompress the MongoShake package:
```
tar xvf <Name of the MongoShake package>
```
Sample code:
```
tar xvf mongo-shake-2.0.7.tar.gz
```
Run the vi collector.conf command to configure MongoShake. For information about the parameters that you must configure, see MongoShake parameters. Find the incr_sync.target_delay parameter in the collector.conf file and configure this parameter based on your business requirements. The parameter value is measured in seconds. In this example, the buffer period is set to 1,800 seconds.
```
incr_sync.target_delay = 1800
```
Save and close the collector.conf file to complete the configuration.
Run the following command to start synchronization based on the collector.conf file and print logs:
```
./collector.linux -conf=collector.conf -verbose
```
Note
MongoShake synchronizes changes in the primary instance to secondary instances 30 minutes after the changes.

Perform a switchover between the primary instance and a secondary instance after a misoperation

When you perform create, read, update, and delete (CURD) operations on the primary instance, a misoperation may occur. For example, a statement is executed to write unwanted data to the primary instance. To respond to a misoperation, you can follow these steps to migrate your workloads to a secondary instance to which the incorrect data is not synchronized.

Query the oplogs of the primary ApsaraDB for MongoDB instance to identify the time when the misoperation occurred. For example, you can run the following command to query all oplogs that were generated from June 1, 2020 to June 2, 2020. For more information about how to query oplogs, see MongoDB official documentation.
```
use local#Switch to the local database.
db.oplog.rs.find({"o.createTime": {$gte:new Date(2020,6,1),$lte:new Date(2020,6,2)}}) #Query oplogs based on the specified conditions.
```
Run the following RESTful API command to inject the ExitPoint parameter to MongoShake to terminate the MongoShake program at a specified point in time:
```
curl -X POST --data '{"ExitPoint": <Unix timestamp>}' <MongoShake server ID>:<Port number>/sentinel/options
```
Sample code:
```
curl -X POST --data '{"ExitPoint": 1593534600}' 127.0.0.1:9100/sentinel/options
```
Note
The string 1593534600 is a UNIX timestamp that indicates 16:30:00 June 30, 2020. MongoShake automatically exits at the specified point in time.
Run the vi collector.conf command to open the collector.conf file and exchange the IP addresses of the primary instance and the secondary instance. For more information, see Use MongoShake to perform one-way synchronization between ApsaraDB for MongoDB instances.
Run the following command to restart synchronization based on the collector.conf file and print the logs:
```
./collector.linux -conf=collector.conf -verbose
```
Migrate your workloads to the new primary instance to complete the switchover operation.

Monitor the MongoShake status

For more information, see Monitor the MongoShake status.