A Guide to Change Block Tracking in Data Management

1 CBT : A Technical Review Absract Change Block Tracking or Change Tracking or CBT is making a record or journal of which blocks of a storage object were changed by I/O. A storage object is any logical- ly contiguous space used for storage of data. Now you need to keep in mind that CBT doesn’t keep track of what data was written, it only logs which areas of the object were written. Such track can be used for various applications, primarily for backup and restore. Usually such tracking is used to know what blocks have changed during a given time frame.

2 CBT : A Technical Review Backup uses snapshot technologies? So where does CBT come in all of this? Backup technologies rely on snapshot to make the backup consistent, however it’s very tedious to take backups of the entire object at every backup cycle. So what you do is either you use incremental or differential snapshots. Using incremental snapshots, one can know what has changed between two snapshots. Backup applications use this information to know what blocks of the object to backup. Let’s look at the following example of backup using snapshots. At the first backup cycle, we take a snapshot of the device and we take a full backup using the snapshot. At the next backup cycle we take an incremental snapshot of the device. This snapshot logic is already provided by the block title. Next we calculate what has changed between the two snapshots and take the backup of only those

3 CBT : A Technical Review We can notice here that: 1) The snapshot has to exist not only for the period of backup, but for over two backup cycles. It needs to exist for more than two backup cycles. 2) Production I/O has always the need to go through this snapshot layer. Meaning that if you have the snapshot in the I/O path, then every I/O that you do may translate into more than one read/write based on the type of the snapshot. 3) These snapshots have to live for the longer time and the size of the snapshot will keep growing as much as you do I/O to it. When you want to delete the snapshot, it will be time consuming and tedious because these snapshots are so large that it takes time to manage all the data in the snapshot areas. The effect of point no. 2&3 is that they are multiplied when the backup interval is large. Imagine that your backup interval is read/write, now in case of redirect on write snapshots it would mean that you are merging the entire reads of data from the data area onto the base device. Will all these points degrade the performance of the produc�on system? That’s true and it’s actually one of the problems that needs to be solved. Backup applications used to relay on snapshots to know what has changed but CBT provides them with information out of band. These backup applications don’t need to keep the snapshots alive after the backup is complete.

4 CBT : A Technical Review How Backup with CBT works To take a backup you need to know what has changed between two snapshots, otherwise you need to keep taking full backup every time, but for this to happen a new tracking instant has to be started atomically at the creation of the snapshot. However changing or integrating with the existing snapshot technology is difficult and is something that’s not necessary. This problem is easily solved by using two simultaneous tracking instances that over- lap across the snapshot creation operation. For Example this is the timeline that is given here as shown in the graph. Starting from left to right, you first start tracking the instance “T1-0” and you take a snapshot (S-0) of the device followed by a full backup. You always start with a full backup. As soon as your backup is done you delete the snapshot. This is because you already have a tracking out of band in place.

5 CBT : A Technical Review In the next backup cycle, which is maybe a day later, a month, a week, or else, you start another tracking instance which is let’s say “TI-1” and then you follow it with another snapshot (S-1). Now after taking the snapshot you stop tracking instance “TI-0” which you have started a week before. Now the tracking instance TI-0 has all changes that has happened to that storage object in that particular week. You use that instance “TI-0” to take an incremental backup, so backup has to know which blocks have changed so it can just read that information and take the backup. Once your backup is complete, you do not need that particular snapshot anymore. You can see that the snapshot only exists till the time the backup needs it and it doesn’t need to live anymore. Essentially there are no incremental snapshots, there’s just one snapshot alive that you need for each backup. Components that encompass the CBT There are two major components that are required for CBT: 1) The hook or the filter. What we mean by hook is something that allows you to take a peek at the I/O on the storage object. Whenever an application to the I/O to that particular storage object, you need to know which block of the storage object it’s touching. The most important requirement for such a hook is that it has to be fast, why? Because the problem that we are trying to solve by using CBT is to lessen the delay on the I/O in case of snapshots, and the same needs to hold to here. This is actually easily solved if you know you have an intermediate fast cache that’s available there for the hook to record all the tracking. This tracking Data can be asynchronously returned to Persistent Record later. Also the hook isn’t concerned with whether the I/O is successful or not.

6 CBT : A Technical Review 2) Persistent Record. It’s not necessary that you have this record on disk, it can be in memory record as well or it can be something like a map disk cover. The format of this persistent record should be in such a way that it’s easily consumable. For instance you have various formats, you have a lock structured format where the tracking data is appended to the end of the record. However it may result in more burden. For example to know what has changed between a given time frame you need to go to the entire log to figure out the areas. VMware uses such a structure because it uses CTK file. Another format of Persistent Record is bitmap which images every bit it represents and logical extent of the storage object. Built bit is set if the logical extent is partially or fully return. If the logical extent is 256 kb even if you write 4 kb of that particular extent, the block is considered as return. This essentially avoids having large bitmap sizes. A multiple bitmap can represent multiple tracking instances, so it’s relatively easy to consume. Can opera�ons like merging of bitmaps be performed? Merging of bitmaps is another operation that can be performed. If you have one tracking instance represented by bitmap and another tracking instance represented by another bitmap, and you want to know what data has changed across both instances, you just need to merge the data and you get the needed data. Things to keep in mind when designing CBT One of the most important components of CBT is the hook or filter. And the problem that needs solving is performance degradation in production I/O. The hook has to be fast enough so as to not degrade the I/O performance. Most important than that it has to capture all the I/O paths. You can never miss tracking one I/O path to that particular block, or device or storage object because if you do the tracking data becomes inconsistent, and it results in a corrupted backup. Moreover, the corruption

7 CBT : A Technical Review message will appear during restore, which is much later in the time frame. You won’t know that during backup, it’s only during the restore and you don’t want that hap- pening. Scenarios that come into play with CBT Some key scenarios that come into mind is during identification of the storage object. Basically your backup application and your tracking filter or the tracking soft- ware, they have to be in consensus on how to identify storage object. It’s simple. For instance if you are tracking SAN (Storage Area Network) devices then SCSI (Small Computer System Interface) ID is one identifier. Another Scenario is reboot. Sometimes a device is accessible in an early stage of the boot. If you are tracking that device, and since one of our goals is not to miss any I/O, your tracking filter or hook needs to be in place before that. Also during shutdown no writes or I/O should be missed. If you are unable to capture or track all the I/O’s during shutdown and boot, you take a full backup and you start a new tracking instance. Another scenario is device being renamed. Of course if you’re using the unique identifier you’re not in so much of trouble. Linux also allows for the storage object to be offline and come back online as a separate device. CBT needs to know that device it was tracking is now coming up as a separate name.

www.stonefly.com 2865, 2869 and 2879 Grove Way, Castro Valley, CA 94546 USA. +1 (510) 265-1616

A Guide to Change Block Tracking in Data Management