Advances in Data Deduplication Tech

– Ganesh Kuppuswamy, Director, Arcserve India & SAARC

Too much data, not enough time, not enough storage space, and not enough budget. Sound familiar?

Since the first mainframes, efforts have been made to optimize storage capacity requirements and data protection processes. In the open systems world, the issues mentioned above are the same as years ago when the first data deduplication technology became mainstream. Backups are failing, taking too much space, and costing way too much.

Globally, data volumes are growing exponentially and organizations of every size are struggling to manage what has become a very expensive problem. Cheaper storage helps, but is not an operationally efficient solution for many workloads. Data needs to be shrunk to more manageable levels. Too much data causes real problems for companies overprovision their backup infrastructure to anticipate rapid future growth.

Today, new technology advances are needed to combat the unstoppable and exponential growth of virtual machines and data. Here are few deduplication technologies prevails in the industry.

Target Deduplication

In recent years, organizations realized that they could not make their backup windows with traditional backup architectures. Besides backup performance, they struggled with the essential cost of storing large amounts of backup data. Storage costs exploded as backup schemas imploded. Deduplication appliances became the preferred solution to address the issue. The process involved taking backup data, optimizing it through deduplication processes and storing it on disk.

Target deduplication has worked very well and is still in use today in many environments. Target deduplication is attractive because it does not require the user to drastically change their backup software configurations or policies, rather only to change the destination of the backup streams.

Distributed deduplication

Certain vendors introduce additional costs by essentially charging for each distributed deduplication agent on each system that needs protecting. While there might be some benefits or “boost” from a disaster recovery standpoint, the multiplication of expensive systems, software licenses and associated bandwidth requirements only add to the tab. In addition, while tools may exist, this type of architecture can be onerous from a management perspective.

Source-side deduplication

Why deduplicate data after the fact if we can only backup the new and unique data needed at the source? Why not share all of the deduplicated intelligence across required across platforms? That is global source-side duplication, and it is where the industry needs to move forward. Global source-side duplication is what many end-users have recognized as the critical backup technology moving forward. The challenge of global source-side duplication is to ensure that the source (client) system is not bogged down by the deduplication software.

Adopt Unified Data Protection Strategy

With a unified data protection strategy, backup administrators can optimize storage requirements and bandwidth as well as accelerate protection and recovery across multiple sites. In addition, the solution allows for in-place re-hydration of data, for fast granular restore, including from tape.

It is the true “Global” deduplication across all the clients in the infrastructure is central to limiting the unnecessary storage and transfer of duplicate backup data. Data is deduplicated across nodes, jobs and across sites.

The global deduplication in a unified data protection strategy goes beyond the common limitations of deduplication where it only applies to the WAN replication cache, and not what is actually stored on disk, thus reducing the overall potential benefits, in terms of bandwidth and storage savings.

A right unified data protection strategy improve resilience and availability. Because the deduplication data is stored centrally in the Recovery Point Server, it is easy to protect the backup infrastructure. For example, all the information in the data store can be replicated to the cloud, another server in the same data center or offsite.

Organizations must easily and efficiently keep up with rapid data growth and manage a combined physical and virtual infrastructure, but specialized data protection ‘point’ solutions are often too limited in scope, and large-enterprise solutions can be too unwieldy for many smaller businesses.

The concept of unified data protection brings significant value to enterprises that are struggling to meet increasingly stringent RPOs and RTOs across physical, virtual and cloud environments. Economics will drive data protection consolidation as administrators look to simplify how they handle backup, recovery, and business continuity requirements in today’s heterogeneous world.