This funding is going to be used by Druva to make a strong push into cloud-based backup. Cloud infrastructure for a bandwidth and storage intensive like backup can be a significant expense, and of course, sales and marketing too.

A few weeks back a partially inaccurate version of this story had been leaked by Economic Times and was reported by PuneTech, but we “withdrew” the story after Druva called us up and let us know that it was premature to talk about it. Talking about a company’s funding round before everything is finalized and the money is in the bank is dangerous for a number of reasons including:

• Funding is a tricky thing and there are no guarantees until the money is in the bank. Many things can, and do go wrong. One bad day on the stock market can cause VCs to reconsider any deals that are not final.
• From the time the startup received a term-sheet from the VC until the deal is finalized, there is usually a no shopping clause which prevents the startup from talking about the details of the deal with anybody else. This is to ensure that the startup does not use this offer to try and create a bidding war between VCs. Hence, if the details leak out the VCs might feel that the startup is trying to violate the no shopping clause
• Most importantly, if word leaks out that a VC is funding a company for amount X, then in next few days is is possible that the VC’s contacts in the industry (probably other VCs) keep saying “Why are you paying X? I don’t think it is worth more than Y?” and this can cause the VC to reconsider the deal. This is very dangerous for the startup.

This time however, the news is official (and is actually better than the deal reported by Indian Express).

As for what Druva does exactly, and why it is one of our favorite Pune companies, just read the previous article, which had a bunch of links. Here are some other interesting tidbits about Druva:

• “Druva’s disruptive innovation reduces the storage footprint and bandwidth requirement for backup by orders of magnitude compared to other industry solutions” -Jishnu Bhattacharjee, Nexus
• Druva, founded in 2007, has amassed more than 750 customers and protects more than 300,000 endpoints (i.e. servers, laptops, PCs) worldwide
• InSync’s global, source-based deduplication reduces bandwidth and storage by 90 percent while providing 100 percent accuracy for Microsoft Outlook and Office applications

Here’s the full press release regarding this news

Here is a comment from Jaspreet Singh, CEO of Druva:

Thanks Navin, but this news is not very accurate. This was unethically leaked and then misreported by Peerzada (abrar.shz@timesgroup.com) of ET for some cheap thrills.

Not sure when would people this these grow up and stop screwing lives of entrepreneurs who are already fighting against all the odds.

You have been a great supporter and I would give you a call sometime next week to give accurate information and some more good news.

Basically, Druva is indeed in an advanced stage in their second round funding process, but it is not done yet, and they cannot talk about the details of the amount or the investors involved. The details that came out in the ET report are inaccurate.

We wish Druva luck, and hope to hear the official good news sometime soon.

Pune-based startup Druva, which sells enterprise backup software, has just closed a second round of funding worth $10 million from Nexus Venture Partners, reports Economic Times.

In April 2010, Druva had raised $5 million from Sequoia and the Indian Angel Network. At that time, these are the reasons we gave for why we liked Druva:

• Druva is a purely homegrown startup. This is not a company started by someone in the US setting up a development center in India.
• Druva is a product startup. It is not a services company. Hence, it has a potential for exponential growth and returns.
• Druva is not done by serial entrepreneurs. The co-founders are all first-time entrepreneurs who quit their big-company jobs to start Druva. This should give hope to all the first-time entrepreneurs in Pune.

Druva has been one of PuneTech’s favorite startups and we have covered it extensively in the past, so, frankly, there isn’t much new that we’ll be able to say about it. Instead, we’ll simply point readers to the older articles:

• “World-class software products can come out of India” – Interview with Jaspreet Singh, CEO of Druva
• Druva: From proto.in presenter to proto.in sponsor in 18 months
• Understanding Data De-duplication – an article about an underlying technology that gives Druva an edge over its competitors
• Technology Overview – Druva Continuous Data Protection – An article about Continuous Data Protection, the first product that Druva came out with. As far as we understand, Druva is no longer selling this product. This is an interesting lesson on how software startups have to ‘pivot’ and change their product line in response to market demands, and how things can go in a completely different direction than what founders originally envisaged
• Understanding RTO and RPO in backups – A simple tech overview of the important parameters on which a backup solution should be evaluated.

We also want to point out that Druva is one of the sponsors of PyCon – the International Python Conference that’s happening in Pune next month.

We wish Druva luck, and although getting another round of VC funding is not as good an indicator of success as an IPO or an acquisition, we would still like to repeat what we said in April 2010:

• We now have in our midst a startup success story that will hopefully inspire a 100 new software product startups in Pune.

Vaultize is next generation data protection: cloud-based backup and disaster recovery that also enables collaboration between users, synchronization of devices and sharing over web. Vaultize turns your zero-returns investment in backup into an asset that improves availability, increases productivity and makes sharing easy.

Anoosmar Technoloies has been founded by Anand Kekre and Ankur Panchbudhe, both of whom are Pune old-timers, with an ex-Veritas (Symantec), and ex-McAfee background. Both of them have been in the data protection, security, and storage space for over 10 years, and have deep expertise in enterprise infrastructure software. Between them they have 64 US patents.

Before you dismiss Vaultize by comparing it with Dropbox, or , remember that Vaultize is not a consumer product – it is targeting the enterprise space. In that sense, I see Vaultize as more of a competitor to Pune’s Druva. However, given the backgrounds of the founders of Druva and founders of Vaultize, I would be tempted to guess that Druva is likely to be more interested in enterprise backup, and replication and generally areas more to do with performance and availability in an enterprise, while Vaultize is likely to move more in the direction of archiving, and e-discovery and generally areas more to do with risk management and legal compliance. But that’s pure speculation – I might be wrong.

Also check out the customer case studies page and the management team page.

Druva is one of the few Pune software product companies that has received funding from well known VCs, and hence, Anoosmar, which has a similar pedigree and similar target markets, is a company to watch closely.

PuneTech and the Pune Open Coffee Club both started about 2 years ago, and the steadily increasing memberships and vitality of these communities points to a very strong startup community in Pune. However, throughout those two years, one question always cast a doubt on the long-term potential of this startup ecosystem. And that question was: Where are the success stories?

Druva Software is a Pune-based backup software product startup. Click on the logo to see all PuneTech articles about backup software (mostly about Druva)

Druva software (previously known as Druvaa) which just closed a $5 million round of funding led by Sequoia Capital answers that question. Of course, getting a round of VC funding is not as good an indicator of success as an IPO or an acquisition. And of course, there have been other successes in the past. But still this news is great, for the following reasons:

• Druva is a purely homegrown startup. This is not a company started by someone in the US setting up a development center in India.
• Druva is a product startup. It is not a services company. Hence, it has a potential for exponential growth and returns.
• Druva is not done by serial entrepreneurs. The co-founders are all first-time entrepreneurs who quit their big-company jobs to start Druva. This should give hope to all the first-time entrepreneurs in Pune.
• There haven’t been many high-profile successes in recent times, and this one comes as a breath of fresh air.

Druva has been one of PuneTech’s favorite startups. With 5 different PuneTech articles, this is probably the company that has received maximum coverage from us. And a quick look at the articles gives hints as to why:

• It is a product company, which is always more interesting than a services company; it’s especially interesting to watch the product evolve over time.
• It requires some very complex technology, not something that any company could easily build. Plus, they are happy to write detailed technical articles about the technology that underlies their products.
• It has repeatedly featured in high profile startup events in India, from proto.in to the NASSCOM summit

PuneTech spoke to Jaspreet Singh, CEO of Druva, over the phone, and here are some quick notes based on this conversation. There are a number of unique features here that other Pune entrepreneurs would do well to take note of.

On the current state of the company

Druva has $2.5 million revenue run rate, coming from about 400+ customer deployments. Most of this is from their flagship product, the inSync remote laptop disk-to-disk backup solution. Recently they also introduced Phoenix a remote server disk-to-disk backup solution. They have about 23 employees, most of them in Pune, with a few sales people elsewhere. The product is developed entirely in Pune.

How do they manage enterprise support for 400 customers with such a small employee base?

Although supporting their customers is a very high priority for Druva, one of the things they focus on very hard is to make the product very easy to use and very easy to support – so that to a large extent, most of their customers don’t really require any support. They have a “release often” philosophy which ensures that customers always have the latest, bug-fixed, version of the software.

Another area that they put a lot of effort in, is in ensuring that the product is easy to install. A lot of their customer testimonials speak of how easy it was to self-install the software. By contrast, the comparable software from the more established players in the market requires professional services help for installation.

How do they manage sales without a strong US/Europe presence?

Instead of the tradition of hand-holding that is a common feature of enterprise sales in this domain, Druva decided to go a different route. They made their software freely downloadable from the web, and made it easy to install and try. As a result, most of their customers approach them after having first tried the product out via the website. And many of their sales, even large ones, have happened over skype/email, with no in-person customer visits.

How do they compete with the large MNCs, the established players in this market?

We were very surprised to learn that Druva does not try to compete with the incumbents on cost. Jaspreet told us that in fact the average Druva sale tends to be 3x more expensive than the comparative offering from the established players. Druva scores on ease of use, simplicity, and most imporantly, the technology.

Jaspreet points out that one of Druva’s strong points is the easy-to-use source-level de-duplication. Which means that when backing up a laptop, they can ignore duplicate content even before the data is sent to the remote backup server. Specifically consider the gigabytes of windows operating system files on your laptop. Most of these files are likely to be identical across all laptops of a company. Druva’s software would know beforehand that there is a copy of those files on the backup server, and would never send it across. Such optimizations ensure that backing up 15 TeraBytes of data from a number of different laptops just results in about 2 or 3 TeraBytes being send across the network. This results in an increase in speed, reduction in network bandwidth consumed, and in disk-space consumed.

By contrast, traditional backup systems do de-duplication at the destination. Which means that all the data is sent to the server over the network, and only then is the server able to remove duplicate content. This means that the speed and network bandwidth improvements are lost.

Also, claims Jaspreet, Druva’s backups are fully searchable – a feature that is not available with most competitors.

What is their primary challenge currently?

Jaspreet says that they want to build a high-quality, world-class product, and for that he needs lots of high-quality, world-class people. While they’ve obviously managed to build a team like that which got them so far, they need many more such people in the coming days, and that’s a significant challenge. He says that it is difficult, if not impossible to find “readymade” world-class talent here (even when “world-class” salary and/or equity is offered!). Instead, he feels that the only approach that works is to find individuals (whether freshers or industry veterans) who have the right attitude and potential and then nurture them into the required shape.

(As an aside, we’d like to point out that is a pattern. Pretty much every startup we talk to mentions hiring of high-quality people as one of their primary challenges. This is a problem that needs a solution, and I’m hoping that some entrepreneur in Pune is looking at this as an opportunity.)

Parting thoughts: In the Druva co-founders, we have people who have been through the entire process, from zero to VC-funding, in Pune, recently. And they are nice guys. Pune entrepreneurs should take advantage of this, and flock to them for guidance, advice and mentorship.

Note: Please turn up the volume. The sound quality is not-so-great. (Hopefully future videos will be better.)

Please also check out the older PuneTech articles about Druvaa:

• Understanding Data De-duplication
• Technology Overview – Druvaa Continuous Data Protection
• Druvaa Presentation at proto.in 3
• Understanding RPO and RTO in Backups

Interesting note: You’ll notice that over the years, Druvaa has shifted gears from selling continuous protection (which they started off with) to remote backups (which is their primary product now). This is a feature of any startup – adapting to the needs of the market.

Druvaa is a Pune-based startup that sells fast, efficient, and cheap backup (Update: see the comments section for Druvaa’s comments on my use of the word “cheap” here – apparently they sell even in cases where their product is priced above the competing offerings) software for enterprises and SMEs. It makes heavy use of data de-duplication technology to deliver on the promise of speed and low-bandwidth consumption. In this article, reproduced with permission from their blog, they explain what exactly data de-duplication is and how it works.

Definition of Data De-duplication

Data deduplication or Single Instancing essentially refers to the elimination of redundant data. In the deduplication process, duplicate data is deleted, leaving only one copy (single instance) of the data to be stored. However, indexing of all data is still retained should that data ever be required.

Example
A typical email system might contain 100 instances of the same 1 MB file attachment. If the email platform is backed up or archived, all 100 instances are saved, requiring 100 MB storage space. With data deduplication, only one instance of the attachment is actually stored; each subsequent instance is just referenced back to the one saved copy reducing storage and bandwidth demand to only 1 MB.

Technological Classification

The practical benefits of this technology depend upon various factors like –

1. Point of Application – Source Vs Target
2. Time of Application – Inline vs Post-Process
3. Granularity – File vs Sub-File level
4. Algorithm – Fixed size blocks Vs Variable length data segments

A simple relation between these factors can be explained using the diagram below -

Target Vs Source based Deduplication

Target based deduplication acts on the target data storage media. In this case the client is unmodified and not aware of any deduplication. The deduplication engine can embedded in the hardware array, which can be used as NAS/SAN device with deduplication capabilities. Alternatively it can also be offered as an independent software or hardware appliance which acts as intermediary between backup server and storage arrays. In both cases it improves only the storage utilization.

On the contrary Source based deduplication acts on the data at the source before it’s moved. A deduplication aware backup agent is installed on the client which backs up only unique data. The result is improved bandwidth and storage utilization. But, this imposes additional computational load on the backup client.

Inline Vs Post-process Deduplication

In target based deduplication, the deduplication engine can either process data for duplicates in real time (i.e. as and when its send to target) or after its been stored in the target storage.

The former is called inline deduplication. The obvious advantages are -

1. Increase in overall efficiency as data is only passed and processed once
2. The processed data is instantaneously available for post storage processes like recovery and replication reducing the RPO and RTO window.

the disadvantages are -

1. Decrease in write throughput
2. Extent of deduplication is less – Only fixed-length block deduplication approach can be use

The inline deduplication only processed incoming raw blocks and does not have any knowledge of the files or file-structure. This forces it to use the fixed-length block approach (discussed in details later).

The post-process deduplication asynchronously acts on the stored data. And has an exact opposite effect on advantages and disadvantages of the inline deduplication listed above.

File vs Sub-file Level Deduplication

The duplicate removal algorithm can be applied on full file or sub-file levels. Full file level duplicates can be easily eliminated by calculating single checksum of the complete file data and comparing it against existing checksums of already backed up files. It’s simple and fast, but the extent of deduplication is very less, as it does not address the problem of duplicate content found inside different files or data-sets (e.g. emails).

The sub-file level deduplication technique breaks the file into smaller fixed or variable size blocks, and then uses standard hash based algorithm to find similar blocks.

Fixed-Length Blocks v/s Variable-Length Data Segments

Fixed-length block approach, as the name suggests, divides the files into fixed size length blocks and uses simple checksum (MD5/SHA etc.) based approach to find duplicates. Although it’s possible to look for repeated blocks, the approach provides very limited effectiveness. The reason is that the primary opportunity for data reduction is in finding duplicate blocks in two transmitted datasets that are made up mostly – but not completely – of the same data segments.

For example, similar data blocks may be present at different offsets in two different datasets. In other words the block boundary of similar data may be different. This is very common when some bytes are inserted in a file, and when the changed file processes again and divides into fixed-length blocks, all blocks appear to have changed.

Therefore, two datasets with a small amount of difference are likely to have very few identical fixed length blocks.

Variable-Length Data Segment technology divides the data stream into variable length data segments using a methodology that can find the same block boundaries in different locations and contexts. This allows the boundaries to “float” within the data stream so that changes in one part of the dataset have little or no impact on the boundaries in other locations of the dataset.

ROI Benefits

Each organization has a capacity to generate data. The extent of savings depends upon – but not directly proportional to – the number of applications or end users generating data. Overall the deduplication savings depend upon following parameters –

1. No. of applications or end users generating data
2. Total data
3. Daily change in data
4. Type of data (emails/ documents/ media etc.)
5. Backup policy (weekly-full – daily-incremental or daily-full)
6. Retention period (90 days, 1 year etc.)
7. Deduplication technology in place

The actual benefits of deduplication are realized once the same dataset is processed multiple times over a span of time for weekly/daily backups. This is especially true for variable length data segment technology which has a much better capability for dealing with arbitrary byte insertions.

Numbers
While some vendors claim 1:300 ratios of bandwidth/storage saving. Our customer statistics show that, the results are between 1:4 to 1:50 for source based deduplication.

Related articles by Zemanta

• Tape is alive and well for storage
• Better storage management key to success
• Dell, Quantum, EMC line up on de-duplication
• Mission Impossible: Restoring Exchange in 30 Seconds
• Cleversafe Goes Open Source With its Storage Software

Druvaa, a Pune-based product startup that makes data protection (i.e. backup and replication) software targeted towards the enterprise market, has been all over the Indian startup scene recently. It was one of the few Pune startups to be funded in recent times (Rs. 1 crore by Indian Angel Network). It was one of the three startups that won the TiE-Canaan Entrepreneural challenge in July this year. It was one of the three startups chosen to present at the showcase of emerging product companies at the NASSCOMM product conclave 2008.

And this is not confined to national boundaries. It is one of only two (as far as I know) Pune-based companies to be featured in TechCrunch (actually TechCrunchIT), one of the most influential tech blogs in the world (the other Pune company featured in TechCrunch is Pubmatic).

Why all this attention for Druvaa? Other than the fact that it has a very strong team that is executing quite well, I think two things stand out:

• It is one of the few Indian product startups that are targeting the enterprise market. This is a very difficult market to break into, both, because of the risk averse nature of the customers, and the very long sales cycles.
• Unlike many other startups (especially consumer oriented web-2.0 startups), Druvaa’s products require some seriously difficult technology.

Rediff has a nice interview with the three co-founders of Druvaa, Ramani Kothundaraman, Milind Borate and Jaspreet Singh, which you should read to get an idea of their background, why they started Druvaa, and their journey so far. Druvaa also has a very interesting and active blog where they talk technology, and is worth reading on a regular basis.

The rest of this article talks about their technology.

Druvaa has two main products. Druvaa inSync allows enterprise desktop and laptop PCs to be backed up to a central server with over 90% savings in bandwidth and disk storage utilization. Druvaa Replicator allows replication of data from a production server to a secondary server near-synchronously and non-disruptively.

We now dig deeper into each of these products to give you a feel for the complex technology that goes into them. If you are not really interested in the technology, skip to the end of the article and come back tomorrow when we’ll be back to talking about google keyword searches and web-2.0 and other such things.

Druvaa Replicator

Overall schematic set-up for Druvaa Replicator

This is Druvaa’s first product, and is a good example of how something that seems simple to you and me can become insanely complicated when the customer is an enterprise. The problem seems rather simple: imagine an enterprise server that needs to be on, serving customer requests, all the time. If this server crashes for some reason, there needs to be a standby server that can immediately take over. This is the easy part. The problem is that the standby server needs to have a copy of the all the latest data, so that no data is lost (or at least very little data is lost). To do this, the replication software continuously copies all the latest updates of the data from the disks on the primary server side to the disks on the standby server side.

This is much harder than it seems. A simple implementation would simply ensure that every write of data that is done on the primary is also done on the standby storage at the same time (synchronously). This is unacceptable because each write would take unacceptably long and this would slow down the primary server too much.

If you are not doing synchronous updates, you need to start worrying about write order fidelity.

Write-order fidelity and file-system consistency

If a database writes a number of pages to the disk on your primary server, and if you have software that is replicating all these writes to a disk on a stand-by server, it is very important that the writes should be done on the stand-by in the same order in which they were done at the primary servers. This section explains why this is important, and also why doing this is difficult. If you know about this stuff already (database and file-system guys) or if you just don’t care about the technical details, skip to the next section.

Imagine a bank database. Account balances are stored as records in the database, which are ultimately stored on the disk. Imagine that I transfer Rs. 50,000 from Basant’s account to Navin’s account. Suppose Basant’s account had Rs. 3,00,000 before the transaction and Navin’s account had Rs. 1,00,000. So, during this transaction, the database software will end up doing two different writes to the disk:

• write #1: Update Basant’s bank balance to 2,50,000
• write #2: Update Navin’s bank balance to 1,50,000

Let us assume that Basant and Navin’s bank balances are stored on different locations on the disk (i.e. on different pages). This means that the above will be two different writes. If there is a power failure, after write #1, but before write #2, then the bank will have reduced Basant’s balance without increasing Navin’s balance. This is unacceptable. When the database server restarts when power is restored, it will have lost Rs. 50,000.

After write #1, the database (and the file-system) is said to be in an inconsistent state. After write #2, consistency is restored.

It is always possible that at the time of a power failure, a database might be inconsistent. This cannot be prevented, but it can be cured. For this, databases typically do something called write-ahead-logging. In this, the database first writes a “log entry” indicating what updates it is going to do as part of the current transaction. And only after the log entry is written does it do the actual updates. Now the sequence of updates is this:

• write #0: Write this log entry “Update Basant’s balance to Rs. 2,50,000; update Navin’s balance to Rs. 1,50,000″ to the logging section of the disk
• write #1: Update Basant’s bank balance to 2,50,000
• write #2: Update Navin’s bank balance to 1,50,000

Now if the power failure occurs between writes #0 and #1 or between #1 and #2, then the database has enough information to fix things later. When it restarts, before the database becomes active, it first reads the logging section of the disk and goes and checks whether all the updates that where claimed in the logs have actually happened. In this case, after reading the log entry, it needs to check whether Basant’s balance is actually 2,50,000 and Navin’s balance is actually 1,50,000. If they are not, the database is inconsisstent, but it has enough information to restore consistency. The recovery procedure consists of simply going ahead and making those updates. After these updates, the database can continue with regular operations.

(Note: This is a huge simplification of what really happens, and has some inaccuracies – the intention here is to give you a feel for what is going on, not a course lecture on database theory. Database people, please don’t write to me about the errors in the above – I already know; I have a Ph.D. in this area.)

Note that in the above scheme the order in which writes happen is very important. Specifically, write #0 must happen before #1 and #2. If for some reason write #1 happens before write #0 we can lose money again. Just imagine a power failure after write #1 but before write #0. On the other hand, it doesn’t really matter whether write #1 happens before write #2 or the other way around. The mathematically inclined will notice that this is a partial order.

Now if there is replication software that is replicating all the writes from the primary to the secondary, it needs to ensure that the writes happen in the same order. Otherwise the database on the stand-by server will be inconsistent, and can result in problems if suddenly the stand-by needs to take over as the main database. (Strictly speaking, we just need to ensure that the partial order is respected. So we can do the writes in this order: #0, #2, #1 and things will be fine. But #2, #0, #1 could lead to an inconsistent database.)

Replication software that ensures this is said to maintain write order fidelity. A large enterprise that runs mission critical databases (and other similar software) will not accept any replication software that does not maintain write order fidelity.

Why is write-order fidelity difficult?

I can here you muttering, “Ok, fine! Do the writes in the same order. Got it. What’s the big deal?” Turns out that maintaining write-order fidelity is easier said than done. Imagine the your database server has multiple CPUs. The different writes are being done by different CPUs. And the different CPUs have different clocks, so that the timestamps used by them are not nessarily in sync. Multiple CPUs is now the default in server class machines. Further imagine that the “logging section” of the database is actually stored on a different disk. For reasons beyond the scope of this article, this is the recommended practice. So, the situation is that different CPUs are writing to different disks, and the poor replication software has to figure out what order this was done in. It gets even worse when you realize that the disks are not simple disks, but complex disk arrays that have a whole lot of intelligence of their own (and hence might not write in the order you specified), and that there is a volume manager layer on the disk (which can be doing striping and RAID and other fancy tricks) and a file-system layer on top of the volume manager layer that is doing buffering of the writes, and you begin to get an idea of why this is not easy.

Naive solutions to this problem, like using locks to serialize the writes, result in unacceptable degradation of performance.

Druvaa Replicator has patent-pending technology in this area, where they are able to automatically figure out the partial order of the writes made at the primary, without significantly increasing the overheads. In this article, I’ve just focused on one aspect of Druvaa Replicator, just to give an idea of why this is so difficult to build. To get a more complete picture of the technology in it, see this white paper.

Druvaa inSync

Druvaa inSync is a solution that allows desktops/laptops in an enterprise to be backed up to a central server. (The central server is also in the enterprise; imagine the central server being in the head office, and the desktops/laptops spread out over a number of satellite offices across the country.) The key features of inSync are:

• The amount of data being sent from the laptop to the backup server is greatly reduced (often by over 90%) compared to standard backup solutions. This results in much faster backups and lower consumption of expensive WAN bandwidth.
• It stores all copies of the data, and hence allows timeline based recovery. You can recover any version of any document as it existed at any point of time in the past. Imagine you plugged in your friend’s USB drive at 2:30pm, and that resulted in a virus that totally screwed up your system. Simply uses inSync to restore your system to the state that existed at 2:29pm and you are done. This is possible because Druvaa backs up your data continuously and automatically. This is far better than having to restore from last night’s backup and losing all data from this morning.
• It intelligently senses the kind of network connection that exists between the laptop and the backup server, and will correspondingly throttle its own usage of the network (possibly based on customer policies) to ensure that it does not interfere with the customer’s YouTube video browsing habits.

Data de-duplication

Overview of Druvaa inSync. 1. Fingerprints computed on laptop sent to backup server. 2. Backup server responds with information about which parts are non-duplicate. 3. Non-duplicate parts compressed, encrypted and sent.

Let’s dig a little deeper into the claim of 90% reduction of data transfer. The basic technology behind this is called data de-duplication. Imagine an enterprise with 10 employees. All their laptops have been backed up to a single central server. At this point, data de-duplication software can realize that there is a lot of data that has been duplicated across the different backups. i.e. the 10 different backups of contain a lot of files that are common. Most of the files in the C:\WINDOWS directory. All those large powerpoint documents that got mail-forwarded around the office. In such cases, the de-duplication software can save diskspace by keeping just one copy of the file and deleting all the other copies. In place of the deleted copies, it can store a shortcut indicating that if this user tries to restore this file, it should be fetched from the other backup and then restored.

Data de-duplication doesn’t have to be at the level of whole files. Imagine a long and complex document you created and sent to your boss. Your boss simply changed the first three lines and saved it into a document with a different name. These files have different names, and different contents, but most of the data (other than the first few lines) is the same. De-duplication software can detect such copies of the data too, and are smart enough to store only one copy of this document in the first backup, and just the differences in the second backup.

The way to detect duplicates is through a mechanism called document fingerprinting. Each document is broken up into smaller chunks. (How do determine what constitutes one chunk is an advanced topic beyond the scope of this article.) Now, a short “fingerprint” is created for each chunk. A fingerprint is a short string (e.g. 16 bytes) that is uniquely determined by the contents of the entire chunk. The computation of a fingerprint is done in such a way that if even a single byte of the chunk is changed, the fingerprint changes. (It’s something like a checksum, but a little more complicated to ensure that two different chunks cannot accidently have the same checksum.)

All the fingerprints of all the chunks are then stored in a database. Now everytime a new document is encountered, it is broken up into chunks, fingerprints computed and these fingerprints are looked up in the database of fingerprints. If a fingerprint is found in the database, then we know that this particular chunk already exists somewhere in one of the backups, and the database will tell us the location of the chunk. Now this chunk in the new file can be replaced by a shortcut to the old chunk. Rinse. Repeat. And we get 90% savings of disk space. The interested reader is encouraged to google Rabin fingerprinting, shingling, Rsync for hours of fascinating algorithms in this area. Before you know it, you’ll be trying to figure out how to use these techniques to find who is plagiarising your blog content on the internet.

Back to Druvaa inSync. inSync does fingerprinting at the laptop itself, before the data is sent to the central server. So, it is able to detect duplicate content before it gets sent over the slow and expensive net connection and consumes time and bandwidth. This is in contrast to most other systems that do de-duplication as a post-processing step at the server. At a Fortune 500 customer site, inSync was able reduce the backup time from 30 minutes to 4 minutes, and the disk space required on the server went down from 7TB to 680GB. (source.)

Again, this was just one example used to give an idea of the complexities involved in building inSync. For more information on other distinguishinging features, check out the inSync product overview page.

Have questions about the technology, or about Druvaa in general? Ask them in the comments section below (or email me). I’m sure Milind/Jaspreet will be happy to answer them.

Also, this long, tech-heavy article was an experiment. Did you like it? Was it too long? Too technical? Do you want more articles like this, or less? Please let me know.

Related articles:

• TechCrunchIT coverage of Druvaa
• Video of Druvaa’s product pitch
• Understanding RPO and RTO in backups
• PC Backups – Six must have features – from the Druvaa Blog
• The fast now gets faster – Druvaa inSync 2.1 releases – from the Druvaa Blog

Click here if you cannot see the video above. From Druvaa’s Blog. See also other other proto.in videos from YouTube

Recovery Point Objective (RPO) and Recovey Time Objective (RTO) are some of the most important parameters of a disaster recovery or data protection plan. These objectives guide the enterprises in choosing an optimal data backup (or rather restore) plan.

RPO – Recovery Point Objective (wikipedia)

“Recovery Point Objective (RPO) describes the amount of data lost – measured in time. Example: After an outage, if the last available good copy of data was from 18 hours ago, then the RPO would be 18 hours.”

In other words it is the answer to the question – “Up to what point in time can the data be recovered ?“.

RTO – Recovery Time Objectives (wikipedia)

“The Recovery Time Objective (RTO) is the duration of time and a service level within which a business process must be restored after a disaster in order to avoid unacceptable consequences associated with a break in continuity.

[...]

It should be noted that the RTO attaches to the business process and not the resources required to support the process.”

In another words it is the answer to the question – “How much time did you take to recover after notification of a business process disruption ?“

The RTO/RPO and the results of the Business Impact Analysis (BIA) in its entirety provide the basis for identifying and analyzing viable strategies for inclusion in the business continuity plan. Viable strategy options would include any which would enable resumption of a business process in a time frame at or near the RTO/RPO. This would include alternate or manual workaround procedures and would not necessarily require computer systems to meet the objectives.

There is always a gap between the actuals (RTA/RPA) and objectives introduced by various manual and automated steps to bring the business application up. These actuals can only be exposed by disaster and business disruption rehearsals.

Some Examples –

Traditional Backups

In traditional tape backups, if your backup plan takes 2 hours for a scheduled backup at 0600 hours and 1800 hours, then a primary site failure at 1400 hrs would leave you with an option of restoring from the 0600 hrs backup which means RPA of 8 hours and 2 hours RTA.

Continuous Replication

Replication provides higher RPO guarantees as the target system contains a mirrored image of the source. The RPA values depend upon how fast the changes are applied and if the replication is synchronous or asynchronous. RPO is dependent only on how soon the data on target/replicated site can be made available to the application.

About Druvaa Replicator

Druvaa Replicator is a Continuous Data Protection and Replication (CDP-R) product which near-synchronously and non-disruptively replicates changes on production sever to target site and provides point-in-time snapshots for instant data access.

The partial synchronous replication ensures that the data is written to a local or remote cache (caching server) before its application can write locally. This ensures up to 5 sec RPO guarantees . CDP technology (still beta) enables up to 1024 snapshots (beta) at that target storage which helps the admin to access current or any past point-in-time consistent image of data instantly, ensuring under 2 sec RTO.

More Information – http://www.druvaa.com/products/replicator/