The Immutable kBlog: thoughts on data integrity

I found an interesting article that explains that a new ruling in the US is forcing companies to preserve their metadata in an immutable way. (NOTE: metadata describes how, when and by whom a particular set of data was collected and how the data is formatted. It is essential for understanding information stored in data warehouses and has become increasingly important in, for example, XML-based Web applications).

In the court case referred in the article (Aguilar v. Immigration & Customs Enforcement Div. of U.S. Dep’t of Homeland Sec.), a U.S. District Court ruled that metadata associated with e-mails and electronic files must be preserved, maintained and produced in the course of legal discovery.

The Aguilar decision emphasizes the importance of metadata preservation in the course of e-discovery. Metadata can be used for authentication, search and analysis while also offering evidential value such as when the file was created or accessed. This ruling shows that organizations now must be ready to present metadata if requested, and it should be kept and preserved in a way that its legal admissibility is not questioned. In other words, organizations must be able to unquestionably prove that metadata is trustworthy and was preserved in an immutable way.

One solution for that would be being prepared to retain more information on WORM format, as this can help preserve the data and metadata. However, using Kinamik Secure Audit Vault is a  more efficient and cheaper alternative for preserving data with integrity (i.e. it can not be altered) than WORM disks. By using Kinamik’s solution, organizations can use any normal disk to achieve immutability of their data with a software.

Once again, this is the proof for the need of anti-tampering solutions.

The British Standards Institution (or BSI) has recently published the BSI 10008, a new standard that focuses on the evidential weight of electronic information. It establishes up a set of requirements organizations should follow in their data management procedures for ensuring… yes, you got it: the integrity of information.

The new standard’s name is quite self-explanatory: “Evidential weight and legal admissibility of electronic information. Specification”. As the BSI website states, “legal admissibility concerns whether or not a piece of evidence would be accepted by a court of law. To ensure the admissibility, information needs to be managed by a secure system throughout its lifetime (which can be for many years). Where doubt can be placed on the information, the evidential weight may well be reduced, potentially harming the legal case”. The BSI 10008 is aimed therefore to ensure that any piece of electronic information used in a Court of Law has the maximum evidential weight.

There are many interesting aspects here. First, it shows the need of clearly establishing guidelines and a common framework for how to deal with electronic data and digital evidence. And second -but no less important- it outlines how data integrity is a key aspect in information management.

We just bought a copy of the standard. We’ll read it and publish some thoughts… Stay tuned.

Pretty much everybody agrees that 2009 will be key on how the current economic crisis develops. It will certainly change many aspects of our personal and professional lives. And when trying to identify how the e-discovery market will evolve, the folks at Clearwell Systems have produced a list of ten predictions for this year. They respond mainly to greater financial and legal stress, calling for more collaboration, control and proactive readiness in the matter.

So here’s the list, via MarketWatch. Enjoy:

1. Government Investigations Increase: the economic tensions and increase in high-profile scandals will lead to a natural rise in government investigations, compliance audits and data requests.

2. Corporations Take More Control Over e-Discovery: e-discovery processes go “in-house” for having more control and reducing costs. Organizations will then see that a proper proactive approach will bring cost-reduction opportunities for organizations when an e-discovery process takes place.

3. Industry Push For Collaboration: improving collaboration efforts will reduce costs and conflicts.

4. Federal Rules of Evidence (FRE) 502 Helps Automated Reviews: the use of automated analytical tools will be on the rise, reducing costs and lowering the time and money associated with inadvertent disclosure of privileged information.

5. “Showing Your Work” Becomes Mandatory: technology must be transparent and auditable, with organizations in the need of not only showing but also proving transparency and good practice.

6. Solving Colloquial E-Discovery Is Top of Mind: new technologies such as voicemail, instant messaging, web 2.0 and others must be included in the e-discovery process. Trustworthy auditing becomes the key aspect here.

7. Global Economic Downturn Drives Global E-Discovery: e-discovery will go international and therefore more complex. E-discovery technologies will be in the need to address privacy and data protection issues, in line with international compliance requirements.

8. Information Stores Will be Mapped: in line with prediction #1, there will be an increasing need for organizations to clearly map their electronically stored information. This means the capacity of retaining, archiving, searching and producing whatever information is required.

9. Integration Happens Across the EDRM Framework: integration will be the key for e-discovery technologies this year.

10. Information Management Shows Positive ROI: proper information management is no longer related solely to good practices, but will also have a clear cost-saving effect. Being unprepared and having unmanaged data stores will bring enormous costs if an e-discovery process comes into play. The key here is having a proper forensic readiness approach.

I came across a very interesting post today.  It  seems that when Google was requested to present evidence for a case early this year,  they alleged that due to the complexity of their e-mail structure they could present it, but  that  it would be very difficult and expensive to search and find exactly what was needed. I guess it makes sense, since they are so technologically behind the curve…

Leaving any sight of sarcasm behind, this puts into focus a simple -and enormous- truth: the high costs of any e-discovery process, specifically when organizations are requested to search among their data to produce evidence. If Google claims the process is difficult and too costly, any other company should really stop for a second and think on how to tackle this issue.

The first solution that may come to mind would be to improve searching capabilities for finding exactly what is needed. This solution, though, may produce what is called  a  “false negative” if a relevant file is not found, or  a “false positive” if something that is not relevant for the case is found  - recent findings suggest that 70% of the total documents revised in an e-discovery process are false positive findings. This proves that the correct approach should be not just having a good searching tool, but -as the columnist mentions- also that  organizations should “take available technological measures to preserve documentation for legal proceedings”. Having “the ability to preserve new documents as they are created” is key to this.

This brings me to what we do at Kinamik… which is exactly that! We build a centralized, independent and Secure Audit Vault that serves as a safe for all the sensitive data -such as audit trails-, making  them tamper-proof in the process for a future proof preservation. And of course, search capabilities are also available in that audit vault. Well … maybe I should give Google a ring?