Why Storing Dark Data and Mining Its Secrets Benefits Businesses

With a name that, alone, suggests it’s something no enterprise should want to keep, Dark Data is often a misunderstood presence in company servers. In fact, despite its ominous name, it is actually a highly-valuable asset, and storing Dark Data and correctly mining it can provide huge benefits to businesses. Storing all data is a given these days, with emails and communiqués in particular kept as a matter of either policy or legal necessity. But the issue for most companies is what to do with all of this information.

Should it sit there until someone works out what to do with it? Or should some be disposed of? And if some is jettisoned ‘to make room’, how wise would it be without first identifying if that data has any value? This uncertainty points to a misunderstanding of what this data actually is, and the true value safely storing Dark Data actually offers.

Understanding Dark Data

Generally speaking, Dark Data is the vast range of information a company gathers as part of its everyday operations, but never subsequently uses. Gartner, the global leaders in IT research and advisory services, describes it as “the information assets organizations collect, process, and store during regular business activities, but generally fail to use for other purposes”.

Common examples of this type of data include:

• Minutes of Board and Executive Meeting
• Notes taken of Phone Calls
• Presentations, Reports, Research & White Papers
• Email conversations
• Customer and Account Information
• Service Tickets
• Customer complaints

Research by the IDC suggests that 90% of the data a company creates is never used again but is lost as unstructured data, buried deep within an often chaotic muddle of other documents and files, making individual items next to impossible to find.

Why Storing Dark Data Matters

At first glance, Dark Data might seem to be irrelevant. But in fact, because it incorporates a wealth of documents, proposals, account details, emails and a variety of other types of communiqués, the vast majority of this data can be considered extremely relevant.

For example, a proposal written to attract a prospective client might have eventually been rejected. If, 5 years later, the same project comes up for tender again, it would be a good idea to view details proposed in the original document and build on them. However, because that document was not opened to in the interim, it could easily have become Dark Data, seemingly lost forever. As a result, considerable time and effort is expended to research and write a completely new proposal from scratch.

Many experts in the field consider this data to be an asset, with even years-old documents retaining value. So, the safe storage of Dark Data is seen as a wise move. In the case of complaints and service histories, for example, data can be hugely beneficial when trying to get a picture of service problems in relation to individual accounts, a set demographic or a specific region. This information can then be used as the basis of a strategic response.

Making Use of Dark Data

So, how can a company make better use of this unseen data? Well, it comes down to careful storage and more effective curation. In essence, this means storing Dark Data safely in a way that permits easy data retrieval, and knowing how best to analyze the information to harness its potential.

Storing Dark Data

The quality of the metadata applied to files and documents is the key. Metadata illuminates Dark Data, casting a spotlight on single documents amongst millions for quicker retrieval. Using a set of carefully chosen specific properties to accurately classify critical elements, metadata can successfully direct searches to the most relevant data for the request made. Of course, poor ‘directions’ make it more difficult to get anywhere. So, getting the metadata right is important. In most systems, files are saved according to general details like date created, file size and file title, but by expanding the range of references, files can actually be identified more precisely.

Try using metadata relating to:

• Project name
• Customer names
• Workflow
• Project Status

What is more, higher quality metadata makes it much easier to locate data stored across multiple repositories in a single search, helping companies reap more intricate insight from the available data.

Analyzing Dark Data

With information like customer complaints, customer account details, and research papers often within Dark Data, there is no shortage of potential value. Arguably, the most valuable information is cultural data, not simply statistical. For example, motivating signals can offer a deeper knowledge of previous successes (or failures), essential details when formulating upcoming sales or development strategies.

Some of the key motivators stored within Dark Data include:

• Revenue Stream histories – by country, region, customer, sales rep
• Product Sales histories – by product, combinations, averages, regions
• Customer preferences correlated from customer-service interactions and social media trends
• Indicators for upselling opportunities

Clearly, storing Dark Data offer hugely valuable returns for a company, though only if dedicated analysis is also provided.

Visit our solutions and find out how Kefron can help manage your information.