Tap into these dark data use cases for IT ops and the business

Dark data -- inside and outside the enterprise

Dark data, despite its name, is nothing nefarious. It can be data from the cloud, IT operations tool sets and the business that IT admins overlook as a way to glean actionable insights for service improvement. Any file type, such as emails, documents, call center recordings, videos, IoT sensor data, server log files, group chat logs and zip files, that a back-end system generates can fall under the definition of dark data.

Dark data sources also exist outside the enterprise. They include data that Google and other search engines don't typically index, such as government studies and surveys, or data from user groups in a particular industry. Data from the dark web -- webpages that are virtually untraceable because Google doesn't index them -- is also considered dark data.

The bulk of dark data is unstructured -- so organizations require advanced analytics capabilities to interpret it and derive business value. Most often, organizations hold onto dark data for compliance purposes only.

Common dark data use cases

However, IT ops teams -- alongside security admins and the business as a whole -- can benefit from shining a light on these dark data sources that exist outside traditional operations silos.

For instance, recognizing and managing dark data strengthens the organization's security posture automatically, as it ensures IT teams protect this class of data against breaches and theft. Untapped dark data can become a vulnerability or compliance issue when it lacks necessary data security and governance controls. Ignored dark data might contain information that a hacker finds interesting, and its theft places an organization at risk.

Start small with a dark data pilot project.

Another dark data use case for IT ops is to optimize storage systems in a hybrid cloud deployment, which span on-site data warehouses and cloud-based services. Data dark can quickly drive up storage costs in these hybrid environments -- but when IT teams tap into that data, they maximize their ROI. To minimize storage costs, teams can also derive value from data before it goes dark.

Dark data outside an organization can also be a wealth of untapped market intelligence data. If your organization is in a competitive market, such as consumer sales, dark data delivers value in real time, which enables management and other stakeholders to make timely decisions that benefit your organization.

Work with dark data

Establish processes to analyze dark data in a timely, cost-effective way. Otherwise, extensive analysis of vast amounts of unstructured or semi-structured data is expensive in terms of money, time and skilled staff.

The first step to harness dark data is to determine the necessary data sources to improve business decisions. Make the most of every data point that cloud services and other back-end systems generate. However, also consider additional, potentially external, data sources that will help answer business questions.

Be proactive and gain support from the organization's stakeholders, who benefit greatly from dark data. Then, move and lock down the dark data sources the IT team plans to use -- any other data should be subject to deletion.

Audit all data sources to maintain a focus on dark data. This audit must include both collected and uncollected data; consider why the IT team has traditionally analyzed certain data sets, but not others. Take stock of data analytics tools and explore options to integrate or consolidate them. A tools assessment is also an excellent time to evaluate where analytics vendors stand in terms of dark data use cases and support. The rapid evolution of analytics tools means that dark data has moved up the priorities list for many vendors -- and their customers.

Start small with a dark data pilot project. A fast data approach -- or the application of analytics to smaller data sets in the organization's application pipeline -- enables IT teams to extract real-time, actionable information. For example, seek out the best dark data sources that can help analyze a specific organizational bottleneck. Another option is to tackle one unanalyzed data source at a time.

Document all lessons learned. Test and use key performance indicators to assess results. Be prepared to iterate on the approach with testing and reassessment. As you learn more, gradually scale up dark data analytics efforts.