From cave paintings to analytics - how data is shaping our future

We take a look at the parallels between the history of exploration and production of oil, and of data.

Comparing data and oil

‘Exploration and production’ is a concept that is used for natural resources such as oil, and now for how we process and extract value from the ever-increasing streams of data in industry and in our daily lives. Data is often referred to as the new natural resource, so it is worth a brief look at the parallels in how the exploration and production of other “natural” resources have evolved. What might we expect from this “new resource”? Will it change how we do things and impact our lives?

Iron ore, gold, oil, water, coal – these are all natural resources that humans have been discovering, processing and distributing for millennia. But we have not always known how to best work with them, or how to make the most effective use of them in a sustainable way.

Take oil as an example. For the first 6000 years or so after humans started using oil, the asphalt/bitumen that seeped to the earth’s surface was primarily used as a glue or a sealant in building and ship construction. The interesting exception of course, was in China. Some 2000 years ago the Chinese discovered the use of gas as a fuel and managed to dig wells to 800ft (240m) and transport the oil in bamboo pipelines. This innovation was subsequently lost – clearly it was well ahead of its time.

Oil as medicine

Later, oil was used as medicine in various forms in many different cultures. Fast forward to the mid-1800s and we have the first “modern” oil wells in Baku (1840s) and in Pennsylvania (1859, Edwin Drake). The latter introduced the mass exploration and production of oil, thus sustaining and making economical use of oil-derived kerosene (paraffin), which had superior properties compared to its coal based predecessor.

The late 1800s saw the introduction of the internal combustion engine. The demand for ever more refined product soared. This story, focusing on oil, shows clearly how we steadily discover more about what we can do with a natural resource, while also learning how to find and recover it efficiently and innovate to find new advantages.

Data exploitation

This is a story not so much about oil, as it is about data and information. Data and its exploitation has at least as long a history as oil, or other natural resources. Neanderthals, humans and animals have always used data to make decisions. You see, you feel, you hear, you smell, you taste and you make associations between them and what there is around you. You learn. You apply the learning to benefit you and your pack. The cycle goes on. 

Original storage capacity was low and somewhat dynamic in our developing brains. When you die, your memory is lost. Does this sound like anything you know? About 40,000 years ago we learned to draw and paint (capture data) by drawings on cave walls. Within the past 5200 years we had the great innovation of writing to capture data and thoughts. Image capturing (photography) came of age within the past 200 years. Increasingly, we have searched for and found ways to capture many more types of data and to find innovations to capture and store it. 

Today, we are at a point of having almost limitless data surrounding us. We have found ways to measure and record data in a way that was just not possible on a practical scale a few years ago. However, compared to the use we make of our natural resources, we are in our infancy as to how we make effective use of data. Similarly to how we used oil as a medicine, we have at times made inappropriate use of data, and have certainly not discovered its full potential.

Technology advancements in oil and data

In many ways, we can be thought of as having, until recently, doing little more than the ancients did 6000 years ago using oil product from seepages as a mortar to build houses or seal their boats. As per Edwin Drake’s “first” truly drilled well through rock and the establishing of the ability to systematically explore, discover and recover oil, we now have the know-how and technology to generate or discover and to recover data in systematic ways and in volume. This has been enabled by the technological advancement in sensors, their cost effectiveness and in the ability to recover data from them in an operational environment. As with oil reserves, data too will continue to be recovered from ever more challenging and complex environments. Arguably, the discovery, recovery and utilization of data is now somewhere around the same level of advancement as the oil industry was in the late 1800s. 

Of course, for many decades, in some cases centuries, we have used data effectively – or at least as effectively as the technology to handle it has allowed. The scientific, medical, social and engineering communities have used data to verify hypotheses, validate physical assumptions or formulate mathematical and physics theories. We have tried to get away with as little data as possible, partly because we couldn’t handle it. We have become experts at making a lot out of a little raw material. 

Much of what we have advanced and become very good at is in making very good approximations from sparse and often unreliable data. We have been limited in the types of data that we could fully harness, for example images or pictorial detail of any kind, where humans have been very good – but machines less so. Large scale processing has been a no-go. Also, high volume or rapid streams of data have challenged man and machine. Often we have been challenged by the capture and storage technology or the analytical capability to extract relevant content. We have just had to make do with substandard data and further compromise it with limited tools to make sense of it. This paradigm is rapidly shifting.

Unimagined volumes of data

Moving from a world of approximation from poor quality data we are now faced with the challenge of extracting or refining meaning from previously unimagined volumes of data. We have had the rudiments of the analytical technology for many years, the multivariate statistical methods, the early generations of machine learning and neural networks but lacked the computing power to exploit and advance them. In addition, it was difficult, especially in engineering, scientific or medical environments, to capture the data. All of this has changed. 

Social media floods us with social and behavioral data, sensors make capture possible in many environments and wireless network technology makes it feasible to recover and store it. This in turn advances the technology to analyse, store and extract meaning from data. In many ways, the technology to handle the data is in advance of the data that is available to it. Like the development of the internal combustion engine, which first ran on crude and inefficient fuels, we now have superior analytics that can do more from richer and more complex data forms. 

The challenge today is to ensure we have the tools needed to make full use of this exceptional resource for improving business performance, whether measured in terms of safety, environmental impact or on the bottom line. Data is the currency of transaction and the lifeblood of the industry of the future. That data is the fuel of predictive analytics that deliver vastly improved actionable insights and to a timeline that makes a difference. Data and processed data in the form of information and insight are a valuable commodity, increasingly shared and traded across whole industries to add value beyond what is possible from one company’s individual resources.

Disruptive

The data industry may have lagged the other more traditional natural resources in its maturity but we are now seeing a breath-taking change of pace in its development. This is manifesting itself as a major disruptive change across many industries. Those embracing it are advancing and improving competitiveness while those who are slow to adapt are falling behind.

DNV is on the forefront of helping its customers and the industries it supports meet these challenges by combining its industry expertise, engineering insight and best practice along with the data analytics tools that are now at the disposal of industry.

Author: Tom Gilmour, Head of Product Strategy, DNV - Digital Solutions

6/1/2017 3:44:19 PM