Some Fundamentals – Process, Data and Models

Some Fundamentals – Process, Data and Models

In the last post we considered shortcomings of existing analytics programs – limitations and gaps in culture and organizations. We discussed building a successful analytics program by justifying and selling the concept throughout the organization. Then avoiding gaps by redesigning or designing the program for success. In this post we move into the basics that are shared across different data-driven techniques before we launch into upcoming psots where we cover each topic independently and more in-depth.

The Hip, the Hype, the Fears, the Intrigue and the Reality

The Hype

"We don't consider customers cargo," said Jaguar's head of R&D, Wolfgang Epple, in 2015. "We don't want to build a robot that delivers the cargo from A to B."

Reality

(According to Thompson Reuters) Jaguar Land Rover has since invested $25 million in Lyft to cash in on the autonomous trend.

Introduction

What motivated us to write this? We have visited with people across many industries and organizations and discovered reoccurring themes.  In the last post, we covered a reoccurring theme – what keeps analytics programs from being as successful as they could be? A different environment, culture, structure makes all the difference in the world toward success or failure, and we provided some guidelines.

This series covers a very broad range of topics, from business intelligence (BI) to artificial intelligence (AI). Some topics are more complex, some more intuitive, some have long histories, some are more novel. So, an obvious question is, what do they have in common? Why all these topics? The singular underlying element in the series is using data to solve real world problems. Other re-occurring themes are data, signal vs noise, models and algorithms. 

This post will cover commonalities of practice and application amongst the various methods covered in the series. What do data science and mathematical programming have in common? What do Statistics and AI have in common? Is there a common process methodology that an organization can use whether the problem they are working on is a decision science problem or a statistical design? Is there a common framework? Let us call these various groups of practitioners, ‘camps’. These different camps speak different languages, yet their conversations are often about the same thing. We will not translate everything, but we will try to synthesize some common practices in this post. 

One of the most important themes of this series is the need for lots of tools in the toolbox. Why do we emphasize a list of tools when there are some very powerful ones that solve a lot of problems, take for example, AI.  Because, there is no universally optimal methodology or analytics paradigm that works best on all problems. We will explore this in this post; There is No Free Lunch. 

Framework for Analytics - Some Fundamentals

We now explore some fundamentals that are often overlooked when applying or consuming analytics. These are fundamental and important because many people assume them away and when there are issues or shortcomings, they look for in the wrong direction to solve the problems. A few fundamentals here:

1)    Processes and systems are everywhere

2)    Data are generated by a process

3)    We collect some of this data

4)    We analyze data and create models

5)    Analytics and models provide insight

6)    Insights inform action

7)    Action improves life

Processes Are Everywhere

We often do not think in these terms, but you participate in and observe thousands of processes and systems in a day. You get out of bed and take a shower, that shower is a process. You dress and ready yourself for the day, which is a process. You fix breakfast, which is a process. Getting to work by walking, car, bus or train is a process or a multi-step process.

A system is a set of interrelated processes. The bus to work contains a large number of systems – an electrical system, an engine, a powertrain, GPS and many more. These systems are composed of thousands of processes. Each and every one of these processes generates an observable result. Each observable result can be recorded. Recorded results are data.

Processes Drive Data

We will soon explore models and algorithms. For algorithms to generate models, we must feed them data. We collect data at a dizzying rate these days as we explained in a previous post and we will talk deeply about all sorts of data in later posts. We collect data in a variety of ways – manual entry into smart phones, computers notepads. Computers collect most of the data without us even knowing – cameras, sensors, voice, electronic sensors, machines, devices, telemetry. At the heart of this collection are again systems and processes. We do not think about it most of the time, but when we do, we realize life consists of processes. Therefore, we have a great opportunity to improve life by understanding these processes via analytics.

 Following are some examples of processes and systems and the data they generate.

Moreover, note, there are millions of things that could be collected as data but only some are collected, although the number of processes collected as data is increasing.   In fact, think about video. The streets of most cities a few years ago were just as full with the bustle of activity as they are today. However, today, that bustle is captured on video, being recorded, generating data that can be analyzed.

As said by a famous statistician (W. Edwards Deming) a long time ago “In God we trust, all others bring data”.  We can do that more than ever today. Data has the ability if used correctly to separate anecdote and hyperbole from facts. In this series, we explore how we can use data intelligently to build analytical models. In the next article, we need to explore ‘what is a model?’.