One of the critical elements found in the Evaluation of Corporate Compliance Programs (Evaluation) is the need to use the information you obtain, whether through risk assessment, root cause analysis, investigation, hotline report or any other manner to remediate the situation which allowed it to arise. In an interview with Matt Kelly on the Radical Compliance podcast, former Department of Justice (DOJ) Compliance Counsel Hui Chen has said about the Evaluation, “We wanted people to see that we put a lot of emphasis on evidence and data. Don’t just tell us that you have a hotline. Show us how you know it’s working and how you’re using the information that you gain from these hotlines. When you say you have a great compliance portal, don’t just show us screenshots of it. Show us the hit rates and how you use that data to help you refine how you communicate with your audience.”

The same was true for the requirement of strong leadership by senior management and tone from the top. Chen related, “If you tell us you have a strong, talented top, show us what concrete actions your leaders have taken personally to demonstrate that. It’s not just some words that they say” but show the evidence. (Here please note the three most important things in compliance still matter: Document, Document, and Document).

Chen emphasized the Evaluation is not simply to be used or even considered as a checklist. It is designed to have Chief Compliance Officers (CCOs) and compliance professionals think about their compliance programs by asking questions. She explained, “Questions invite people to think. I like to call them evaluation questions. My goal is really to get people to really think about what they’re doing, what is the goal they’re trying to accomplish, how are they going to measure the results, how do they know it’s working. I’m a big fan of asking questions. The result of that, I’m hoping is that people really get to think about what they’re doing and why they’re doing it and how do they know that they’re successful at it.”

The Evaluation stated, under Prong 9 Continuous Improvement, Periodic Testing and Review, the following:

Evolving Updates How often has the company updated its risk assessments and reviewed its compliance policies, procedures, and practices? What steps has the company taken to determine whether policies/procedures/practices make sense for particular business segments/subsidiaries?

One of the questions for the compliance practitioner is how to put into practice these requirements laid out in the Evaluation and expounded on by Chen in her remarks about it. It was detailed in a chapter in an eBook, entitled “Planning for Big Data – A CIO’s Handbook to the Changing Data Landscape, by the O’Reilly Radar Team. The chapter was authored by Alistair Croll, entitled “The Feedback Economy. Croll believes that big data will allow innovation through the “feedback economy”. This is a step beyond the information economy because you are using the information that you have generated and collected as a source of information to guide you going forward. Information itself is not the greatest advantage but using it to make your business more agile, efficient and profitable is the greatest advantage.

Croll draws on military theory to illustrate his concept of a feedback loop. It is the OODA loop, which stands for observe, orient, decide and act. This comes from military strategist John Boyd who realized that combat “consisted of observing your circumstances, orienting yourself to your enemy’s way of thinking and your environment, deciding on a course of action and then acting on it.” Croll believes that the success of OODA is in large part “the fact it’s a loop” so that the results of “earlier actions feedback into later, hopefully wiser, ones.” This should allow combatants to “get inside their opponent’s loop, outsmarting and outmaneuvering them” because the system itself learns. For the CCO, this means that if your company can collect and analyze information better, you can act on that information faster.

Croll believes one of the greatest impediments to using this OODA feedback loop is the surplus of noise in the data; “We need to capture and analyze it well, separating the digital wheat from the digital chaff, identifying meaningful undercurrents while ignoring meaningless flotsam. To do this we need to move to more robust system to put the data into a more usable format.” Croll moves through each of the steps in how a company collects, analyzes and acts on data.

The first step is data collection where the challenge is both the sheer amount of data coming in and its size. Once the data comes in it must be ingested and cleaned. If it comes into your organization in an unstructured format, you will need to cut it up and put into the correct database format for use. Croll touches on the storage component of where you place the data, whether in servers or on the cloud.

A key insight from Croll is the issue of platforms, which are the frameworks used to crunch large amounts of data more quickly. His key insight is to break up the data “into chunks that can be analyzed in parallel” so the data can be considered and acted upon more quickly. Another technique he considers is “to build a pipeline of processing steps, each optimized for a particular task.”

Another important component is machine learning and its importance in the data supply chain. Croll observes, “we’re trying to find signal within the noise, to discern patterns. Humans can’t find signal well by themselves. Just as astronomers use algorithms to scan the night’s sky for signals, then verify any promising anomalies themselves, so too can data analysts use machines to find interesting dimensions, groupings or patterns within the data. Machines can work at a lower signal-to-noise ratio than people.”

Yet Croll correctly notes that as important as machine learning is in big data collection and analysis, there is “no substitute for human eyes and ears.” However, for many business leaders, displaying the data is most difficult because it is not generally in a readable form. It is important to portray the data in more visual style to help convey the “dozens of independent data sources” into navigable 3D environments.

Of course, having all this data is of zero use unless you act on it. Big data can be used in a wide variety of decision making, from employment evaluations around hiring and firing decisions, to strategic planning, to risk management and compliance programs. But it does take a shift in compliance thinking to use such data. It advocates “fast, iterative learning.” Big data allows you to make a quicker assessment of the impact of measured risks.

Croll ends his chapter by noting that the “big data supply chain is the organizational OODA loop.” But unlike the OODA loop, it is more than simply about the loop and plugging information as you move through it. He believes “big data is mostly about feedback”; that is, obtaining the impact of the risks you have accepted. For this to work in compliance, a company’s compliance discipline needs to both understand and “choose a course of action based upon the results, then observe what happens and use that information to collect new data or analyze things in a different way. It’s a process of continuous optimization”.

Whether you consider the OODA loop or the big data supply chain feedback, this process, coupled with the data that is available to you, should facilitate a more agile and directed business. The feedback components in both processes allow you to make adjustments literally on the fly. If that does not meet the definition of innovation, I do not know what does.

The bottom line for every CCO is that your compliance is dynamic not static. You must continually review, refine and update your compliance program based upon new information made available to you. The feedback components in both processes allow you to make adjustments literally on the fly. If that does not meet the definition of innovation, I do not know what does. 

Three Key Takeaways

  1. Innovation can come through a new way to think about and use data going forward.
  2. The OODA loop stands for observe, orient, decide and act.
  3. Always remember with machine learning and analysis, there is no substitute for human eyes and ears.

This month’s podcast sponsor is Convercent. Convercent provides your teams with a centralized platform and automated processes that connect your business goals with your ethics and values. The result? A highly strategic program that drives ethics and values to the center of your business. For more information go to