June 1, 2017
Systematic compliance program evaluations are an important part in the life cycle of any compliance program. Not only do they improve program effectiveness and efficiency, but if problems arise, they also show government enforcement agencies that you are serious about compliance. In fact, the US Department of Justice has recently emphasized the importance of information gathering and analysis to show that a compliance program is effective.
The basic performance evaluation process involves several steps. Of course, you need to know your business and understand the systems through which transactions occur (including those related to compliance). But appropriately evaluating the health of your program involves going a step further and defining exactly what you want to measure by setting up key performance indicators (KPIs) and metrics for monitoring performance. Creating KPIs allows you to gather appropriate data for analysis, which in turn provides the best insight on how to enhance your compliance program to better address your company’s unique characteristics, risks, and objectives.
UNDERSTAND YOUR BUSINESS
Understanding your business is an important first step in creating effective KPIs. A formal self-assessment of your systems and processes is a great way to get a clear picture of the current state of affairs and should focus not only on your company’s operations, but also on your company’s compliance efforts.
For instance, if you want to assess your company’s compliance with export controls, find out (if you do not already know) what goods you manufacture; your geographic footprint (in terms of business sites, engineering and manufacturing centers, customer locations, and infrastructure); what drives your transactions (e.g., customer requests, local agents, or company marketing efforts); your company’s strategic objectives; the regulatory frameworks that apply to your export activities (both in the US and abroad); and the touchpoints where your company’s activities implicate these regulatory regimes. From the compliance perspective, you should also learn about the processes and systems your company already has in place to comply with export controls requirements, how they are working, and how they compare to industry standards regarding export controls compliance. Armed with a thorough understanding of your business, you are now (almost) ready to start defining your KPIs.
DEFINE SPECIFIC KPIS BASED ON YOUR COMPANY’S RISK PROFILE AND GOALS
The truth of the matter is, you cannot know everything, and you cannot measure everything either. To avoid “death by data,” any performance evaluation needs to have a defined scope of analysis. In other words, based on your understanding of your business goals and risk profile (through a formal self-assessment or otherwise), determine which issues are most important to monitor and articulate KPIs against those issues.
Every business is unique, which is why a self-assessment exercise is so critical to defining the issues your company should examine. In addition, every company defines “success” differently when it comes to compliance. For example, you could view success as actual adherence to your compliance program and the law, as mitigating risk, as proactively addressing business imperatives, or as some combination of these factors. Your company’s history of regulatory or compliance problems should also inform your focus, as should your knowledge of areas known to present “weak links” in your operations and compliance program.
DECIDE WHAT QUESTIONS TO ASK
Once you have decided where to focus your efforts, make a list of questions for each issue area that will help you understand how your company is performing in each area. For example, you might want to consider the following issue areas and questions (among others) when assessing the performance of your export controls compliance program:
|EXAMPLE ISSUE AREAS TO ASSESS||EXAMPLE KPI QUESTIONS|
|ABILITY OF THE COMPANY TO MANAGE EXPORT COMPLIANCE||How many export compliance professionals does the company have?
How many export authorizations does the company currently hold?
What is the quarterly growth?
Which specific functions or programs do they support?
|TIMELY EXECUTION OF EXPORT REQUESTS||In the last quarter, what percentage of export requests were reviewed and approved within 5 days of request?|
|REGULATORY COMPLIANCE PERFORMANCE||What percentage of agreement management notices last quarter were sent to DDTC on time?
What percentage of name/address change amendments were executed and submitted on time?
How many export violations were reported last year?
How many reported export violations involved repeat offenders?
|COMPLIANCE AWARENESS AND KNOWLEDGE||What percentage of the workforce is certified as an export liaison?
What percentage of the workforce is trained on ITAR/EAR?
What is the distribution of these among different business units?
|OPERATIONAL RISK||Who are the top three export reviewers in the company and what percentage of exports are they reviewing?
What is the export activity to export resources ratio of different business units?
|REGULATORY RISK||How many ITAR-controlled items did the company export to France last month?
What percentage of the company’s exports are highly controlled?
Where are exports of ITAR products going?
CREATE A SCORECARD
Once you have defined what you want to measure – and what questions to ask in these issue areas – create a scorecard for assessing performance of these measures. To structure the scorecard, start with the qualitative description of what you want to measure, translate that into one or a set of metrics, and finally, assign performance targets for those metrics.
For example, if Company A wants to assess the timeliness of its export request review and approval process, it might consider the following metrics to examine the efficiency of its export request approval process:
|KPI 1||TIMELINESS OF EXPORT REQUEST REVIEW AND APPROVAL||17Q1|
|METRIC 1||IN THE LAST QUARTER, WHAT PERCENTAGE OF EXPORT REQUESTS WERE REVIEWED AND APPROVED WITHIN 5 DAYS OF REQUEST?||91%|
|METRIC 2||IN THE LAST QUARTER, WHAT WAS THE AVERAGE TIME (IN DAYS) TO APPROVE AN EXPORT REQUEST?||4.1 DAYS|
Once you understand your company and have a list of KPIs based on your company’s unique characteristics, you can gather data for each KPI and evaluate that data to determine if you need to adjust your company’s compliance program.
For example, going to the KPI above, if you find that a low percentage of export requests are being reviewed and approved within five days, you can ask further questions to get to the heart of the matter. For example, where is the bottleneck in this approval process, and why is it occurring? Is it due to resource issues, such as a lack of appropriately trained personnel to review the requests? Or is it perhaps due to the number of high-risk reviews needed, which reflects on the nature of the company’s risk profile? The answers to these questions will help the company determine what processes and mechanisms it can implement to overcome the identified problem.
In short, knowing where you stand with compliance requires knowing your business, understanding your risks, and then engaging in a systematic analysis that relies on real data about your company’s performance in specifically identified areas. Not only does engaging in this process help you perform better where compliance is concerned, but it also creates an important record for the future.
“Evaluating an Export Compliance Program: Defining Key Performance Indicators,” Ankura Consulting, Shahid,Waqas; Cook,Randall H.; Giambalvo, Rosanne.