Maria Forni
Dealing with fraud risk management in the context of CNP transactions for the past six years has brought to my attention and interest the beauty of uncertainty. One of the key resources science has provided us with to deal with uncertainty are statistics and the mathematical concept of probability.
Many anti-fraud providers rely on those resources as the core functionality of their solution and yet most will face difficulties meeting the merchants expectations of results in an ever-changing ecosystem. That happens because dealing with uncertainty requires more than the use of fancy algorithms. Uncertainty requires good decision-making and pushes the boundaries of how we deal with it as humans. Here are some of the common mistakes I've made and watched others make over and over again when dealing with uncertainty while working with fraud risk management:
1. Poor response variables management and understanding for fraud
Most algorithms used in anti-fraud solutions rely on techniques that require response variables in the process of statistical modeling. In other words, in an extensive dataset of historical transactions, they need to know which ones are fraudulent and which ones are not so they can evaluate the characteristics that distinguish one from the other. The response variables in the last instance are the chargeback notifications received by the merchants which might take up to 90 days to be fully available. Some anti-fraud solution providers might work with other response variables, such as manual review of the most risky transactions in order to fulfill their anti-fraud decision chain with information in a more timely fashion. All those types of response variables have some sort of degree of uncertainty such as:
- Chargeback Notification: Does this chargeback notification come from a truly fraudulent transaction? Or does it come from a legit transaction being disputed?
- Manual Reviews: How sure am I that this is a fraudulent transaction? Was I able to speak to the victim?
Dealing with uncertainty at a response variable level may require good processes that reduce those uncertainties and pursue their accuracy at the highest rates possible.
2. Weak hypothesis elaboration and testing for fraud management
When dealing with fraud peaks, people commonly fail in elaborating on good hypotheses and consistent/creative testing to evaluate them. Good hypothesis-making usually requires deep knowledge of how your anti-fraud solution works, how ecommerce functions and how fraudsters operate so you can assess what vulnerabilities are taking place and find good evidence to support them. This is why it is so important for analytics professionals in the field to be genuinely interested in the subject.
Some curious hypotheses that are commonly neglected and happen quite often:
- "This is not fraud. The chargeback notifications are associated with the wrong (Non fraudulent) transactions"
- "This is not fraud. This is a legit consumer abusing a promotional initiative"
- "Merchant changed their processes and is sending incorrect data to anti-fraud provided (ex. Sending Home Address as the Shipping Address)
3. Super estimating patterns and underestimating phenomena when dealing with fraud:
Especially in environments that deal with fraud using analytical approaches it is a recurring lapse to focus on data patterns instead of phenomena behind them. Most of these data patterns emerge as symptoms of a given vulnerability/behavior and usually are interpreted as the root cause.
Some consequences of this behavior might be:
- Statistical Model Level: Excessive number of correlated explanatory variables; Overfitting (Statistical Model loses its ability to generalize)
- Decision Making Level: Excessive number of actions that lead to no satisfactory results in terms of fraud control. Will feel like you are bailing water out of a sinking boat
4. Data analysis bias when making fraud management decisions:
Data is vital when making decisions but it is only as good as the interpretation we give them. All data analysis is subject to bias and recognizing they do exist makes us one step ahead when dealing with uncertainty. The most frequent causes of analysis bias are:
- Visualization Distortions;
- Assuming correlations imply causation;
- Incomplete data sets;
- Evaluating too short/ too long time series;
- Lack of good understanding of parameter references - what is expected for a given measure behavior? What is not?
5. Trying to have everything covered when dealing with fraud risk uncertainty:
"What if fraudsters do this? What if fraudsters do that?" Usually this behavior is a result of being proactive and one step ahead of the game. However, people fail to evaluate the odds of all those things actually happening. Dealing with uncertainty means that everything is possible, but assuming all of those things are likely to happen is inefficient and will leave you with excessively cautious decisions. In excessive conditions of "What if" questioning, teams might be confronted with so much uncertainty that they won't even know where to start.
6. Panicking as a fraud analyst:
Once a vulnerability is spotted, fraud takes place at an exponential speed, which leads to both financial and image damage for merchants. Whenever we go through fraud instability periods, it's very easy to panic. Ironically, this type of situation is when the teams tasked with finding a solution need support and tranquility the most in order to be able to do the best analysis and diagnosis possible. It's very easy to unknowingly transfer your panic onto the team, which can only result in poor reactions and decision-making in those high-stress moments.
Some common mistakes fraud management leadership unintentionally produce are:
- Priorioritizing solutions to a problem not yet diagnosed
- Not stimulating a hypothesis-making mindset
- Stimulating excessive "What if" assumptions that are not likely to happen
- Stimulating an excessive set of analyses that lead to no conclusions
- Stimulating excessive plans of action and inefficiency (a good diagnosis leads to 2 or 3 outputs);
- Not trusting your technical teams
Uncertainty might be frightening because it exposes us to the risk of failure and it does not get any better with time. That means you will make mistakes, it doesn't matter how long you've been in this business. On the other hand, uncertainty is so beautiful because it implies there is always something you don´t know and as long as that is so, you will be continually open to learning.
