AMEX spent a sizeable amount of resources at providing Big Data training to its employees. The result was phenomenal. 

American Express is one of the behemoths in the global banking and financial services sector. AMEX handles almost 25% of the entire credit card transactions in the United States. It holds approximately 107.2 million cards in force accounting for over $952.4 billion in global bill payments. In 2014, its volume of card transactions crossed the $1 trillion mark by a huge margin.

With business sprawling over 130 countries, AMEX is literally sitting on a large data bank of customer information like spending habits and demographics. The c-suite at AMEX was smart enough to spot the massive potential that leveraging such customer data can give to the business.

And hence, the efforts to deploy Big Data analytics at AMEX began in 2010. This was when the rest of the world was just catching up to ERP and cloud based solutions.

A decision that improved millions of everyday decisions

Within American Express, at least a million decisions and more are made every day. From sanctioning credit to detecting fraud, the volumes of customer transaction data stored in the company servers are dissected to and fro many times to minimize losses and frauds.

Often, this data analysis has to be done on a real-time basis all throughout the year. Traditional database management programs are severely handicapped to take on the volume and velocity of such financial data analysis.

AMEX’s long-serving CEO Ken Chenault did not waste much time deducing the positive impact that Big Data could deliver. After all AMEX had always remained a company which had at its core a business logic driven by data.

With its Big Data deployment AMEX was able to attain better control and competitive edge on its multiple faceted business model covering customer marketing, customer management, fraud detection, loyalty marketing, information, management, etc. 

Challenges encountered

The Big Data Journey was not an arrow straight road for AMEX. There were plenty of difficult roads, U-turns and dead ends before the fast lane to success opened up.

AMEX encountered 3 major challenges in Big Data implementation: Scarcity of Big Data scientists, organizational adaptation to change and settling upon the right infrastructure.

Scarcity of Big Data scientists

AMEX started toying with Big Data analytics as early as 2010. There were hardly any data scientists in this field which itself was in its nascent stages. And the few who existed were already taken by competitors like Visa. Providing Big Data training within the organization and ensuring that the recruits understand the business context of Big Data was not an easy task. There was also the persistent problem of employee retention from competitors.

Organizational adaptation

Forging data-centric internal culture was the next hurdle that AMEX had to clamber over. The transformation had to begin from the top. Old processes that everybody were habituated to had to be obliterated. There was an enormous challenge to make the entire organizations adaptive to change through learning and improvement.

Settling upon the right infrastructure

AMEX’s usual back-end data processing software was highly incapable of standing up to the tall order of Big Data analytics. It took sometime before the company could settle down on Hadoop Mapr as the ideal data processing infrastructure. It was when Mapr was adopted that the company was able to see visible results in spotting frauds, alerting customers of malicious transactions and in reducing customer acquisition costs.

Did the effort pay off for AMEX?

AMEX was not only able to reap returns out of its Big Data strategy but also went on to meet and exceed industry standards of benchmarking. The metrics show that AMEX achieved 45 second TeraSort and 1.65TB MinuteSort through deploying Hadoop and Strata. The flexible infrastructure is also expected to drive further business values for AMEX in core functions like customer service, targeted marketing, online customer acquisition, fraud detection, credit appraisal, etc.

More than 40% of the AMEX customer base was improved through Big Data tactics. The resources spent in Big Data training for employees was proven to be a well-thought strategy by AMEX.

The success story of AMEX in Big Data is an exemplary instance for anyone who wants to break it big in data sciences through Big Data training or Hadoop Administration certification.