20 October 2022
Security experts say artificial intelligence (AI) systems used by businesses can make serious, costly mistakes. But one way to avoid such mistakes is for companies to employ humans to closely watch the AI.
One example of AI problems that can affect businesses happened early in the COVID-19 pandemic. It involved the credit scoring company Fair Isaac Corporation, which is known as FICO.
FICO is used by about two-thirds of the world's largest banks to help make lending decisions. The company's systems are also used to identify possible cases of credit fraud.
FILE - An employee works inside a server room at a company in Bangkok, Thailand, November 22, 2016. (REUTERS/Athit Perawongmetha)
FICO officials recently told Reuters news agency that one of the company's AI systems misidentified a large number of credit card fraud cases. At the time, the pandemic had caused a large increase in online shopping. The AI tool considered the rise in online shopping to be the result of fraudulent activity.
As a result, the AI system told banks to deny millions of purchase attempts from online buyers. The incident happened just as people were hurrying to buy products that were in short supply in stores.
But FICO told Reuters that in the end, very few buyers had their purchase requests denied. This is because a group of experts the company employs to observe, or monitor, its AI systems recognized the false fraud identifications. The workers made temporary adjustments to avoid an AI-ordered block on spending.
FICO says the expert team is quickly informed about any unusual buying activity that the AI systems might misidentify.
But these kinds of corporate teams are not that common, Reuters reports. Last year, FICO and the business advisory company McKinsey & Company carried out separate studies on the subject. They found that most organizations involved in the study were not closely watching their AI-based programs.
Experts say AI systems mainly make mistakes when real-world situations differ from the situations used in creating the intelligence. In FICO's case, it said its software expected more in-person than online shopping. This led the system to identify a greater share of financial activity as problematic.
Seasonal differences, data-quality changes or extremely unusual events – such as the pandemic – can lead to a series of bad AI predictions.
Aleksander Madry is the director of the Center for Deployable Machine Learning at the Massachusetts Institute of Technology. He told Reuters the pandemic must have been a "wake-up call" for businesses not closely monitoring their AI systems. This is because AI mistakes can cause huge problems for businesses that do not effectively manage the systems.
"That's what really stops us currently from this dream of AI revolutionizing everything," Madry said.
Urgency has been added to the issue since the European Union plans to pass a new AI law as soon as next year. The law requires companies to do some observation of their AI systems. Earlier this month, the U.S. administration also proposed new guidelines aimed at protecting citizens from the harmful effects of AI. In the guidelines, U.S. officials called for observers to ensure AI system "performance does not fall below an acceptable level over time."
I'm Bryan Lynn.
Reuters reported this story. Bryan Lynn adapted the report for VOA Learning English.
____________________________________________________________________
Words in This Story
artificial intelligence – n. the development of computer systems with the ability to perform work that normally requires human intelligence
lend – v. to give something to someone for a period of time with the expectation that they will then give it back later
fraud – n. the crime of using dishonest methods to take something valuable from another person; trickery; deceit
adjustment – n. a slight change made to something so that it works better
confuse – v. to make someone unable to think clearly or understand something
manage – v. to do something or deal with something successfully