Beware the known unknowns when finance meets AI

The Collingridge dilemma sounds like the mysterious title of Sherlock Holmes. In fact, this is one of the best explanations of how difficult it is to control high-risk technology. In essence, it involves an imbalance between incomplete information and established power.

“If it’s easy to change, we can’t foresee its need. When the need for change became apparent, it became expensive, difficult, and time-consuming,” says scholar David Collingridge. Is writing in his book. NS Social control of technology.

How can you act when dealing with known and unknown things? That dilemma is faced today when regulators are trying to assess the impact of artificial intelligence on the financial industry.As two recent reports from Bank for International Settlements And that OECD Let me be clear, we are now at a critical turning point. The benefits of AI are clear, but the risks are often ambiguous in terms of increased efficiency and improved service.

In a previous treatise published while attending the Massachusetts Institute of Technology, Gary Gensler He warned that widespread adoption of deep learning AI models could increase the vulnerability of financial systems. Gensler is currently chairing the US Securities and Exchange Commission and is in a position to address previously raised concerns.

There is no shortage of views on the principles that should govern AI. At least a non-profit organization, according to Algorithm Watch 173 sets of AI principles It is open to the public all over the world. We cannot challenge the valuable intent contained in these guidelines and promise fairness, accountability and transparency. But the challenge is to translate lofty principles into everyday practice. Complexity, ubiquity, opacity in so many cases of AI use..

An automated decision-making system approves mortgages and consumer loans and assigns credit scores. The natural language processing system performs sentiment analysis of corporate financial statements and creates personalized investment advice for individual investors. Insurance companies use image recognition systems to assess the cost of car repairs.

Using AI in these cases can affect the rights and wealth of individuals and clients, but does not pose a systemic risk.Many of these concerns are covered by Future legislation including EU AI rule. These legislative initiatives use appropriate, unbiased data for organizations deploying AI systems, ensure that their output is in line with their goals, explain how they work, and if problems arise. Make wise responsibilities that help you determine accountability.

A lesser-known question is about the use of AI-powered trading systems that can destabilize financial markets. Sarah Gadd, Head of Data and AI Solutions at Credit Suisse, states that there is a risk of grazing, games, or collaborative behavior if the system is all trained with the same data and the same type of algorithm.

“These need to be monitored very carefully, or should not be used,” she says. “To turn off the power in milliseconds and place someone who can fall back, you need a proper kill switch. You can’t replace human intelligence with mechanical intelligence.”

However, some have pointed out that a flash crash occurred long before AI was used in financial markets. The question is whether AI systems make them worse. AI systems aren’t magical, they say they’re just statistical methods Ewan Kirk, Founder and Former CEO of Cantab Capital Partners, Investment funds that use trading algorithms. “AI is good at finding only incredibly subtle effects that contain vast amounts of data and are probably not systematic in nature,” he says. He adds that the reason for Kill Switch is probably not because the AI ​​program could bring down the financial system, but because of a bug.

The best way to address the Collingridge dilemma is to increase your knowledge of AI within your organization and across society and identify the power of persistent interest that can prevent the necessary changes. Several regulatory agencies have already filed proceedings, hosting AI forums, developing regulatory sandboxes for testing and validating algorithms, and implementing their own machine learning systems to monitor the market. ..

However, as former US Defense Secretary Donald Rumsfeld said, there are some unknowns and unknowns, and it is argued that the precautionary principle should be developed in some circumstances. Regulators need to be prepared to ban the use of the most exotic or improperly designed AI systems until they have a better understanding of how they work in the real world.

john.thornhill@ft.com

Beware the known unknowns when finance meets AI Source link Beware the known unknowns when finance meets AI

Source: https://www.ft.com/content/01c366db-e1b8-49ff-9952-ef40403991ee