For example, AI recognizes a horse as a frog by changing a pixel in an image, as revealed in the study of a pixel attack on a polluted deep neural network. Samuel Finlayson, a computer scientist at Harvard Medical School and a biomedical informatics scientist, also found that medical images can be modified in ways that are imperceptible to human eyes, and then artificial intelligence misdiagnoses cancer 100%.
In previous studies, there is mathematical evidence that there are stable and accurate neural networks used to solve various problems. Recently, however, researchers from Cambridge University and Oslo University found that these artificial intelligence systems may be stable and accurate only in limited circumstances. Neural network, which is stable and accurate in theory, may not accurately describe what may happen in reality.
"Theoretically, there are few restrictions on neural networks," said Matthew Colebrook, a mathematician at Cambridge University in England. However, when trying to calculate these neural networks, problems arise.
"Digital computers can only calculate certain neural networks," said Vegard Antun, a mathematician at the University of Oslo in Norway. "Sometimes it is impossible to calculate an ideal neural network."
This statement may sound a little confusing. When talking about this research, IEEE Spectrum compares cakes. "It seems that some people say that there may be a cake, but there is no secret recipe for making it."
"We will say that the problem is not the formula. On the contrary, the problem lies in the tools necessary for making cakes. " Anders Hansen, a mathematician at Cambridge University, said, "We said there might be a cake recipe, but no matter what blender you have, you may not be able to make the cake you want. Besides, when you try to make a cake with a blender in the kitchen, you will get a completely different cake. "
Based on this, continue the analogy, "You can't even judge whether the cake is incorrect before you try it, and then it's too late." Colebrook said, "However, in some cases, your blender is enough to make the cake you want, or at least it can be very similar to the cake."
These new discoveries about the limitations of neural networks are related to mathematician Kurt G? Del and computer scientist alan turing echoed previous studies on the limitations of computing. Roughly speaking, they revealed that "some mathematical statements can never be proved or refuted, and there are some basic calculation problems that computers can't solve." Antong said.
This research is based on "Difficulties in Computing Small and Accurate Neural Networks: Obstacles of Deep Learning and Artificial Neural Networks". D Smale's question 18) was published in the Proceedings of the National Academy of Sciences on March 16.
In artificial neural networks, components called "neurons" are input with data and cooperate to solve problems, such as recognizing images. Neural network repeatedly adjusts the relationship between neurons to see if the generated behavior pattern can find a better solution. As time goes on, the network will find the most suitable mode for the calculation results. Then it uses these as default values to imitate the learning process in the human brain. If a neural network has multiple neurons, it is called "depth".
In previous studies, there is mathematical evidence that there are stable and accurate neural networks used to solve various problems. However, in this new study, researchers now find that although there may be stable and accurate neural networks to solve many problems in theory, paradoxically, there may not be any algorithm that can successfully calculate them.
This new study found that no matter how much data the algorithm can access or how accurate the data is, the algorithm may not be able to calculate a stable and accurate neural network for a given problem. Hansen said that this is similar to Turing's argument that computers may not be able to solve some problems regardless of their computing power and running time.
"There are inherent limitations on the functions that computers can achieve, and these limitations will also appear in AI," Colebrook said. "This means that a neural network with good characteristics in theory may not accurately describe what may happen in reality."
These new findings do not mean that all neural networks are completely flawed, but they may be stable and accurate only in limited cases. "In some cases, a stable and accurate neural network can be calculated," Antong said. "The key issue is the part of' in some cases', and the biggest problem is to find these situations. At present, people know very little about how to do this. "
Researchers have found that the stability and accuracy of neural networks often need to be weighed. Hansen said: "The problem is that we want stability and accuracy at the same time." . "In practice, people may have to sacrifice some accuracy for safety-related key applications to ensure stability."
As part of the new research, researchers have developed their "Fast Iterative Restart Network" (FIRENET), which is expected to be realized when tasks such as analyzing medical images are involved. Neural network can provide stable and accurate results.
Researchers believe that these new discoveries about the limitations of neural networks are not intended to inhibit artificial intelligence research. "In the long run, it is healthy for artificial intelligence research to figure out what can and can't be done. Please pay attention. Turing and g? The negative result of del has caused great changes in mathematics foundation and computer science, which has led to most of the development of modern computer science and modern logic respectively. " Colebrook said,
Specifically, in this study, the researchers believe that these new findings mean that there is a classification theory that can describe which stable neural networks with given accuracy can be calculated by algorithms. Using the cake analogy mentioned earlier, "this will be a classification theory that describes which cakes can be baked with a physically designed mixer." If we can't bake a cake, we also want to know how close it is to the type of cake we want. "Antong said.
Proofreading: Zhang