Improving convolutional neural network accuracy using Gabor filter
Recently, convolutional neural networks (CNNs) have been used as a powerful tool to solve many problems of machine learning and computer vision. In this article, we aim to provide insight on how using a Gabor filter to improve the performance of many CNN architectures. Specifically, existing CNN models (ResNet, AlexNet, VGG16, InceptionV3).
1-Introduction
In recent years, convolutional neural networks (CNNs) have achieved great success in many problems of machine learning and computer vision. Besides, a wide range of techniques have been developed to enhance the performance or ease the training of CNNs. In 2007 Peijin Ji [1] et al presented a vision-based vehicle recognition approach based on Gabor filter banks. Partial Gabor filter bank and partial sampling method were introduced to reduce the dimension and computation. The partial Gabor feature-based method can reduce the dimension and redundancy of the features and therefore much fewer memory and computation was involved. Experimental results showed that a partial Gabor filter bank outperforms the global Gabor filter bank in some situations. The best recognition rate they got was 95.17% using partial Gabor features and a global sampling method on the edge vehicle image. By comparison, a team of researchers from Purdue University [2] did exploit the error resiliency of CNN applications and the usefulness of Gabor filters to propose an energy-efficient and fast training methodology for CNNs. They designed and implemented several Gabor filter based CNN configurations to obtain the best trade-off between accuracy and energy. Hence, [1, 2] show us that by using Gabor filters we eliminate a significant fraction of the power-hungry components of the back-propagation training, thereby achieving a considerable reduction in training energy. And for accuracy improvement [3] showed that CNNs with Gabor Layers show better performance on several datasets. CNN with utilizing Gabor Layer on «Dogs vs Cat» dataset significantly outperforms «classic» CCN up to 6% in accuracy score. In this article we show how using Gabor filter with progressive resizing in CNN can improve your model accuracy and also reduce training time.
2-Gabor filters
Gabor wavelets [5] were invented by Dennis Gabor using complex functions to serve as a basis for Fourier transforms in information theory applications. An important property of the wavelets is that the product of its standard deviations is minimized in both time and frequency domains. Gabor filters are widely used to model receptive fields of simple cells of the visual cortex. The Gabor wavelets (kernels or filters) are defined as follows:
where (x, y) is the pixel position in the spatial domain, ω the radial center frequency, θ the orientation of Gabor filter, and σ the standard deviation of the round Gaussian function along the x- and y-axes.
In real image processing applications, the features are usually extracted using a bank of Gabor filters, which parameters are set according to the approach proposed in [6]. Frequencies ω𝑛 and orientations 𝜃𝑚 of the Gabor filters are obtained by the following equations:
The 𝜎 is set by 𝜎 ≈ 𝜋/𝜔, which allows to define the relationship between 𝜎 and 𝜔. The 𝜓 is set by uniform distribution U(0, 𝜋). In this work, we initialized the Gabor Layer weights the same way.
The advantage of using Gabor filters appears to stand in the ability of such filters to provide some degree of invariance to intensity, translation, and orientation . For these reasons, Gabor filters are finding increasing usage in many applications, such as fingerprint image enhancement, texture classification and segmentation, image recognition, and motion tracking.
3-Progressive resizing
This approach is meant to demonstrate a workflow and technique for building neural networks known as “progressive resizing”. Progressive resizing has been used to good effect by Jeremy Howard, who used to achieve a top 10% finish in the Planet Kaggle Competition, and he uses it throughout his fast.ai course “Practical Deep Learning for Coders”.
4-Conception
In this article, we will be implementing specifically the method used in the paper [3] by replacing the first convolutional layer by Gabor layer and train the models.
after implementing Gabor CNN we did follow a specific training process to apply progressive resizing.
5-Implementation
5.1 Dataset
- The dataset we worked is for plant diseases and it contains 39 classes, The healthy plants are apart of these classes, there is also Background class, it refers to the images where there is no plant or plants which those not exist in our classes.
- The link for the data is here, a special thanks to Marko Arsenovic who provided this dataset.
5.2-Gabor filter implementation :
I this project we did work with Pytorch and fastai to build our models, the following code is an implementation of Gabor layer in pytorch:
This code is provided by iKintosh
5.3-Training process
All the models were trained to respect the following steps :
- Step 1: The last layer was trained on 39 classes. At the beginning we choose the image size to be 128 so we will be able to do progressive resizing later on.
- Step 2: we unfreeze all the layers of the model and we train it again.
- Step 3: we apply the progressive resizing so we change the size of the image size from 128 to the original size which is 265, then we unfreeze just the last two layers of the model and we train it.
- Step 4: we unfreeze the last three layers of the model and we train it.
- Last step: we unfreeze all the model and we train it.
6-Results
In this section, we present the benefits of using Gabor filters in CNNs alongside progressive resizing. the following table presents the overall results of our experiment.
PR : progressive resizing
* : not trained
Gabor CNN achieves better results most of the time after progressive resizing, We can notice that our Gabor models outperform the normal CNNs, as we can see for ResNet18 we’ve been able to reach accuracy 99.31% instead of 98.99% in normal ResNet18. And the best accuracy we’ve had was 99.55% with Gabor ResNet34, The result is very good and promising. Overall the models were improved for 1% of the normal performance which proves that the use of Gabor filter inside a CNN helps us get better results. But on the other hand even the good result on accuracy the Gabor models take more time than the normal ones for the prediction, it’s almost 4x higher than the normal CNNs, which makes it a bad choice for real-time classification problems.
7-Conclusion
In recent times, deep learning methods have outperformed traditional machine learning approaches on virtually every single metric. CNNs are one of the chief contributors to this success. To meet the ever-growing demand of solving more challenging tasks, the deep learning networks are becoming larger and larger. However, training of these large networks require high computational effort and energy requirements. In this work, The extensive experiments show that Gabor CNNs are significantly improved when using Gabor in comparison with normal CNNs.
Papers:
[1] P. Ji, L. Jin, X. Li, Vision-based Vehicle Type Classification Using Partial Gabor Filter Bank, in 2007 IEEE International Conference on Automation and Logistics, IEEE, 2007, pp. 1037–1040.
[2] S. Shakib Sarwar, P. Panda, K. Roy, Gabor Filter Assisted Energy Efficient Fast Learning Convolutional Neural Networks, in 2017 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED).
[3] A.Alekseev, A. Bobe, GaborNet: Gabor filters with learnable parameters in deep convolutional neural networks.
[4] S. BERISHA, Image classification using gabor filters and machine learning, 2009.
[5] D. Gabor. Theory of communication. part 1: The analysis of information. Journal of the Institution of Electrical Engineers-Part III: Radio and Communication Engineering, 93(26):429–441, 1946.
[6] S. Meshgini, A. Aghagolzadeh and H. Seyedarabi, “Face recognition using Gabor filter bank, kernel principal component analysis and support vector machine,” International Journal of Computer Theory and Engineering, vol. 4, pp. 767–771, 2012.
[7] S. Luan, B. Zhang, S. Zhou , C. Chen, J. Han, W. Yang, J. Liu Gabor Convolutional Networks, in 2018 IEEE Transactions on Image Processing
Webography:
- Gabor Layer Implementation :https://github.com/iKintosh/GaborNet
- https://github.com/ResidentMario/progressive-resizing
- fast.ai
