Automated Apple Fruit Recognition System based on Conventional Neural Network

Abstract:

Artificial Intelligence is playing a vital role in all the research areas. Fruit recognition systems with deep learning methods are the popular research work in computer vision. The deep learning algorithms are powerful and efficiently working in challenging areas. Image processing techniques and feature extraction process are the essential activities in this fruit recognition system. The existing methods in the fruit recognition system are not producing effective results. To get improved results in fruit recognition, the conventional neural network method is adopted. This method is used for dimensionality reduction from the large dataset. The image transformation is useful to transform the image in a structure that is more flexible to make the computation. The recognition accuracy is calculated in the training phase and testing phase.

Keywords: Fruit recognition, Conventional Neural Network, MATLAB

Introduction

The objective of the coursework

Computer vision technology is one of the powerful techniques which are mainly used for the recognition process. The fruit recognition with deep learning methods can get higher accuracy than the existing methods. The deep learning methods consist of several algorithms. In this, the Conventional Neural network is the most popular deep learning algorithm which is widely used. The fruit image recognition is based on the shape and colour features of the fruits in the digital images. The conventional neural network gives an effective performance in the image detection and recognition process. This conventional neural network is not fully connected architecture instead of connecting the local receptive field. This method provides a strong mechanism to recognize the objects such as faces, leaves and fruits etc.

The conventional neural network contains various layers which are mainly involved in pattern recognition. Filtering is the main objective of the conventional layers. The training phase considers the proper recognition of images to the conventional neural networks. After completing the process of the conventional layer, the pooling layer is activated. This pooling layer is used to segment the images into a set of non-overlapping rectangles. Each part of this partition is known as a sub-region. The main aim of this process is to find the features of the selected image. There are two main processes involved in pooling layers such as max-pooling and average pooling. The max-pooling concentrates the maximum output value from the sub-region.  The average pooling concentrates the average value from the sub-region. This pooling layer is used to reduce spatial dimensionality. This spatial reduction can improve the computation performance in fruit recognition. This also supports providing fewer parameters in the training phase to avoid the process of over-fitting. The activation layer is the next layer of the pooling layer. The ReLu is the function that is used for the activation process.

Literature Review

Overview of the coursework

The deep learning methods are used for the fruit recognition process effectively. The Fruit-360 is the dataset that is used for this recognition(Horea Muresan. 2018). This contains 82213 images of fruits with 120 different types. The preprocessing methods were implemented to reduce the noise in the dataset. The deep learning method was preferred for the fruit recognition from the huge amount of dataset. The conventional neural network was used for the recognition of fruits from the digital images. This conventional neural network consists of different layers to give improve prediction results. Those are conventional layers, pooling layer, activation layer, fully connected layer and the loss layer. After preprocessing, the noisy images were deleted from the dataset and the dataset consist of 120 classification labels. The TensorFlow tool is used for the implementation of this fruit recognition. The training model and testing models were used to evaluate the performance of the conventional neural network in the process of fruit recognition. The accuracy of the testing model and the training model was calculated in the validation phase.

A deep conventional neural network is used to make the efficient recognition of fruits(Shadman Sakib. 2018).  They selected a dataset that contained 17,823 images with 25 different categories. The database is divided into two main datasets for training and testing. The implementation model consists of two conventional layers and the pooling layer with two fully connected layers. The input layer for this process contained 30,000 neurons to consist of input values. The pooling layer reduced the size of the image which helps to make the dimensionality reduction. Then the activation layer contains Rectified Linear Units(ReLu) function. This activation method is used in the first conventional layer. The second conventional layer consists of 64 filters to make the feature extraction from the images. This implementation model provided the best accuracy of 100%. The conventional layers are used to make the iterative process reach maximum accuracy. The produced result of this work highlighted that the adaptation of conventional layers and the pooling layers can improve the prediction accuracy of the existing methods.

Commercial source tracking system is focusing the fruit recognition based on the Deep Conventional Neural Network(Israr Hussain. 2018). The database for this recognition process consists of 15 different categories with 44406 images. The deep conventional neural network is used to make the image processing techniques and extracting the features of the image.  Image processing methods were used to make the sequential alignment of the collected images. The feature extraction was used to focus the important features like colour, shape and texture of the images.  These images were directly used for the training and testing process. The representation learning method was used to recognize the images through training. The classification methods are used to classify the class labels after the recognition of the fruit. The dataset was divided into three main processes such as training dataset, testing dataset and validation dataset. The implementation was done with the help of MATLAB and the achieved accuracy was 99%.

Fruit and vegetable classification was done with the help of a conventional neural network. This system also adapted the Image Saliency method for fruit and vegetable recognition(Zeng 2017).  The images for the recognition were collected from the websites. The collected images were in different sizes. So the image processing methods were used to align the images and maintain all the images in the same size which was more convenient for the computation process. Image saliency was used to predict the intuitive notice of the object. This method was used to extract the features from the collected images. They used the Bottom-up graph-based visual saliency model to predict the important feature in the image. The VGG model was also used in this object recognition process. This method was used to increase the depth of the network. The prediction accuracy of this system was 95.6%. The researcher of this work gave more importance to collect the real-time images and applied the image processing techniques to form the dataset for computation.

The fruit recognition system was based on the colour, shape and texture of the images(Ruaa Adeeb Abdulmunem Al-Falluja 2016). The real-time images were used for this recognition process. The dataset contains five different fruits. The RGB colour images were selected for the computation method. The Gray Level Co-occurrence method was used to calculate the texture feature of the image. The image processing methods were used to make the dataset is ready to execute state. This method helps to arrange all the images in the same size and the rotation techniques were also used to make the proper alignment of the images. The feature extraction was the main process to take the important attributes from the images. Colour, Shape and texture were the essential features of image processing. The Deep learning methods were used to recognize the fruits based on the given training. The evaluation of the prediction process is based on the number of instances that were correctly identified. The incorrect instances were also considered to calculate the error rate in the recognition process.

A Fruit detection system was developed with the help of deep learning methods(Inkyu Sa. 2016). The Faster Region-based Conventional neural network was adapted for the implementation of the fruit recognition system. The proposed method for fruit recognition involved the transfer learning process with two different modalities such as colour and near-infrared. The implementation of the multi-model R-CNN method helped to make the improved recognition of the fruit. The Pre-trained methods were used for the recognition of fruit such as Image Net. The main advantage of this proposed model was handling the different image modalities for efficient prediction. The evaluation metrics considered the accuracy, precision and recall of the computation process. The taken dataset was divided for training and testing processes. The faster R-CNN was used to make the computation process by sharing the conventional neural features with classification networks. This consist of two main methods such as region proposal and region classifier. The higher accuracy was achieved by using this faster R-CNN method. The precision value was 0.57 and the recall value was 0.8.

Kiwi fruit detection system was developed with the adaptation of the Faster R-CNN method(L. F. Fu 2018). The deep learning methods are powerful which are used for image processing methods. The Faster R-CNN neural network is used to make the computation process with the support of the backpropagation method. The stochastic gradient descent method was implemented with the help of the Zeiler and Fergus network (ZFNet). The image acquisition method indicted the data collection for the proposed process. The collected images were arranged with the support of image processing techniques. The conversion of colour images into grayscale was done to make the recognition. The Faster R-CNN method was trained to recognize the kiwi fruit in the images. The feature extraction method was used to extract the essential features which were used to recognize the correct fruit. The overall prediction accuracy was calculated. This model produced 92.3% accuracy in the kiwi fruit recognition.

The development of a control system for the classification of fruit is developed using a Conventional neural network(Zaw Min Khaing. 2018). The computer vision process is a popular method in the research field. The object detection and recognition method were focused on by many researchers and the results are produced to solve the object recognition problems. The Fruit-360 dataset was taken for the recognition process. This dataset consist number of digital images related to fruits. The image processing techniques were used. The conventional neural network method was used to recognize the fruits based on the training. The parameter optimization was consisted of the conventional neural network to recognize the fruits in the dataset. This is also considered the layered process to get a better prediction result. The prediction accuracy was calculated and achieved 94 % of accuracy if fruit recognition process which consist of 30 different classes and 971 images. The vision subsystem based control applications are used in the detection system.

Methodology

Convolutional Layer

The convolutional layers are the major layer which is considered as a building block of the conventional neural network. This makes the filtering process which is applied to the input images. This also supports the activation function to reduce the dimensionality reduction. This convolutional neural network has the ability to learn a large number of filters from a huge dataset. This method applies the filter to an input image to create a feature map which is used to summarize the detected features of input images. This is a liner operation that supports the multiplication of a set of weights in the input of the network. A dot-product is an element-wise multiplication process that is processed between the filter-sized patch of the input and the filter. Then the sum values are produced as a single value. This operation is known as the scalar product.

Pooling Layer:

This is the other essential building block of conventional neural networks. The main objective of this method is to reduce the spatial size of the image and producing fewer parameter values which is more important to make the training process for the prediction. This pooling layer is used to handle every individual feature map to reduce the size. The pooling layer provides the downsampling process by summarizing all the features which are in the patches of the feature map. The pooling method has two different processes as average pooling and max pooling. The average pooling method is used to find the average values of the feature. The max-pooling is used to calculate the maximum values of the features.

Fully connected layer

The fully connected layer is used for the classification of images. This layer consists of the output from the convolutional or pooling layer for the computation process. This is also used to provide the best class labels for the predicted images. This layer consists of the network connection with all the nodes which will precede the next layer. This method consists of the flattening (output) from the previous layers and makes the values into a single value for further process. There are two fully connected layers are used in this Apple fruit recognition system. the first fully connected layer consists of the input values from the previous layers and applies weight for the prediction of appropriate labels. The fully connected output layer produced the final probability for every label.

Activation Function:

The activation function in the neural network is divided into two types. Those are Linear Activation and Non-Linear Activation Functions. There are several activation functions are available such as Sigmoid, tanh, Softmax, ReLu and Leaky ReLu etc. For this Apple fruit recognition process, the ReLu activation function is used. This is the piecewise linear function. This overcomes the vanishing gradient problem to provide the prediction results in a faster manner. The default activation function for this ReLu is a multilayer perceptron.  The ReLu activation function uses the backpropagation of error which is used to train the deep learning algorithms.  This ReLu activation method is the widely used method that can provide higher accuracy in the prediction.

Simulation

Workflow Model for the Apple Fruit Recognition System using Conventional Neural Network

The database was taken from the Benchmark datasets from online sites. The apple dataset consists of 6161 images that are used for the apple fruit recognition system.

Fig No: 1 System Workflow Model

Experimental Results:

The dataset for this Apple fruit recognition system consists of 6161 images.

Step-1: Data set collection

Step-2: Input layer formulation

Step-3: Hidden layer reduction using kernel

Step-4: Activation function process

Step-5: The output layer produces the result

Dataset introduction:

The fruit 360 dataset is used for this research work. This dataset contains more than 6000 images with six different class labels. This dataset is divided into three main categories. The first category contains 70 percentages of images from the dataset which is used for training. The second category is used for validation that contains 10 percentages of images from the dataset. The third category was issued for testing. This contains 20 percentages of images from the dataset.

Dataset details

Total images 6161

Table No:1 Dataset details

Execution Results:

Fig No: 2 Sample images of first-class

Fig No: 3 Sample Images of Class 2

Fig No:4Sample images of class 3

Fig No: 5 Sample images of Class 4

Fig No: 6 Sample Images of Class 5

Fig No: 7 Sample Images of Class 6

Fig No: 8 Reducing Mean Square Error

Training Models

The Training models are used to provide training to the algorithm for recognizing the apples from the huge dataset. The feature extraction method is used to make the effective recognition of Apple fruit from the digital images.

Fig No:9 Training accuracy

The dataset is divided into three main types. 70% of the dataset is used for providing training to the model for making a better prediction.

Testing Models

The testing models are used to check the computation process of the model based on the given training.

Fig No: 10Testing Accuracy

20% of the dataset is used for the testing phase. The testing evaluation is done with the correctly predicted images.

Validation Model

The validation is used to check the efficiency of the used model in the apple fruit prediction process.

Fig No: 11 Validation Analysis

For validating the performance, 10% of the dataset is used for the validation analysis. The validation method is used to find the efficiency of the used model. If the desired output is not achieved, then the tuning methods are applied to change the weight of the network and ageing repeat the computation process.

Accuracy Calculation

Accuracy prediction is an important process for every research. The main objective of this calculation is to evaluate the success rate of the used model for the identified problem. The accuracy is calculated by using the formula:

The achieved percentage ensures the success of the implemented model.

Confusion Matrix

The confusion matrix is the process for calculating the prediction accuracy. For this process, four main components are needed. Those are True Positive, True Negative, False Positive and False Negative.

Using these values the accuracy is calculated. In this Apple fruit recognition system, the accuracy was calculated in three different phases. Those are

• Training Phase Accuracy
• Validation Phase Accuracy
• Testing Phase Accuracy

Table No: 2 The Training Phase Accuracy

Fig No: 12 Confusion Matrix for testing dataset

The accuracy analysis focused on all the three phases such as the training phase, validation phase and testing phase. With this analysis, the highest accuracy is reached in the training and validation phase. The testing accuracy is also high but comparatively the training and validation accuracies are higher than the testing accuracy.

Discussion

A convolutional neural network provides a better classification of apple fruit. The overall training accuracy for this classification is 95%. The overall validation accuracy is 94%. The overall testing accuracy is  90%.

Conclusion:

The Automated Apple Fruit Recognition System is implemented with the adaptation of Deep learning algorithms. The conventional neural network method is used for this apple fruit recognition process. The conventional neural network consists of the convolutional layer, pooling layer, fully connected layer and the activation layer for the effective prediction of the images. The MATLAB tool is used to implement the computation process of this automated apple fruit recognition system. The accuracy in the training phase, validation phase and testing phase is calculated. The results highlighted that the training and validation accuracy is high when compared with the testing accuracy.

References:

Bibliography

Bargoti, S., & Underwood, J.,. ” Image segmentation for fruit detection and rield estimation in apple orchards. .” Journal of Field Robotics, 2017: Vol-34, issue-6, PP 1039-1060.

Fu, L., Feng, Y., Elkamil T, Liu, Z., Li, R., & Cui, Y. “Image recognition method of multi-cluster kiwifruit in a field based on convolutional neural networks.” Transactions of the Chinese Society of Agricultural, 2018: Vol-34, issue-2, PP 205-211.

Fu, L., Feng, Y., Majeed, Y., Zhang, X., Zhang, J., Karkee, M., & Zhang, Q. “Kiwi Fruit detection method using Faster R-CNN with ZFNet.” ScienceDirect, 2018: PP 45-50.

6. Cheng, L. Damerow, Y. Sun, and M. Blanke.,. “Early yield prediction using image analysis of apple fruit and tree canopy features with a neural network.” Journal of Imaging, 2017: vol. 3, p. 6.

Horea Muresan., Mihai Olten.,. “Fruit recognition from images using deep learning .” Acta Universitatis Sapientiae, Informatica, 2018: Vol-10, issue-1, PP 26-42.

Inkyu Sa., Zongyuan Ge., Feras Dayoub.,Ben Upcroft.,Tristan Perez.,Chris McCool.,. “DeepFruits: A Fruit Detection System Using Deep Neural Networks.” sensors, 2016: Vol-6, issue-8.

Israr Hussain., Qianhua He.,Zhuliang Chen.,. “AUTOMATIC FRUIT RECOGNITION BASED ON DCNN FOR COMMERCIAL SOURCE TRACE SYSTEM.” International Journal on Computational Science & Applications, 2018: Vol.8, No.2/3.

Ruaa Adeeb Abdulmunem Al-falluji. “Color, Shape and Texture based Fruit Recognition System.” International Journal of Advanced Research in Computer Engineering & Technology, 2016: Volume 5, Issue 7.

Shadman Sakib., Zahidun Ashraf.,Md. Abu Bakr Sidique.,. “Implementation of Fruits Recognition Classifier using Convolutional Neural Network Algorithm for Observation of Accuracies for Various Hidden Layers.” ArXiv e-Journal, 2018.

Zaw Min Khaing., Ye Naung., Phyo Hylam Htut.,. “Development of control system for fruit classification based on convolutional neural network.” 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), 2018.

Zeng, Guoxiang. “Fruit and Vegetables Classification System Using Image Saliency and Convolutional Neural Network.” IEEE 3rd Information Technology and Mechatronics Engineering Conference, 2017: PP 613-617.