AutomatedApple Fruit Recognition System based on Conventional Neural Network

Abstract:

Artificial Intelligence is playing the vital role in all the research areas. Fruit recognition system with deep learning methods are the popular research work in the computer vision. The deep learning algorithms are powerful and efficiently working in the challenging areas. Image processing techniques and feature extraction process are the essential activities in this fruit recognition system. The existing methods in the fruit recognition system are not producing the effective results. To get the improved results in the fruit recognition, the conventional neural network method is adapted. This method is used to dimensionality reduction from the large dataset. The image transformation is useful to transform the image in a structure which is more flexible to make the computation. The recognition accuracy is calculated in training phase and testing phase.

Keywords: Fruit recognition, Conventional Neural Network, MATLAB

 

Introduction

Objective of the coursework

Computer vision technique is one of the powerful techniques which are mainly used for the recognition process. The fruit recognition with deep learning methods can get the higher accuracy than the existing methods. The deep learning methods consistseveral algorithms. In this, Conventional Neural network is the most popular deep learning algorithm which is widely used. The fruit image recognition is based on the shape and color features of the fruits in the digital images. The conventional neural network gives effective performance in the image detection and recognition process. This conventional neural network is not fully connected architecture instead of connecting the local receptive field. This method provides the strong mechanism to recognize the objects such as face, leaf and fruits etc.

The conventional neural network contains various layers which are mainly involved in the pattern recognition. Filtering is the main objective of the conventional layers. The training phase considers the proper recognition of image to the conventional neutral networks. After completing the process of conventional layer, the pooling layer is activated. This pooling layer is used to segment the images into the set of non-overlapping rectangles. Each part of this partition is known as sub-region. The main aim of this process is to find the features of the selected image. There are two main process involved in pooling layer such as max-pooling and average pooling. The max-pooling concentrates the maximum output value form the sub-region.  The average pooling concentrates the average value from the sub-region. This pooling layer is used to reduce the spatial dimensionality. This spatial reduction can improve the computation performance in the fruit recognition. This also supports to provide less parameter in the training phase to avoid the process of over-fitting. Activation layer is the next layer of pooling layer. The ReLu is the function which is used for the activation process.

Literature Review

Overview of the coursework

The deep learning methods are used for the fruit recognition process effectively. The Fruit-360 is the dataset which is used for this recognition(Horea Muresan. 2018). This contains 82213 images of fruits with 120 different types. The preprocessing methods were implemented to reduce the noise in the dataset. The deep learning method was preferred for the fruit recognition from the huge amount of dataset. The conventional neural network was used for the recognition of fruits from the digital images. This conventional neural network consist different layers to give improve prediction results. Those are conventional layers, pooling layer, activation layer, fully connected layer and the loss layer. After preprocess, the noisy images were deleted from the dataset and dataset consist 120 classification labels. The TensorFlow tool is used for the implementation of this fruit recognition. The training model and testing models were used to evaluate the performance of the conventional neural network in the process of fruit recognition. The accuracy of testing model and the training model was calculate din the validation phase.

Deep conventional neural network is used to make the efficient recognition of fruits(Shadman Sakib. 2018).  They selected a dataset which contained 17,823 images with 25 different categories. The database is divided into two main dataset for training and testing. The implementation model consist two conventional layers and the pooling layer with two fully connected layers. The input layer for this process contained 30,000 neurons to consist input values. The pooling layer reduced the size of the image which helps to make the dimensionality reduction. Then the activation layer contains Rectified Linear Units(ReLu) function. This activation method is used in the first conventional layer. The second conventional layer consist 64 filters to make the feature extraction form the images. This implementation model provided the best accuracy of 100%. The conventional layers are used to make the iterative process to reach the maximum accuracy. The produced result of this work highlighted that the adaptation of conventional layers and the pooling layers can improve the prediction accuracy then the existing methods.

Commercial source tracking system is focusing the fruit recognition based on the Deep Conventional Neural Network(Israr Hussain. 2018). The database for this recognition process consist 15 different categories with 44406 images. The deep conventional neural network is used to make the image processing techniques and extracting the features of the image.  Image processing methods were used to make the sequential alignment of the collected images. The feature extraction was used to focus the important features like color, shape and texture of the images.  These images were directly used for the training and testing process. The representation learning method was used to recognize the images through training. The classification methods are used to classify the class labels after the recognition of the fruit. The dataset was divided into three main processes such as training dataset, testing dataset and validation dataset. The implementation was done with the help of MATLAB and the achieved accuracy was 99%.

Fruit and vegetables classification was done with the help of conventional neural network. This system also adapted the Image Saliency method for the fruit and vegetable recognition(Zeng 2017).  The images for the recognition were collected form the websites. The collected images were in different sizes. So the image processing methods were used to align the images and maintain all the images in the same size which was more convenient for the computation process. Image saliency was used to predict the intuitive notice of the object. This method was used to extract the features form the collected images. They used Bottom-up graph-based visual saliency model to predict the important feature in the image. The VGG model was also used in this object recognition process. This method was used to increase the depth of the network. The prediction accuracy of this system was 95.6%. The researcher of this work gave more importance to collect the real-time images and applied the image processing techniques to form the dataset for computation.

The fruit recognition system was based on the color, shape and texture of the images(Ruaa Adeeb Abdulmunem Al-falluji 2016). The real time images were used for this recognition process. The dataset contains five different fruits. The RGB color images were selected for the computation method. The Gray level Co-occurrence method was used to calculate the texture feature of the image. The image processing methods were used to make the dataset in ready to execute state. This method helps to arrange all the images in the same size and the rotation techniques was also used to make the proper alignment of the images. The feature extraction was the main process to take the important attributes from the images. Color, Shape and texture were the essential features of image processing. The Deep learning methods were used to recognize the fruits based on the given training. The evaluation of the prediction process is based on the number of instances which were correctly identified. The incorrect instances were also considered to calculate the error rate in the recognition process.

A Fruit detection system was developed with the help of deep learning methods(Inkyu Sa. 2016). The Faster Region based Conventional neural network was adapted for the implementation of the fruit recognition system. The proposed method for fruit recognition involved the transfer learning process with two different modalities such as color and near-infrared. The implementation of multi-model R-CNN method helped to make the improved recognition of the fruit. The Pre-trained methods were used for the recognition of fruit such as Image Net. The main advantage of this proposed model was handling the different image modalities for the efficient prediction. The evaluation metrics considered the accuracy, precision and recall of the computation process. The taken dataset was divided for training and testing processes. The faster R-CNN was used to make the computation process by sharing the conventional neural features with classification networks. This consist two main methods such as region proposal and region classifier. The higher accuracy was achieved by using this faster R-CNN method. The precision value was 0.57 and the recall value was 0.8.

Kiwi fruit detection system was developed with the adaptation of Faster R-CNN method(L. F. Fu 2018). The deep learning methods are powerful which are used for image processing methods. The Faster R-CNN neural network is used to make the computation process with the support of back propagation method. The stochastic gradient descent method was implemented with the help of Zeiler and Fergus network (ZFNet). The image acquisition method indicted the data collection for the proposed process. The collected images were arranges with the support of image processing techniques. The conversion of color images into gray scale was done to make the recognition. The Faster R-CNN method was trained to recognize the kiwi fruit in the images. The feature extraction method was used to extract the essential features which were used to recognize the correct fruit. The overall prediction accuracy was calculated. This model produced 92.3% accuracy in the kiwi fruit recognition.

The development of control system for the classification of fruit is developed using Conventional neural network(Zaw Min Khaing. 2018). The computer vision process is the popular method in the research field. The object detection and recognition method was focused by many researchers and the results are produced to solve the object recognition problems. The Fruit-360 dataset was taken for the recognition process. This dataset consist number of digital images related to fruits. The image processing techniques were used. The conventional neural network method was used to recognize the fruits based the training. The parameter optimization was consisted by the conventional neural network to recognize the fruits in the dataset. This also considered the layered process to get the better prediction result. The prediction accuracy was calculated and achieved 94 % of accuracy if fruit recognition process which consist 30 different classes and 971 images. The vision subsystem based control applications are used in the detection system.

Methodology

Convolutional Layer

The convolutional layers are the major layer which is considered as a building block of the conventional neural network. This makes the filtering process which is applied on the input images. This also supports the activation function to reduce the dimensionality reduction. This convolutional neural network has the ability to learn large number of filters from the huge dataset. This method applies the filter to an input image to create a feature map which is used to summarize the detected features of input images. This is a liner operation which supports the multiplication of set of weights in the input of the network. A dot-product is an element-wise multiplication process which is processed between the filter-sized patch of the input and the filter. Then the sum values are produced as a single value. This operation is known as scalar product.

Pooling Layer:

This is the other essential building block of conventional neural network. The main objective of this method is reducing the spatial size of the image and producing the less parameter values which more important to make the training process for the prediction. This pooling layer is used to handle every individual feature map to reduce the size. The pooling layer provides the down sampling process by summarizing all the features which are in the patches of the feature map. The pooling method has two different processes such as average pooling and the max pooling. The average pooling method is used to find the average values of the feature. The max pooling is used to calculate the maximum values of the features.

Fully connected layer

The fully connected layer is used for the classification of images. This layer consist the output from the convolutional or pooling layer for the computation process. This is also used to provide the best class labels for the predicted images. This layer consist the network connection with all the nodes which will precede the next layer. This method consist the flatten (output) from the previous layers and make the values into the single value for further process. There are two fully connected layers are used in this Apple fruit recognition system. the first fully connected layer consist the input values form the previous layers and applies weight for the prediction of appropriate labels. The fully connected output layer produced the final probability for every label.

Activation Function:

The activation function in the neural network is divided into two types. Those are Linear Activation and Non-Linear Activation Functions. There are several activation function are available such as Sigmoid, tanh, Softmax, ReLu and Leaky ReLu etc. For this Apple fruit recognition process the ReLu activation function is used. This is the piece wise linear function. This overcomes the vanishing gradient problem to provide the prediction results in a faster manner. The default activation function for this ReLu is multilayer perceptron.  The ReLu activation function uses the back propagation of error   which is used to train the deep learning algorithms.  This ReLu activation method is the widely used method which can provide higher accuracy in the prediction.

Simulation

Workflow Model for the Apple Fruit Recognition System using Conventional Neural Network

The database was taken from the Benchmark datasets from online sites. The apple dataset consist 6161 images which are used for the apple fruit recognition system.

Fig No: 1 System Workflow Model

Experimental Results:

The dataset for this Apple fruit recognition system consist 6161 images.

Step-1: Data set collection

Step-2: Input layer formulation

Step-3: Hidden layer reduction using kernel

Step-4: Activation function process

Step-5: Output layer produces the result

Dataset introduction:

The fruit 360 dataset is used for this research work. This dataset contains more than 6000 images with six different class labels. This dataset is divided into three main categories. The first category contains 70 percentages of images from the dataset which is used for training. The second category is used for validation that contains 10 percentages of images form the dataset. The third category is sued for testing. This contains 20 percentages of images from the dataset.

Dataset details

ClassImagesTrainingValidationTesting
Apple Braeburn

 

692484

 

69

 

138

 

Apple Crimson Snow

 

740518

 

74

 

148

 

Apple Golden 1

 

1002

 

701

 

100

 

200

 

Apple Golden 2

 

1033

 

723

 

103

 

207

 

Apple Golden 3

 

664

 

465

 

66

 

133

 

Apple Pink Lady

 

2030

 

1421

 

203

 

406

 

Total images 6161

Table No:1 Dataset details

Execution Results:

Fig No: 2 Sample images of first class

Fig No: 3 Sample Images of Class 2

Fig No:4Sample images of class 3

Fig No: 5 Sample images of Class 4

Fig No: 6 Sample Images of Class 5

Fig No: 7 Sample Images of Class 6

Fig No: 8 Reducing Mean Square Error

Training Models

The Training models are used to provide training to the algorithm for recognizing the apples from huge dataset. The feature extraction method is used to make the effective recognition of Apple fruit from the digital images.

Fig No:9 Training accuracy

The dataset is divided into three main types. The 70% of dataset is used for providing training to the model for making the better prediction.

Testing Models

The testing models are used to check the computation process of the model based on the given training.

Fig No: 10Testing Accuracy

The 20% of dataset is used for the testing phase. The testing evaluation is done with the correctly predicted images.

Validation Model

The validation is used to check the efficiency of the used model in the apple fruit prediction process.

Fig No: 11 Validation Analysis

For validating the performance, 10% of dataset is used for the validation analysis. The validation method is used to find the efficiency of the used model. If the desired output is not achieved, then the tuning methods are applied to change the weight of the network and aging repeat the computation process.

Accuracy Calculation

Accuracy prediction is the important process for every research. The main objective of this calculation is to evaluate the  successive rate of the used model for the identified problem. The accuracy is calculated by using the formula:

The achieved percentage ensures the success of the implemented model.

Confusion Matrix

The confusion matrix is the process for calculating the prediction accuracy. For this process four main components are needed. Those are True Positive, True Negative, False Positive and False Negative.

Using these values the accuracy is calculated. In this Apple fruit recognition system, the accuracy was calculated in three different phases. Those are

  • Training Phase Accuracy
  • Validation Phase Accuracy
  • Testing Phase Accuracy
 TrainingValidationTesting
Apple Braeburn

 

95.25

 

94.20

 

90.21

 

Apple Crimson Snow

 

95.56

 

94.59

 

90.44

 

Apple Golden 1

 

95.29

 

94.00

 

90.13

 

Apple Golden 2

 

95.02

 

94.17

 

91.14

 

Apple Golden 3

 

95.48

 

93.94

 

90.80

 

Apple Pink Lady

 

95.21

 

95.50

 

91.11

 

 

Table No: 2 The Training Phase Accuracy

Fig No: 12 Confusion Matrix for testing dataset

The accuracy analysis focused in all the three phases such as training phase, validation phase and testing phase. With this analysis, the highest accuracy is reaches in the training and validation phase. The testing accuracy is also high but comparatively the training and validation accuracies are higher than the testing accuracy.

Discussion

Convolutional neural network provides better classification of apple fruit. The overall training accuracy for this classification is 95%. The overall validation accuracy is 94%. The overall testing accuracy is  90%.

Conclusion:

The Automated Apple Fruit Recognition System is implemented with the adaptation of Deep learning algorithms. The conventional neural network method is used for this apple fruit recognition process. The conventional neural network consist the convolutional layer, pooling layer, fully connected layer and the activation layer for the effective prediction of the images. The MATLAB tool is used to implement the computation process of this automated apple fruit recognition system. The accuracy in training phase, validation phase and testing phase are calculated. The results highlighted that the training and validation accuracy is high when compared with the testing accuracy.

References:

Bibliography

Bargoti, S., & Underwood, J.,. ” Image segmentation for fruit detection and rield estimation in apple orchards. .” Journal of Field Robotics, 2017: Vol-34, issue-6, PP 1039-1060.

Fu, L., Feng, Y., Elkamil T, Liu, Z., Li, R., & Cui, Y. “Image recognition method of multi-cluster kiwifruit in field based on convolutional neural networks.” Transactions of the Chinese Society of Agricultural, 2018: Vol-34, issue-2, PP 205-211.

Fu, L., Feng, Y., Majeed, Y., Zhang, X., Zhang, J., Karkee, M., & Zhang, Q. “Kiwi Fruit detection method using Faster R-CNN with ZFNet.” ScienceDirect, 2018: PP 45-50.

  1. Cheng, L. Damerow, Y. Sun, and M. Blanke.,. “Early yield prediction using image analysis of apple fruit and tree canopy features with neural network.” Journal of Imaging, 2017: vol. 3, p. 6.

Horea Muresan., Mihai Olten.,. “Fruit recognition from images using deeplearning .” Acta Universitatis Sapientiae, Informatica, 2018: Vol-10, issue-1, PP 26-42.

Inkyu Sa., Zongyuan Ge., Feras Dayoub.,Ben Upcroft.,Tristan Perez.,Chris McCool.,. “DeepFruits: A Fruit Detection System Using Deep Neural Networks.” sensors, 2016: Vol-6, issue-8.

Israr Hussain., Qianhua He.,Zhuliang Chen.,. “AUTOMATIC FRUIT RECOGNITION BASED ON DCNN FOR COMMERCIAL SOURCE TRACE SYSTEM.” International Journal on Computational Science & Applications, 2018: Vol.8, No.2/3.

Ruaa Adeeb Abdulmunem Al-falluji. “Color ,Shape and Texture based Fruit Recognition System.” International Journal of Advanced Research in Computer Engineering & Technology, 2016: Volume 5, Issue 7.

Shadman Sakib., Zahidun Ashraf.,Md. Abu Bakr Sidique.,. “Implementation of Fruits Recognition Classifier using Convolutional Neural Network Algorithm for Observation of Accuracies for Various Hidden Layers.” ArXiv e-Journal, 2018.

Zaw Min Khaing., Ye Naung., Phyo Hylam Htut.,. “Development of control system for fruit classification based on convolutional neural network.” 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), 2018.

Zeng, Guoxiang. “Fruit and Vegetables Classification System Using Image Saliency and Convolutional Neural Network.” IEEE 3rd Information Technology and Mechatronics Engineering Conference, 2017: PP 613-617.

Leave a Comment