The development of reliable fruit detection and localization systems is essential for future sustainable agronomic management of high-value crops. RGB-D sensors have shown potential for fruit detection and localization since they provide 3D information with color data.

The KFuji RGB-DS database is composed by 967 multi-modal images of Fuji apples on trees captured using Microsoft Kinect v2. Each image contains information from 3 different modalities: color (RGB), depth (D) and range corrected intensity (S) -related with the reflectance-. Find more information about how these multimodal images were built in [1, 2].

Color channels are saved in .jpg format, while D and S are saved in .mat format (64-bits). To load D and S channels execute the following lines:

• Matlab:

                            load(‘directory\filename.mat’);

                            D = NIR_DEPTH_res_crop(:,:,2);

                            S = NIR_DEPTH_res_crop(:,:,1);

• Python:

                            D_S = scipy.io.loadmat(‘directory\filename.mat’)

                            D = D_S['NIR_DEPTH_res_crop'][:, :, 1:]

                            S =D_S['NIR_DEPTH_res_crop'][:,:,:1]

All images were manually annotated with rectangular bounding boxes, labelling a total of 12,839 apples in all the dataset. Find annotations in .csv format inside “annotations” folder. Each row corresponds to an apple annotation, giving the following information: item, topleft-x, topleft-y, width, height, label id. You can also find annotations in .xlm format at “square_annotations1” folder.

Figure 1: Sample of 3 multi-modal images extracted from training dataset and their associated fruit location ground truth (red bounding boxes). First column corresponds to RGBhr, second column to S and the third column to D channel. Source: [1]

The database can be downloaded from the following link: KFuji RGB-DS database (2.89 GB)

Find the baseline used for fruit detection in [1] at RGBD_fruit_detection_faster-rcnn.pytorch

This database is available only for research and educational purpose and not for any commercial use. If you use the database in any publications or reports, you should refer to the following papers:

[1] Gené-Mola J, Vilaplana V, Rosell-Polo JR, Morros JR, Ruiz-Hidalgo J, Gregorio E. 2019. Multi-modal Deep Learning for Fruit Detection Using RGB-D Cameras and their Radiometric Capabilities. Computers and Electronics in Agriculture, 162, 689-698. DOI: 10.1016/j.compag.2019.05.016

[2] Gené-Mola J, Vilaplana V, Rosell-Polo JR, Morros JR, Ruiz-Hidalgo J, Gregorio E. 2019. KFuji RGB-DS database: Fuji apple multi-modal images for fruit detection with color, depth and range-corrected IR data. Data in brief, 25 (2019), 104289. DOI: 10.1016/j.dib.2019.104289