It is well known that agriculture plays a significant role both in the economic and nutrition sectors, while the challenges are the environmental sustainability, the safety of operations and the relief of heavy duties for the workers, especially in a world where the population is increasing tremendously together with the demand for food. Traditional methods, used by farmers, are not sufficient enough to fulfill these requirements. Indeed, the automation process in agriculture started early last century, but today the technology evolution in the field of ICT is driving a deep transformation thanks to the injection of intelligence through IoTs, Big Data analytics, robotics and Artificial Intelligence techniques. In fact, digitization of data collection processes in the crop fields will be at the heart of the next agricultural revolution. A stronger convergence between mechatronics and Artificial Intelligence is expected, mostly in the field of designing autonomous robots with both sensing and acting capabilities, such as ground sampling, fertilizing injection, seeding, etc. The present project proposes the design and development of a middleware software architecture equipped with several Artificial Intelligence algorithms capable of acquiring data related to the field, through suitable sensor devices, and processing them to improve a wide range of agriculture-related tasks, within the field of autonomous precision agriculture. The software architecture is in charge of processing low-level information coming from smart sensors on-board to an agri-robot (e.g. cameras, lidar, suitable soil sensors) solving some advanced recognition tasks through algorithms and techniques mediated by the computer vision fields such as state-of-art deep learning architectures. The main design principles adopted in this context are the control over models' computational complexity and the explainability of both models and autonomous decisions taken by the proposed system.
The proposed software architecture is conceived to process two distinct data levels, the ones coming directly from sensors and those consisting in processing the output of the former. The software system foresees the possibility of storing information (raw data and processed data) in suitable data structures for further processing and performing scenario analysis to provide high level decision-aiding tools for farmers. It is worth to note that information, once processed and transformed in a kind of domain knowledge, needs to be organized in a well-suited form to be used by domain experts (i.e. farmers or agronomists). Hence, the core of the lower intelligent level is grounded on an inductive system with generalization capabilities, declined in different ways (i.e. adopting different Machine Learning techniques) depending on the application at hand (e.g. pests and diseases recognition, chemical-physical structure of the soil). Instead the top level is constituted by a suitable data fusion approach capable of transforming low-level information to actionable knowledge.
- Autonomous systems for agronomist assistance with smart agriculture
In general, humans are reluctant to adopt techniques that are not directly interpretable, tractable and trustworthy. It is customary to think that by focusing solely on performance, the systems will be increasingly opaque [ARRIETA]. This is true in the sense that there is a trade-off between the performance of a model and its transparency. In agriculture applications, the employment of classification systems that can provide a justification about why a prediction has been made can be very effective: firstly, an expert can gain a full insight about a possible threat for the environment and counteracting accordingly if the autonomous system was not equipped with efficient weapons to solve the problem by itself; additionally, the observation of an unreasonable justifications can be used as a starting point for taking corrective actions when in presence of clear system inefficient behaviors, e.g. abuse or tight use of pesticides, disproportionate or limited water provisioning etc...
- Visual reasoning and semantic image retrieval with geometric deep learning
Despite powerful image classification abilities, CNNs alone are unable to provide a clear description of the image features and object relationships which is an utmost peculiarity of an intelligent system that has to provide interpretable information for further analysis of a field expert. On the contrary, the semantic image retrieval task is to retrieve an image from a database by explicitly describing their contents, while the semantic part arises from the fact that not only we are specifying the "objects'' in the image for the retrieval, but also structured relationships and attributes involving these objects. In our proposal, Scene graphs [AGARWAL,XU1] is the candidate technology able to capture the detailed semantics of visual scenes by explicitly modeling objects along with their attributes and relationships with other objects. With the nodes representing the objects in the scene and the edges linking them representing the relationships between the various objects in a dynamic graphical structure, it is possible to obtain a rich representation of the given scene in an image. In such approaches, CNNs are usually exploited as a preprocessing step for the extraction of relevant features. Eventually, such features undergo a refinement phase in charge to refine the node and edge features by analyzing both spatial as well as statistical features present in the image in question by typically involving geometric deep learning methods. Hence, designing a suitable Graph Convolutional Neural network is of paramount importance since it lies at the core of the visual reasoning process, determining the semantic level and the correct relationship between objects.
Neural Architectures parameter pruning
Another important task is the definition of suitable evolutionary procedures for network complexity pruning, allowing to remove neural connections or filters in CNN, maintaining an acceptable performance level with the aim of lowering the computational burden [XU1]. Specifically, working with onboard embedded devices such as the nVidia Xavier Jetson it is of extreme importance to maintain low the memory footprint of neural architectures.
ARRIETA, Alejandro Barredo, et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 2020, 58: 82-115.
AGARWAL, Aniket, et al. Visual Relationship Detection using Scene Graphs: A Survey. arXiv e-prints, 2020, arXiv: 2005.08045.
XU1, Pengfei, et al. A survey of scene graph: Generation and application. EasyChair Preprint, 2020, 3385.
XU2, A. Huang, L. Chen and B. Zhang, Convolutional Neural Network Pruning: A Survey, 2020 39th Chinese Control Conference (CCC), 2020, pp. 7458-7463