An adaptive model-free reinforcement learning based communication algorithm for underwater sensor networks
Underwater emerging applications such as monitoring the critical infrastructure, coastline protection, ecosystem analysis, pollution control and predicting disasters like underwater seismic and volcanic events, are becoming more and more sophisticated and produce more complex data which need to be delivered to the collection points on the surface. However, due to the time-varying and unstable underwater environment, designing and deploying a reliable, low latency and energy efficient underwater sensor network is yet a challenge. This research aims to design an adaptive model-free reinforcement learning based communication algorithm for underwater sensor networks that keeps up with the dynamic changes of the environment and obtains knowledge on the underwater channel conditions in real-time to automatically adapt to the system and make decisions on how to transmit the data packets accordingly. Model-free reinforcement learning approach seems to be a great fit for the time-varying underwater scenario as no predefined model or static assumptions of the environment is required in advance and the knowledge on the environment can be learnt and estimated by experience in the field. The decision on how to send the packets will be made based on the transmission cost, link quality and battery status. These decision factors will be updated depending on the current state of the underwater network and indeed are the key to make this system adaptive to the environment. The performance of this approach will then be realized and evaluated with "SUNSET", a simulator developed by Sapienza University UWSNs group, that models a wide variety of details of the underwater channel and environment realistically, before being deployed in the underwater fields.