Fog Computing is today a widely used computing paradigm, since it is able to drastically reduce service latency by physically moving the computing units in a location that is near to the end users. The core feature of Fog Computing is that the computation is distributed within a set of Fog nodes, that are usually spread across a geographic domain, in general the user requests a service to the nearest node, and the installation of nodes at the cellular base stations is straightforward. In this context, we wonder how we can enable these nodes to share resources, indeed due to their physical position some of them can be overloaded and others can be completely idle. Moreover, in a green-oriented environment, a device could be backed up with a battery and some scheduling decision could be made taking into account how much energy they will require. The role of this project is to study algorithms and solutions to enable the cooperation of fog nodes by using the Reinforcement Learning approach, optimizing latency, energy consumption and computation performances. Reinforcement Learning is a clear step forward with respect to all of the fixed strategies that come into play in this context since it enables an agent to learn scheduling policies by means of the experience that is driven by reward signals. The developed strategies will be studied with simulations, based on event simulators but also by using a pseudo-real deployment of fog nodes based on Single-Board-Computers (SBCs) like the Raspberry Pis. Indeed, thanks to their low cost and to their non-negligible computing power they can be easily envisioned as a Fog Computing environment.
The main innovations of this research are deriving of the lacks of the already given solutions in literature that they do not specifically target the Fog Computing environment. In particular, the main novelty points of this research project can be summarized as follows.
- device heterogeneity most of the studies available in literature are not targeting the fog computing and therefore they do not consider the fact that devices can be very different in their computing capabilities, indeed, training a neural network is not always feasible; this project will also focus on this particular aspect;
- decentralized learning, the innovation that will be brought by this project refers to the fact that the learning strategies are fully distributed without a single entity that is in charge of collecting all the data from the nodes and start the training process, therefore the learning agents (fog nodes) will indirectly learn from the behavior of the others in the environment;
- online scheduling, the principal focus of this project is the online scheduling, this means that the scheduling decision (regarding where to execute the task that has been requested), is made per request and not per batches of requests, and this is a crucial innovation aspect because it is indispensable when we deal with tasks that have strict deadlines;
- deadlines, another point of innovation regards the facing of the scheduling problem with Reinforcement Learning but considering tasks that have deadlines, and deadlines arises a entire new set of issues when dealing with learning a scheduling policy, indeed if a task has a deadline the learner needs to decide whether to take the risk of offloading it to another node or execute it locally; this is aspect is not explicitly faced in literature and it is another point of innovation of this project;
- green energy, the role of an adaptive scheduling algorithm in Fog Computing should also take into consideration the energy consumption, especially if that particular fog node is powered by solar energy, for example; this aspect will be investigated in the project;
- geographic approach, the last innovation point resides on considering all of the previous points in a geographic domain, this means that the solutions will be studied (is simulation or in a pseudo-real environment) by explicitly positioning fog nodes in a city, for example; by studying the traffic data, we are allowed to understand how resilient will be our solution if it is deployed in a real environment and this is another lack of the current state-of-the-art within this context.