Network coding for distributed cloud, fog and data center storage

Content Provider	Semantic Scholar
Author	Sipos, Márton Ákos
Copyright Year	2018
Abstract	As the amount of data created and consumed each day increases at an alarming rate, distributed storage systems have turned towards erasure coding to keep up with demand. Several well-known codes exist that decrease storage costs significantly, but introduce new challenges that limit their use. Furthermore, there is no single technique that is widely applied across different types of systems. One technique that may be an exception due to its flexibility is network coding. We set out to look at how network coding can be applied in three distinct scenarios ranging from the mostly static data center environment to the highly dynamic fog computing setting. We also looked at how aggregating multiple cloud storage services can alleviate some of the issues related to single–cloud solutions and examined the problem of updating data after it has been erasure coded. We used network flows to model repair and reconstruction processes, followed by results from measurements and simulations to validate theory. When looking at erasure coding for fog computing, we used real–world traces to evaluate its feasibility. We assessed the effectiveness of our proposed solution on updating erasure coded data using git and a publicly available repository. We proposed techniques to reduce the burden of repairs by up to 35% on data center networks and presented a system of checks for network coding that determines whether a repair maintains the required level of reliability. We showed that aggregating multiple cloud storage services and applying network coding improves retrieval performance by between 34%–61% and alleviates some of the the reliability and privacy concerns related to single–cloud solutions. We proposed a schema that adapts data distribution with the goal of reducing retrieval time. For fog computing, we compared different erasure codes on their ability to maintain the integrity of an edge cloud of mobile devices. We also touched on predicting node availability and showed that mobile storage clouds are feasible with as little as 25% redundancy. Finally, we proposed a mechanism to solve a challenge common across most storage systems, updating erasure coded data after it has changed. Given realistic update patterns, it requires 5 orders of magnitudes less storage compared to state-of-the-art solutions. Our hope is that the results presented in this dissertation advance the state of the art by enabling erasure coding to be applied in more cases, more effectively.
File Format	PDF HTM / HTML
Alternate Webpage(s)	http://vbn.aau.dk/files/294609209/PHD_Marton_A_Sipos_E_pdf.pdf
Alternate Webpage(s)	https://vbn.aau.dk/ws/portalfiles/portal/294609209/PHD_Marton_A_Sipos_E_pdf.pdf
Language	English
Access Restriction	Open
Content Type	Text
Resource Type	Article