资 源 简 介
This code is a prototype for Shared Cyclical File Scanning whereby instead of different MapReduce jobs scanning files simultaneously, the scan is shared between different jobs, sharing the overhead of slow and I/O intensive work between different consumers.
This prototype was produced as part of an Open University MSc (in Networks and Distributed Systems) and this code has been shared following the submission of the dissertation project.
The conclusion from benchmarking the prototype was that there are situations where the performance of the prototype exceeds a non-SCFS prototype, although the experimentation was limited to a low-specification cluster and narrow set of jobs.
