YARN is more than a tall story. It is central to a new generation of Hadoop.
YARN – Yet Another Resource Negotiator – is a new layer between file storage and data processing that takes cluster management over from MapReduce. According to Hortonworks’ Hadoop web site, this allows Hadoop processing to run more efficiently because it can access multiple data processing centers. The site points out that by utilizing YARN, Hadoop users can run interactive SQL, real-time streaming, data science and batch processing to handle data stored in a single platform.
With YARN, MapReduce’s resource management and scheduling capabilities are removed from data processing, allowing Hadoop to support a greater variety of processing approaches and applications, including performing analysis in real time. YARN also eliminates the need for a highly skilled Java developer. Users only need to be able to drag tools onto a canvas and wire them up into their data flow.
