Business Data Lake Conceptual Framework
Download 493.56 Kb.
|
Analytics Engine
Analytics Engines represent the computational frameworks that host Analytics. They enable “business” oriented Analytics to be executed by the Business Data Lake infrastructures. Compared to traditional approaches, Big Data engines automatically deal with the distributed infrastructures, the eventual failure of nodes (hardware or software) so that when a user (a Data Scientist) is designing Analytics, he/she can focus only on the business and mathematical parts, without explicitly coding resources management parts Analytics Engines are key components inside the Business Data Lake platform that enable Analytics to be created and executed. According to the nature of Analytics and the performance needs, we identify three main types of Analytics Engines: Disk-Based engines are distributed engines that read data from disk (ideally locally on the same node but also from another node through the network) and write results to disk. Apache Map Reduce is an example of Disk-Based Engine. In-Memory Engines have the ability to keep data into RAM so that it’s possible to drastically speed-up the processing when data sets fit in RAM (still on multiple nodes). Selected the data sets to be cached in RAM can be either automatic or explicitly stated into the analysis code by the user. Apache Spark is an example of such In-memory Engine. Coming from the High-Performance Computing world, we start to see for some specific analyses that can be massively distributed on thousands of small units the use of hardware-accelerated engines based on GPUs (Graphical Processing Units) or MICs (Many Integrated Cores). “Deep learning” analyses, which rely on training very large neural networks (with billions of connections) with very large datasets, can typically leverage these kind of solutions. Directory: bigdata bigdata -> Use Cases from nbd(nist big Data) Requirements wg bigdata -> Use Cases from nbd(nist big Data) Requirements wg 0 bigdata -> Nist big Data Public Working Group (nbd-pwg) nbd-pwd-2015/6a,DW. abbreviated rr (M0444) Source: nbd-pwg status: Draft Title: Big Data Use Case #6 Implementation, using nbdra author: Afzal Godil bigdata -> Nist special Publication 1500-4 draft: nist big Data Interoperability Framework: Volume 4, Security and Privacy Download 493.56 Kb. Share with your friends:
|