What is Spark? Apache Spark is an In-memory, massively parallel computing Engine and framework, which was specially designed to work in petabyte size of data and to overcome demerits of MapReduce engine (A Hadoop Big Data processing framework) Origin of Hero: Spark is the fastest in-memory data processing Big Data framework which operates in cluster mode. Spark was born in Berkeley AMPlab in 2009 by Matei Zaharia and became Apache open source project in April 2010. In February 2014 Apache spark became most famous big data tool, And the escalation of apache spark kept on spreading and in 2016 it had the most number of open source contributors. Note: Later Matei Zaharia started his company named Databricks, which provides a cloud-based Analytical platform for Big Data development base on Apache Spark What spark can do? To answer this question I would like to ask you a simple question “what a superhero can do?”, Almost everything right. Just like a superhero, Apache spark also can do almost everything in the big data world, from Data collecting, cleansing, and Mining to making Advance Machine Learning models. It can do whatever you need, with lightning-fast processing speed. Apache Spark can be used for

• Data analytics
• Data Processing
• Streaming jobs
• Making ML models
• Developing big data application

Spark is a Fast and flexible tool, it can be connected or communicate with any Big Data tool and Database or Application. Spark has incredible open source community support where any problem or issue gets resolved quickly. Specialty of Spark Apache Spark is Swiss army knife, it can do analytics on data in rest and also on Live data (Streaming data). It has very powerful spectacular APIs to perform challenging task in a very smooth way. Apache Spark APIs:

• Spark core (RDD)
• Spark DataFrame
• SQL API
• Streaming APIs
• MlLib API
• Graphx

Spark is like a tool which is given by the God because it supports most popular and powerful programming languages like Python, Scala, R, and Java. Developers can write codes in their favorite language while having the advantage of robust and comprehensible APIs. Big tech giants like Facebook, Google, Apple, and IBM also uses Apache Spark for Data processing, Analyzing and making ML modeling.