Spark2

1. academic nicety : 学术细节
2. How are stages split into tasks in Spark?
3. Apache Arrow

Spark Shell Startups:

1. Local Mode : bin/spark-shell --master local[3]
2. Standalone Cluster: ./bin/spark-shell --master spark://IP:PORT
3. Big data applications are good candidates for utilizing the Kubernetes architecture because of the scalability and extensibility of Kubernetes clusters.

References

1. How to Install Apache Spark on Multi-Node Cluster (Standalone)
2. Spark On YARN 集群安装部署

3. Spark优化：禁止应用程序将依赖的Jar包传到HDFS – 过往记忆

4. Spark基本工作流程及YARN cluster模式原理

5. Understanding your Apache Spark Application Through Visualization

6. Tuning Apache Spark Jobs the Easy Way: Web UI Stage Detail View

7. 深入研究 spark 运行原理之 job, stage, task

8. Remote Spark Jobs on YARN ---Great

9. Spark project: Spark + Scala + Gradle + IntelliJ

10. Spark应用remote debugging

11. How to print the contents of RDD?

12. A Tale of Three Apache Spark APIs: RDDs, DataFrames, and Datasets

13. How-to: Tune Your Spark Jobs (Part II)
14. Spark testing base
15. Distributed SQL Query Engine : Presto
16. cleanframes — data cleansing library for Apache Spark!