Spark Performance Tuning – Part 1 | Data Exposed

This week’s Data Exposed show welcomes back Maxim Lukiyanov to kick off a 4-part series on Spark performance tuning with Spark 2.x. Maxim is a Senior PM on the big data HDInsight team and is in the studio today to present Part 1 of his 4-part series.

[03:00] – Spark Use Cases as it pertains to performance.

[06:00] – Performance Demos and why performance matters. Query graphs and query performance depending on query optimization.

[11:00] – Spark 2.x project Tungsten overview, which includes: [13:45] Catalyst query optimizer,  [15:20] Direct Memory management, [18:10] WholeStage Code generation.

 

from Channel 9 https://channel9.msdn.com/Shows/Data-Exposed/Spark-Performance-Series-1-with-Maxim-Lukiyanov

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s