Spark Performance Tuning – Part 1 | Data Exposed

This week’s Data Exposed show welcomes back Maxim Lukiyanov to kick off a 4-part series on Spark performance tuning with Spark 2.x. Maxim is a Senior PM on the big data HDInsight team and is in the studio today to present Part 1 of his 4-part series.

[03:00] – Spark Use Cases as it pertains to performance.

[06:00] – Performance Demos and why performance matters. Query graphs and query performance depending on query optimization.

[11:00] – Spark 2.x project Tungsten overview, which includes: [13:45] Catalyst query optimizer,  [15:20] Direct Memory management, [18:10] WholeStage Code generation.


from Channel 9


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s