Data Engineering #interview

so clustering is used for compression in snowflakes. nice

Can anyone answer the predicate pushdown and partition pruning question ? I think it will not happen if the filtering is on a different column. Also I have a question whether it will be diff between spark 2.x and spark 3 ?

Questions were tricky this time Nisha in good manner. At last should have discussion about Answers and feedback also if you can add in next videos.

Time complexity to search from a dictionary is O(1).

is the answer why snowflake instead of redshift is right? I believe the points she told are applicable to redshift as well, role based access, time travel etc

Your Round1 interview videos are really insighful. Could you please do some Round 2 videos atleast few. It will really help.

Cluster mode - Driver will run in application master container in any one node of a cluster.
Client mode - Driver will run in client jvm (SparkSubmit) in edge node