Amit Singh RathoreinDev GeniusSpark — Cache, Persist, Checkpoint & write to HDFSOptions to save intermediate results to speed up spark job runtime·4 min read·5 days ago----
Amit Singh RathoreinDev GeniusSpark Interview Questions — XIIIThe next part of the Spark Interview question series·4 min read·May 9, 2024----
Amit Singh RathoreinDev GeniusK8s for Data Engineers — KubeconfigA configuration file that stores config of k8s cluster(s)·3 min read·May 9, 2024----
Amit Singh RathoreSpark memory-linked errorsCommon memory-related issues in Apache Spark applications·3 min read·May 6, 2024----
Amit Singh RathoreK8s for Data Engineer — Container exit codesexit code, their meaning & how to handle them·7 min read·May 5, 2024----
Amit Singh RathoreinDev GeniusBash output redirection for Data Engineerswriting bash command output/error to file·4 min read·May 4, 2024----
Amit Singh RathoreinDev GeniusData Engineering — On call 8Spark Issue faced in BAU·2 min read·May 3, 2024----
Amit Singh RathoreinDev GeniusMastering PySpark — File formatsCheatsheet to work with different file formats in spark.·2 min read·May 3, 2024----
Amit Singh RathoreinDev GeniusAmazon Redshift — An introA brief introduction to Amazon Redshift architecture & its features·12 min read·May 1, 2024--1--1
Amit Singh RathoreinDev GeniusSpark speed-up — AcceleratorsPlugins/engines to improve spark’s runtime performance·3 min read·Apr 27, 2024--1--1