Earlier for each file spark.read.json was called, now spark.read.json is called only one at the root folder. This reduces the number of stages.