Published in Dev Genius·2 days agoRecursive CTE — DemystifiedSolving SQL problems recursively CTE Common Table Expression(CTE) is a temporary result set, that can be referenced by DML statements that immediately follow the CTE. The syntax for CTE is as follows: WITH cte AS ( SELECT a, b FROM table1 ) SELECT * FROM cte We can even create more…Database4 min read
Published in Dev Genius·3 days agoDuplicates in SQLAll you need to know for handling duplicates in SQL. Identifying the duplicate entries The first step should be to define which column or combination of columns forms the unique row. …Data3 min read
Published in Dev Genius·4 days agoData Processing ArchitecturesLambda & Kappa Modern data processing platforms need to process real-time streaming events in addition to more traditional data pipelines. Deciding on the right architecture, Lambda or Kappa is an important step in the direction of a good platform strategy. Design goals for Data processing Architectures: Read consistent data Read incrementally from large tables/datasets Rollback Capability…Data Engineering3 min read
Published in Geek Culture·5 days agoOpen Lakehouse — A journeySQL on anything with Dremio Nowadays every company relies on data to make decisions. So every company needs a data infrastructure that will handle their use cases over time. And the use cases have evolved over time. …Data4 min read
Published in Dev Genius·Aug 4Change Data Capture — Intro!A basic introduction to CDC CDC Change Data Capture (CDC) is a data integration design pattern that allows us to track row-level changes in database tables in response to CREATE, UPDATE and DELETE operations. CDC allows us to treat the changes in DB as events and respond to them. Using this…Change Data Capture3 min read
Published in Dev Genius·Aug 2Data Engineer — Learning PathSkills you need to become a data engineer Programming Language Data engineering leads to various roles in the future like Data Analyst, Data Scientist, ML Engineer, and BI Developer. So choosing the right language is important. …Data Engineering3 min read
Jul 25SQL Window functionDissecting the window function Window functions compute the result based on a sliding window frame. A window frame is a set of rows that are related to the current row. The relation is that all the rows have the same value for all terms of the PARTITION BY clause in…Sql5 min read
Published in Dev Genius·Jul 23Shortest Path Algorithm — Floyd Warshall & Johnson’sPart III of Graph traversal algorithm series. Read Part I and Part II In the last blog we learned about two Single Source Shortest Path algorithms, In this blog, we will see two algorithms for finding the shortest path among all node pairs. Floyd Warshall Algorithm This algorithm takes a dynamic programming approach…Python3 min read
Published in Dev Genius·Jul 20Shortest path algorithms — Dijkstra & Bellman-FordPart II of Graph traversal algorithm series. Read Part I and Part IIIAlgorithms4 min read
Published in Dev Genius·Jul 18Databricks Notebook WidgetsReusable Databricks notebook with parameters Input widgets are utility in Databricks that allow us to add parameters to our Dashboards and Notebooks. The Widget API provides methods for creating getting values and removing them. In this blog, we will see how we can use those APIs. Help All the APIs for…Databricks3 min read
