One of the responsibilities of a data engineer is bringing data together from different sources into a data warehouse. In a data warehouse you gather facts and dimensions. The latter contain relatively static data about entities as geographical locations, customers, or products.
This dimension data can change slowly and irregularly over time. So-called Slowly Changing Dimensions (SCD) should be handled to make sure the referential integrity within the data warehouse is correct. This blog consists of two parts: in the first we will talk about the types of Slowly Changing Dimensions with a focus on SCD type 2; In the second blog we will cover how to implement SCD type 2 in Talend.