loading...

Discussion on: What was your win this week?

Collapse
waylonwalker profile image
Waylon Walker

I learned how to combine kedro catalogs and submitted it as a feature request.

[KED-1512] Ability to add catalogs together #302

Description

I am would like to request the ability to add two DataCatalog objects together to get the addition of the two. This will improve the ability to reuse pipelines and catalogs from existing projects.

Example

Let's say I already have two DataCatalogs loaded in c1 and c2. I would like the ability to add them together.

full_catalog = c1 + c2

Context

I am trying to combine pipelines from two different projects. Kedro natively gives me the ability to add the pipelines together, but not the DataCatalog

Possible Implementation

Create a new DataCatalog using dictionary unpacking from the individual DataCatalogs. In the case of duplicates the second catalog will take preference.

def __add__(self, other):
   DataCatalog({**self.__dict__['_data_sets'], **other.__dict__['_data_sets']})