Skip to main content

Data source qualification

Data source rule

Menu: Library/Data

You can measure the quality level of any source by assigning rules to the columns and then measure the source quality.

If you take the rule described in the previous sections, it can be assigned to the zip code column.

Go to the BikeStores.sales.customers.zip code column page in the catalog view and click on edit to edit the asset.

Then click on edit in the Quality section. Select a rule in the popup window, move it to the right and save.

data_source_qualification

Save the asset via ok.

data_source_qualification

Make sure that a calendar has been assigned to the source level page in the catalog.

data_source_qualification

You can now measure the quality of the data source.

Data source measurement

Menu: Administration/Measurement/Execution

Select the earliest future calendar date displayed on the left, select the data source, and click on run.

data_source_qualification

Click on refresh to refresh the screen.

Then go to the cockpit to the see the quality level of the source.

You can navigate from the source level to the column level on the left.

data_source_qualification

Rule template

If you have the similar columns in different sources, instead of creating and assigning a different rule to each column, you can create a template rule and assign it to both columns.

Example in BikeStores, there is a column zip_code in table customers and a column zip_code in table stores.

Create a template rule under the rule set ‘Sales Process’.

info

Reminder : with Yavantha-CE, you can only create 10 rules.

note

You have to deselect the measurement flags for a template rule.

data_source_qualification

Go to Menu: Library/Rule/Script

data_source_qualification

Click on + to create a new script.

data_source_qualification

The first line of a template script must be : -- @template

Click on ok to save the script

Go back to the rule.

It should be in draft status. Click on edit to enter edit mode.

From the Script section, click on edit and associate the script

data_source_qualification

Save the rule via ok and publish it via publish.

Now you can associate the rule to the two zip_code columns.

data_source_qualification

You can now measure the quality of the data source

Go to menu: Administration/Measurement/Execution

Select the earliest future calendar date displayed on the left, select the data source, and click on run.

data_source_qualification

data_source_qualification

Click on the Source name to be directed to the corresponding cockpit or go to Menu: Governance/Cockpit (Catalog)

You can see the quality measured for the two zip_code columns , for the tables, for the schemas and for the source.

data_source_qualification

data_source_qualification