Discovery Algorithms for Embedded Uniqueness Constraints

Abstract

Data profiling is an enabler for efficient data management and effective analytics. The discovery of data dependencies is at the core of data profiling. We conduct the first study on the discovery of embedded uniqueness constraints (eUCs), a recently introduced class of data dependencies that represent unique column combinations embedded in complete fragments of incomplete data. We show that the decision variant of finding a minimal eUC is NP-complete and W[2]-complete in the input size. We also characterize the maximum possible solution size, and show which families of eUCs attain that size. The size is much larger than for the special case of minimal SQL uniques. Despite these challenges, our column-efficient, rowefficient, and hybrid discovery algorithms perform effectively and fast on real-world benchmark and synthetic data. We also propose the computation of small semantic samples of given data sets as a new direction in data profiling. These samples satisfy the same eUCs as the given data set and we showcase how discovery and sampling together provide a pathway towards effective data cleansing and business rule acquisition.