Satisfying ternary permutation constraints by multiple linear orders or phylogenetic trees

Abstract

A ternary permutation constraint satisfaction problem (CSP) is specified by a subset Π of the symmetric group S3. An instance of such a problem consists of a set of variables V and a set of constraints C, where each constraint is an ordered triple of distinct elements from V. The goal is to construct a linear ordering α on V such that, for each constraint (a,b,c)∈C, the ordering of a, b, c induced by α is in Π. Excluding symmetries and trivial cases there are eleven such problems, and their complexity is well known. Here we consider the variant of the problem, denoted 2-Π, where we are allowed to construct two linear orders α and β and each constraint needs to be satisfied by at least one of the two. We give a full complexity classification of all eleven 2-Π problems, observing that in the switch from one to two linear orders the complexity landscape changes quite abruptly and that hardness proofs become rather intricate. To establish the proofs, we use several computer-aided searches. Each such search returns a small instance with a unique solution for a given problem. We then focus on one of the eleven problems in particular, which is closely related to the 2-Caterpillar Compatibility problem in the phylogenetics literature. We show that this particular CSP remains hard on three linear orders, and also in the biologically relevant case when we swap three linear orders for three phylogenetic trees, yielding the 3-Tree Compatibility problem. Lastly, we give extremal results concerning the minimum number of trees required, in the worst case, to satisfy a set of rooted triplet constraints on n leaf labels.