Near-Optimal Distributed Band-Joins through Recursive Partitioning.

ABSTRACT: We consider running-time optimization for band-joins in a distributed system, e.g., the cloud. To balance load across worker machines, input has to be partitioned, which causes duplication. We explore how to resolve this tension between maximum load per worker and input duplication for band-joins between two relations. Previous work suffered from high optimization cost or considered partitionings that were too restricted (resulting in suboptimal join performance). Our main insight is that recursive partitioning of the join-attribute space with the appropriate split scoring measure can achieve both low optimization cost and low join cost. It is the first approach that is not only effective for one-dimensional band-joins but also for joins on multiple attributes. Experim

SUBMITTER: Li R 

PROVIDER: S-EPMC7872589 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature