Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Random access with a distributed Bitmap Join Index for Star Joins.

ABSTRACT: Indices improve the performance of relational databases, especially on queries that return a small portion of the data (i.e., low-selectivity queries). Star joins are particularly expensive operations that commonly rely on indices for improved performance at scale. The development and support of index-based solutions for Star Joins are still at very early stages. To address this gap, we propose a distributed Bitmap Join Index (dBJI) and a framework-agnostic strategy to solve join predicates in linear time. For empirical analysis, we used common Hadoop technologies (e.g., HBase and Spark) to show that dBJI significantly outperforms full scan approaches by a factor between 59% and 88% in queries with low selectivity from the Star Schema Benchmark (SSB). Thus, distributed indices may significantly enhance low-selectivity query performance even in very large databases.

SUBMITTER: Brito JJ

PROVIDER: S-EPMC7030998 | biostudies-literature | 2020 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Random access with a distributed Bitmap Join Index for Star Joins.

Brito Jaqueline J JJ Mosqueiro Thiago T Ciferri Ricardo R RR Ciferri Cristina D A CDA

Heliyon 20200217 2

Indices improve the performance of relational databases, especially on queries that return a small portion of the data (i.e., low-selectivity queries). Star joins are particularly expensive operations that commonly rely on indices for improved performance at scale. The development and support of index-based solutions for Star Joins are still at very early stages. To address this gap, we propose a distributed Bitmap Join Index (dBJI) and a framework-agnostic strategy to solve join predicates in l ...[more]

PMID: 32099915

Similar Datasets

Near-Optimal Distributed Band-Joins through Recursive Partitioning.

Project description:We consider running-time optimization for band-joins in a distributed system, e.g., the cloud. To balance load across worker machines, input has to be partitioned, which causes duplication. We explore how to resolve this tension between maximum load per worker and input duplication for band-joins between two relations. Previous work suffered from high optimization cost or considered partitionings that were too restricted (resulting in suboptimal join performance). Our main insight is that recursive partitioning of the join-attribute space with the appropriate split scoring measure can achieve both low optimization cost and low join cost. It is the first approach that is not only effective for one-dimensional band-joins but also for joins on multiple attributes. Experiments indicate that our method is able to find partitionings that are within 10% of the lower bound for both maximum load per worker and input duplication for a broad range of settings, significantly improving over previous work.

| S-EPMC7872589 | biostudies-literature

Efficient string similarity join in multi-core and distributed systems.

Project description:In big data area a significant challenge about string similarity join is to find all similar pairs more efficiently. In this paper, we propose a parallel processing framework for efficient string similarity join. First, the input is split into some disjoint small subsets according to the joint frequency distribution and the interval distribution of strings. Then the filter-verification strategy is adopted in the computation of string similarity for each subset so that the number of candidate pairs is reduced before an effective pruning strategy is used to improve the performance. Finally, the operation of string join is executed in parallel. Para-Join algorithm based on the multi-threading technique is proposed to implement the framework in a multi-core system while Pada-Join algorithm based on Spark platform is proposed to implement the framework in a cluster system. We prove that Para-Join and Pada-Join cannot only avoid reduplicate computation but also ensure the completeness of the result. Experimental results show that Para-Join can achieve high efficiency and significantly outperform than state-of-the-art approaches, meanwhile, Pada-Join can work on large datasets.

| S-EPMC5344375 | biostudies-other

Random access parallel microscopy.

Project description:We introduce a random-access parallel (RAP) imaging modality that uses a novel design inspired by a Newtonian telescope to image multiple spatially separated samples without moving parts or robotics. This scheme enables near-simultaneous image capture of multiple petri dishes and random-access imaging with sub-millisecond switching times at the full resolution of the camera. This enables the RAP system to capture long-duration records from different samples in parallel, which is not possible using conventional automated microscopes. The system is demonstrated by continuously imaging multiple cardiac monolayer and Caenorhabditis elegans preparations.

| S-EPMC7843131 | biostudies-literature

Spectral correlations in a random distributed feedback fibre laser.

Project description:Random distributed feedback fibre lasers belong to the class of random lasers, where the feedback is provided by amplified Rayleigh scattering on sub-micron refractive index inhomogenities randomly distributed over the fibre length. Despite the elastic nature of Rayleigh scattering, the feedback mechanism has been insofar deemed incoherent, which corresponds to the commonly observed smooth generation spectra. Here, using a real-time spectral measurement technique based on a scanning Fabry-Pérot interferometer, we observe long-living narrowband components in the random fibre laser's spectrum. Statistical analysis of the ∼104 single-scan spectra reveals a preferential interspacing for the components and their anticorrelation in intensities. Furthermore, using mutual information analysis, we confirm the existence of nonlinear correlations between different parts of the random fibre laser spectra. The existence of such narrowband spectral components, together with their observed correlations, establishes a long-missing parallel between the fields of random fibre lasers and conventional random lasers.

| S-EPMC5454388 | biostudies-literature

Purely antiferromagnetic magnetoelectric random access memory.

Project description:Magnetic random access memory schemes employing magnetoelectric coupling to write binary information promise outstanding energy efficiency. We propose and demonstrate a purely antiferromagnetic magnetoelectric random access memory (AF-MERAM) that offers a remarkable 50-fold reduction of the writing threshold compared with ferromagnet-based counterparts, is robust against magnetic disturbances and exhibits no ferromagnetic hysteresis losses. Using the magnetoelectric antiferromagnet Cr2O3, we demonstrate reliable isothermal switching via gate voltage pulses and all-electric readout at room temperature. As no ferromagnetic component is present in the system, the writing magnetic field does not need to be pulsed for readout, allowing permanent magnets to be used. Based on our prototypes, we construct a comprehensive model of the magnetoelectric selection mechanisms in thin films of magnetoelectric antiferromagnets, revealing misfit induced ferrimagnetism as an important factor. Beyond memory applications, the AF-MERAM concept introduces a general all-electric interface for antiferromagnets and should find wide applicability in antiferromagnetic spintronics.

| S-EPMC5216083 | biostudies-literature

Random distributed logistic regression framework for predicting potential lncRNA‒disease association.

Project description: Not available

| S-EPMC8373264 | biostudies-literature

High-scale random access on DNA storage systems.

Project description:Due to the rapid cost decline of synthesizing and sequencing deoxyribonucleic acid (DNA), high information density, and its durability of up to centuries, utilizing DNA as an information storage medium has received the attention of many scientists. State-of-the-art DNA storage systems exploit the high capacity of DNA and enable random access (predominantly random reads) by primers, which serve as unique identifiers for directly accessing data. However, primers come with a significant limitation regarding the maximum available number per DNA library. The number of different primers within a library is typically very small (e.g. ≈10). We propose a method to overcome this deficiency and present a general-purpose technique for addressing and directly accessing thousands to potentially millions of different data objects within the same DNA pool. Our approach utilizes a fountain code, sophisticated probe design, and microarray technologies. A key component is locality-sensitive hashing, making checks for dissimilarity among such a large number of probes and data objects feasible.

| S-EPMC8829907 | biostudies-literature

A Rewritable, Random-Access DNA-Based Storage System.

Project description:We describe the first DNA-based storage architecture that enables random access to data blocks and rewriting of information stored at arbitrary locations within the blocks. The newly developed architecture overcomes drawbacks of existing read-only methods that require decoding the whole file in order to read one data fragment. Our system is based on new constrained coding techniques and accompanying DNA editing methods that ensure data reliability, specificity and sensitivity of access, and at the same time provide exceptionally high data storage capacity. As a proof of concept, we encoded parts of the Wikipedia pages of six universities in the USA, and selected and edited parts of the text written in DNA corresponding to three of these schools. The results suggest that DNA is a versatile media suitable for both ultrahigh density archival and rewritable storage applications.

| S-EPMC4585656 | biostudies-literature

Expected value of first Zagreb connection index in random cyclooctatetraene chain, random polyphenyls chain, and random chain network.

Project description:The Zagreb connection indices are the known topological descriptors of the graphs that are constructed from the connection cardinality (degree of given nodes lying at a distance 2) presented in 1972 to determine the total electron energy of the alternate hydrocarbons. For a long time, these connection indices did not receive much research attention. Ali and Trinajstić [Mol. Inform. 37, Art. No. 1800008, 2018] examined the Zagreb connection indices and found that they compared to basic Zagreb indices and that they provide a finer value for the correlation coefficient for the 13 physico-chemical characteristics of the octane isomers. This article acquires the formulae of expected values of the first Zagreb connection index of a random cyclooctatetraene chain, a random polyphenyls chain, and a random chain network with l number of octagons, hexagons, and pentagons, respectively. The article presents extreme and average values of all the above random chains concerning a set of special chains, including the meta-chain, the ortho-chain, and the para-chain.

| S-EPMC9846538 | biostudies-literature

Three-dimensional multi-site random access photostimulation (3D-MAP).

Project description:Optical control of neural ensemble activity is crucial for understanding brain function and disease, yet no technology can achieve optogenetic control of very large numbers of neurons at an extremely fast rate over a large volume. State-of-the-art multiphoton holographic optogenetics requires high-power illumination that only addresses relatively small populations of neurons in parallel. Conversely, one-photon holographic techniques can stimulate more neurons with two to three orders lower power, but with limited resolution or addressable volume. Perhaps most problematically, two-photon holographic optogenetic systems are extremely expensive and sophisticated which has precluded their broader adoption in the neuroscience community. To address this technical gap, we introduce a new one-photon light sculpting technique, three-dimensional multi-site random access photostimulation (3D-MAP), that overcomes these limitations by modulating light dynamically, both in the spatial and in the angular domain at multi-kHz rates. We use 3D-MAP to interrogate neural circuits in 3D and demonstrate simultaneous photostimulation and imaging of dozens of user-selected neurons in the intact mouse brain in vivo with high spatio-temporal resolution. 3D-MAP can be broadly adopted for high-throughput all-optical interrogation of brain circuits owing to its powerful combination of scale, speed, simplicity, and cost.

| S-EPMC8843094 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data