Dataset Information

Pilot-job provisioning on grid resources: Collecting analysis and performance evaluation data.

ABSTRACT: To take advantage of the computing power offered by grid and opportunistic resources, the CERN Large Hadron Collider (LHC) experiments have adopted the Pilot-Job paradigm. In this work, we study the DIRAC Site Director, one of the existing Pilot-Job provisioning solutions, mainly developed and used by the beauty experiment (LHCb). The purpose is to improve the Pilot-Job submission rates and the throughput of the jobs on grid resources. To analyze the DIRAC Site Director mechanisms and assess our contributions, we collected data over 12 months from the LHCbDIRAC instance. We extracted data from the DIRAC databases and the logs. Data include (i) evolution of the number of Pilot-Jobs/jobs over time; (ii) slots available in grid Sites; (iii) number of jobs processed per Pilot-Job.

SUBMITTER: Boyer AF

PROVIDER: S-EPMC9011029 | biostudies-literature | 2022 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Pilot-job provisioning on grid resources: Collecting analysis and performance evaluation data.

Boyer Alexandre F AF Haen Christophe C Stagni Federico F Hill David R C DRC

Data in brief 20220326

To take advantage of the computing power offered by grid and opportunistic resources, the CERN Large Hadron Collider (LHC) experiments have adopted the Pilot-Job paradigm. In this work, we study the DIRAC Site Director, one of the existing Pilot-Job provisioning solutions, mainly developed and used by the beauty experiment (LHCb). The purpose is to improve the Pilot-Job submission rates and the throughput of the jobs on grid resources. To analyze the DIRAC Site Director mechanisms and assess our ...[more]

PMID: 35434224

Similar Datasets

Project description:BackgroundSeveral preprocessing methods are available for the analysis of Affymetrix Genechips arrays. The most popular algorithms analyze the measured fluorescence intensities with statistical methods. Here we focus on a novel algorithm, AffyILM, available from Bioconductor, which relies on inputs from hybridization thermodynamics and uses an extended Langmuir isotherm model to compute transcript concentrations. These concentrations are then employed in the statistical analysis. We compared the performance of AffyILM and other traditional methods both in the old and in the newest generation of GeneChips.ResultsTissue mixture and Latin Square datasets (provided by Affymetrix) were used to assess the performances of the differential expression analysis depending on the preprocessing strategy. A correlation analysis conducted on the tissue mixture data reveals that the median-polish algorithm allows to best summarize AffyILM concentrations computed at the probe-level. Those correlation results are equivalent to the best correlations observed using popular preprocessing methods relying on intensity values. The performances of each tested preprocessing algorithm were quantified using the Latin Square HG-U133A dataset, thanks to the comparison of differential analysis results with the list of spiked genes. The figures of merit generated illustrates that the performances associated to AffyILM(medianpolish), inferred from the present statistical analysis, are comparable to the best performing strategies previously reported.ConclusionsConverting probe intensities to estimates of target concentrations prior to the statistical analysis, AffyILM(medianpolish) is one of the best performing strategy currently available. Using hybridization theory, probe-level estimates of target concentrations should be identically distributed. In the future, a probe-level multivariate analysis of the concentrations should be compared to the univariate analysis of probe-set summarized expression data.

Dataset Information

Pilot-job provisioning on grid resources: Collecting analysis and performance evaluation data.

Publications

Pilot-job provisioning on grid resources: Collecting analysis and performance evaluation data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets