Multi-resource collaborative optimization for adaptive virtual machine placement.
Ontology highlight
ABSTRACT:
SUBMITTER: Li Z
PROVIDER: S-EPMC8771770 | biostudies-literature | 2022
REPOSITORIES: biostudies-literature
Publications
Multi-resource collaborative optimization for adaptive virtual machine placement.
Li Zhihua Z Pan Meini M Yu Lei L
PeerJ. Computer science 20220106
The unbalanced resource utilization of physical machines (PMs) in cloud data centers could cause resource wasting, workload imbalance and even negatively impact quality of service (QoS). To address this problem, this paper proposes a multi-resource collaborative optimization control (MCOC) mechanism for virtual machine (VM) migration. It uses Gaussian model to adaptively estimate the probability that the running PMs are in the multi-resource utilization balance status. Given the estimated probab ...[more]
Similar Datasets
Project description:The demand for virtual machine requests has increased recently due to the growing number of users and applications. Therefore, virtual machine placement (VMP) is now critical for the provision of efficient resource management in cloud data centers. The VMP process considers the placement of a set of virtual machines onto a set of physical machines, in accordance with a set of criteria. The optimal solution for multi-objective VMP can be determined by using a fitness function that combines the objectives. This paper proposes a novel model to enhance the performance of the VMP decision-making process. Placement decisions are made based on a fitness function that combines three criteria: placement time, power consumption, and resource wastage. The proposed model aims to satisfy minimum values for the three objectives for placement onto all available physical machines. To optimize the VMP solution, the proposed fitness function was implemented using three optimization algorithms: particle swarm optimization with Lévy flight (PSOLF), flower pollination optimization (FPO), and a proposed hybrid algorithm (HPSOLF-FPO). Each algorithm was tested experimentally. The results of the comparative study between the three algorithms show that the hybrid algorithm has the strongest performance. Moreover, the proposed algorithm was tested against the bin packing best fit strategy. The results show that the proposed algorithm outperforms the best fit strategy in total server utilization.
| S-EPMC8771797 | biostudies-literature
Project description:Collaboration among logistics facilities in a multicenter logistics delivery network can significantly improve the utilization of logistics resources through resource sharing including logistics facilities, vehicles, and customer services. This study proposes and tests different resource sharing schemes to solve the optimization problem of a collaborative multicenter logistics delivery network based on resource sharing (CMCLDN-RS). The CMCLDN-RS problem aims to establish a collaborative mechanism of allocating logistics resources in a manner that improves the operational efficiency of a logistics network. A bi-objective optimization model is proposed with consideration of various resource sharing schemes in multiple service periods to minimize the total cost and number of vehicles. An adaptive grid particle swarm optimization (AGPSO) algorithm based on customer clustering is devised to solve the CMCLDN-RS problem and find Pareto optimal solutions. An effective elite iteration and selective endowment mechanism is designed for the algorithm to combine global and local search to improve search capabilities. The solution of CMCLDN-RS guarantees that cost savings are fairly allocated to the collaborative participants through a suitable profit allocation model. Compared with the computation performance of the existing nondominated sorting genetic algorithm-II and multi-objective evolutionary algorithm, AGPSO is more computationally efficient. An empirical case study in Chengdu, China suggests that the proposed collaborative mechanism with resource sharing can effectively reduce total operational costs and number of vehicles, thereby enhancing the operational efficiency of the logistics network.
| S-EPMC7682872 | biostudies-literature
Project description:Fine-tuning is an important technique in transfer learning that has achieved significant success in tasks that lack training data. However, as it is difficult to extract effective features for single-source domain fine-tuning when the data distribution difference between the source and the target domain is large, we propose a transfer learning framework based on multi-source domain called adaptive multi-source domain collaborative fine-tuning (AMCF) to address this issue. AMCF utilizes multiple source domain models for collaborative fine-tuning, thereby improving the feature extraction capability of model in the target task. Specifically, AMCF employs an adaptive multi-source domain layer selection strategy to customize appropriate layer fine-tuning schemes for the target task among multiple source domain models, aiming to extract more efficient features. Furthermore, a novel multi-source domain collaborative loss function is designed to facilitate the precise extraction of target data features by each source domain model. Simultaneously, it works towards minimizing the output difference among various source domain models, thereby enhancing the adaptability of the source domain model to the target data. In order to validate the effectiveness of AMCF, it is applied to seven public visual classification datasets commonly used in transfer learning, and compared with the most widely used single-source domain fine-tuning methods. Experimental results demonstrate that, in comparison with the existing fine-tuning methods, our method not only enhances the accuracy of feature extraction in the model but also provides precise layer fine-tuning schemes for the target task, thereby significantly improving the fine-tuning performance.
| S-EPMC11232598 | biostudies-literature
Project description:Regarding the fulfillment optimization of online retail orders, many researchers focus more on warehouse optimization and distribution center optimization. However, under the background of new retailing, traditional retailers carry out online services, forming an order fulfillment model with physical stores as front warehouses. Studies that focus on physical stores and consider both order splitting and store delivery are rare, which cannot meet the order optimization needs of traditional retailers. To this end, this study proposes a new problem called the "Multi-Store Collaborative Delivery Optimization (MCDO)", in which not only make the order-split plans for stores but also design the order-delivery routes for them, such that the order fulfillment cost is minimized. To solve the problem, a Top-K breadth-first search and a local search are integrated to construct a hybrid heuristic algorithm, named "Top-K Recommendation & Improved Local Search (TKILS)". This study optimizes the search efficiency of the breadth-first search by controlling the number of sub-orders and improving the initial solution of the local search using a greedy cost function. Then achieve the joint optimization of order-split and order-delivery by improving the local optimization operators. Finally, extensive experiments on synthetic and real datasets validate the effectiveness and applicability of the algorithm this study proposed.
| S-EPMC9997996 | biostudies-literature
Project description:Guiding experiments to find materials with targeted properties is a crucial aspect of materials discovery and design, and typically multiple properties, which often compete, are involved. In the case of two properties, new compounds are sought that will provide improvement to existing data points lying on the Pareto front (PF) in as few experiments or calculations as possible. Here we address this problem by using the concept and methods of optimal learning to determine their suitability and performance on three materials data sets; an experimental data set of over 100 shape memory alloys, a data set of 223 M2AX phases obtained from density functional theory calculations, and a computational data set of 704 piezoelectric compounds. We show that the Maximin and Centroid design strategies, based on value of information criteria, are more efficient in determining points on the PF from the data than random selection, pure exploitation of the surrogate model prediction or pure exploration by maximum uncertainty from the learning model. Although the datasets varied in size and source, the Maximin algorithm showed superior performance across all the data sets, particularly when the accuracy of the machine learning model fits were not high, emphasizing that the design appears to be quite forgiving of relatively poor surrogate models.
| S-EPMC5829239 | biostudies-literature
Project description:Faced with the land use heterogeneity and trip distribution non-equilibrium along the corridor connecting urban district and rural area, the operation characteristics of the all-stop bus, the express bus and the shuttle bus running on the transportation corridor should be given full consideration, as well as the tradeoff between passenger trip cost and bus operation cost during the peak-hour period. In order to realize the reliable decision of bus resources configuration for urban-rural public transportation, a bi-objective programming model of multi-modal urban-rural bus operation scheme is established, where the decision variables include the departure frequency of different bus modals, the terminals of shuttle buses, and the stop scheme of express buses, under the constraints of departure interval, passenger volume, vehicle occupancy, section capacity usage and modal competition. To validate the feasibility of proposed optimization model, an urban-rural bus corridor from Rural Jasmine Ecological Park in Hanjiang District to West Xiangyang Bridge in Songqiao Town was chosen as a study case, and the operation organization scheme of three bus modals were solved by the mathematical solver Lingo using mixed integer programming. Meanwhile, the scheme differences under different subobjective weights were discussed, taking into account the uncertain passenger demand and operation supply. The result shows that the increase of passenger trip cost weight will increase the departure frequency of all three bus modals, while the increase of operation cost weight will decrease the total operating vehicles, but the stop frequency at middle stations for express buses will increase accordingly. Compared to the current operation scheme, the optimized scheme can greatly enhance the section capacity usage at a slight sacrifice of passenger trip time with fewer bus vehicles.
| S-EPMC11460678 | biostudies-literature
Project description:BackgroundMachine Learning (ML) has demonstrated its great potential on medical data analysis. Large datasets collected from diverse sources and settings are essential for ML models in healthcare to achieve better accuracy and generalizability. Sharing data across different healthcare institutions or jurisdictions is challenging because of complex and varying privacy and regulatory requirements. Hence, it is hard but crucial to allow multiple parties to collaboratively train an ML model leveraging the private datasets available at each party without the need for direct sharing of those datasets or compromising the privacy of the datasets through collaboration.MethodsIn this paper, we address this challenge by proposing Decentralized, Collaborative, and Privacy-preserving ML for Multi-Hospital Data (DeCaPH). This framework offers the following key benefits: (1) it allows different parties to collaboratively train an ML model without transferring their private datasets (i.e., no data centralization); (2) it safeguards patients' privacy by limiting the potential privacy leakage arising from any contents shared across the parties during the training process; and (3) it facilitates the ML model training without relying on a centralized party/server.FindingsWe demonstrate the generalizability and power of DeCaPH on three distinct tasks using real-world distributed medical datasets: patient mortality prediction using electronic health records, cell-type classification using single-cell human genomes, and pathology identification using chest radiology images. The ML models trained with DeCaPH framework have less than 3.2% drop in model performance comparing to those trained by the non-privacy-preserving collaborative framework. Meanwhile, the average vulnerability to privacy attacks of the models trained with DeCaPH decreased by up to 16%. In addition, models trained with our DeCaPH framework achieve better performance than those models trained solely with the private datasets from individual parties without collaboration and those trained with the previous privacy-preserving collaborative training framework under the same privacy guarantee by up to 70% and 18.2% respectively.InterpretationWe demonstrate that the ML models trained with DeCaPH framework have an improved utility-privacy trade-off, showing DeCaPH enables the models to have good performance while preserving the privacy of the training data points. In addition, the ML models trained with DeCaPH framework in general outperform those trained solely with the private datasets from individual parties, showing that DeCaPH enhances the model generalizability.FundingThis work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC, RGPIN-2020-06189 and DGECR-2020-00294), Canadian Institute for Advanced Research (CIFAR) AI Catalyst Grants, CIFAR AI Chair programs, Temerty Professor of AI Research and Education in Medicine, University of Toronto, Amazon, Apple, DARPA through the GARD project, Intel, Meta, the Ontario Early Researcher Award, and the Sloan Foundation. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute.
| S-EPMC10884342 | biostudies-literature
Project description:Abstract The National Microbiome Data Collaborative (NMDC) Data Portal (https://data.microbiomedata.org) supports microbiome multi-omics data exploration and access through an integrated, distributed data framework aligned with the FAIR (Findable, Accessible, Interoperable and Reusable) data principles (1). The NMDC Data Portal currently hosts 10.2 terabytes of multi-omics microbiome data, spanning five data types (metagenomes, metatranscriptomes, metaproteomes, metabolomes, and natural organic matter characterizations), generated at two Department of Energy User Facilities, the Joint Genome Institute (JGI) at Lawrence Berkeley National Laboratory (LBNL) and the Environmental Molecular Systems Laboratory (EMSL) at Pacific Northwest National Laboratory (PNNL). A flexible data schema (https://github.com/microbiomedata/nmdc-schema) leveraging community-driven standards underpins how data is managed and integrated. Annotated multi-omic data products are produced by the NMDC workflows and linked through common biosamples to enable search capabilities based on environmental context, instrumentation, and functional attributes. As a pilot system, the NMDC Data Portal offers download capabilities and several search components, including interactive geographic visualization of samples; environmental classification distribution visualized through an interactive Sankey diagram; time-series slider to select longitudinal samples of interest; and an upset plot displaying the number of multi-omics data generated from the same biosample within a study.
| S-EPMC8958897 | biostudies-literature
Project description:ObjectiveWe sought to several develop parsimonious machine learning models to predict resource utilization and clinical outcomes following cardiac operations using only preoperative factors.MethodsAll patients undergoing coronary artery bypass grafting and/or valve operations were identified in the 2015-2021 University of California Cardiac Surgery Consortium repository. The primary end point of the study was length of stay (LOS). Secondary endpoints included 30-day mortality, acute kidney injury, reoperation, postoperative blood transfusion and duration of intensive care unit admission (ICU LOS). Linear regression, gradient boosted machines, random forest, extreme gradient boosting predictive models were developed. The coefficient of determination and area under the receiver operating characteristic (AUC) were used to compare models. Important predictors of increased resource use were identified using SHapley summary plots.ResultsCompared with all other modeling strategies, gradient boosted machines demonstrated the greatest performance in the prediction of LOS (coefficient of determination, 0.42), ICU LOS (coefficient of determination, 0.23) and 30-day mortality (AUC, 0.69). Advancing age, reduced hematocrit, and multiple-valve procedures were associated with increased LOS and ICU LOS. Furthermore, the gradient boosted machine model best predicted acute kidney injury (AUC, 0.76), whereas random forest exhibited greatest discrimination in the prediction of postoperative transfusion (AUC, 0.73). We observed no difference in performance between modeling strategies for reoperation (AUC, 0.80).ConclusionsOur findings affirm the utility of machine learning in the estimation of resource use and clinical outcomes following cardiac operations. We identified several risk factors associated with increased resource use, which may be used to guide case scheduling in times of limited hospital capacity.
| S-EPMC9510828 | biostudies-literature
Project description:The pharmaceutical industry has increasingly adopted model-informed drug discovery and development (MID3) to enhance productivity in drug discovery and development. Quantitative systems pharmacology (QSP), which integrates drug action mechanisms and disease complexities to predict clinical endpoints and biomarkers is central to MID3. QSP modeling has proven successful in metabolic and cardiovascular diseases and has expanded into oncology, immunotherapy, and infectious diseases. Despite its benefits, QSP model validation through clinical trial simulations using virtual patients (VPs) is challenging because of parameter variability and high computational costs. To address these challenges, this study proposes a hybrid method that combines Bayesian optimization with machine learning for efficient parameter screening. Our approach achieved an acceptance rate of 27.5% in QSP simulations, which is in sharp contrast with the 2.5% rate of conventional random search methods, indicating more than 10-fold improvement in efficiency. By facilitating faster and more diverse VPs generation, this method promises to advance clinical trial simulations and accelerate drug development in pharmaceutical research.
| S-EPMC11919265 | biostudies-literature