Constructing a metadata knowledge graph as an atlas for demystifying AI pipeline optimization.

ABSTRACT: The emergence of advanced artificial intelligence (AI) models has driven the development of frameworks and approaches that focus on automating model training and hyperparameter tuning of end-to-end AI pipelines. However, other crucial stages of these pipelines such as dataset selection, feature engineering, and model optimization for deployment have received less attention. Improving efficiency of end-to-end AI pipelines requires metadata of past executions of AI pipelines and all their stages. Regenerating metadata history by re-executing existing AI pipelines is computationally challenging and impractical. To address this issue, we propose to source AI pipeline metadata from open-source platforms such as Papers-with-Code, OpenML, and Hugging Face. However, integrating and unifying the va

SUBMITTER: Venkataramanan R 

PROVIDER: S-EPMC11748301 | biostudies-literature | 2024

REPOSITORIES: biostudies-literature