{"database":"biostudies-literature","file_versions":[],"scores":null,"additional":{"omics_type":["Unknown"],"volume":["PP"],"submitter":["Dai B"],"funding":["NSF","NIA NIH HHS","The Chinese University of Hong Kong Faulty of Science Direct Grant","NIH","NIGMS NIH HHS"],"pubmed_abstract":["An exciting recent development is the uptake of deep neural networks in many scientific fields, where the main objective is outcome prediction with a black-box nature. Significance testing is promising to address the black-box issue and explore novel scientific insights and interpretations of the decision-making process based on a deep learning model. However, testing for a neural network poses a challenge because of its black-box nature and unknown limiting distributions of parameter estimates while existing methods require strong assumptions or excessive computation. In this article, we derive one-split and two-split tests relaxing the assumptions and computational complexity of existing black-box tests and extending to examine the significance of a collection of features of interest in a dataset of possibly a complex type, such as an image. The one-split test estimates and evaluates a black-box model based on estimation and inference subsets through sample splitting and data perturbation. The two-split test further splits the inference subset into two but requires no perturbation. Also, we develop their combined versions by aggregating the p -values based on repeated sample splitting. By deflating the bias-sd-ratio, we establish asymptotic null distributions of the test statistics and the consistency in terms of Type 2 error. Numerically, we demonstrate the utility of the proposed tests on seven simulated examples and six real datasets. Accompanying this article is our python library dnn-inference (https://dnn-inference.readthedocs.io/en/latest/) that implements the proposed tests."],"journal":["IEEE transactions on neural networks and learning systems"],"full_dataset_link":["https://www.ebi.ac.uk/biostudies/studies/S-EPMC10915654"],"repository":["biostudies-literature"],"pubmed_title":["Significance Tests of Feature Relevance for a Black-Box Learner."],"pmcid":["PMC10915654"],"funding_grant_id":["DMS-1952539","R01AG065636","R01AG074858","DMS-1721216","R01 AG074858","R01AG069895","DMS-1712564","R01 AG065636","R01 AG069895","R01GM126002","U01 AG073079","U01AG073079","R01 GM126002"],"pubmed_authors":["Pan W","Dai B","Shen X"],"additional_accession":[]},"is_claimable":false,"name":"Significance Tests of Feature Relevance for a Black-Box Learner.","description":"An exciting recent development is the uptake of deep neural networks in many scientific fields, where the main objective is outcome prediction with a black-box nature. Significance testing is promising to address the black-box issue and explore novel scientific insights and interpretations of the decision-making process based on a deep learning model. However, testing for a neural network poses a challenge because of its black-box nature and unknown limiting distributions of parameter estimates while existing methods require strong assumptions or excessive computation. In this article, we derive one-split and two-split tests relaxing the assumptions and computational complexity of existing black-box tests and extending to examine the significance of a collection of features of interest in a dataset of possibly a complex type, such as an image. The one-split test estimates and evaluates a black-box model based on estimation and inference subsets through sample splitting and data perturbation. The two-split test further splits the inference subset into two but requires no perturbation. Also, we develop their combined versions by aggregating the p -values based on repeated sample splitting. By deflating the bias-sd-ratio, we establish asymptotic null distributions of the test statistics and the consistency in terms of Type 2 error. Numerically, we demonstrate the utility of the proposed tests on seven simulated examples and six real datasets. Accompanying this article is our python library dnn-inference (https://dnn-inference.readthedocs.io/en/latest/) that implements the proposed tests.","dates":{"release":"2022-01-01T00:00:00Z","publication":"2022 Jun","modification":"2024-11-13T19:16:30.287Z","creation":"2024-11-13T19:16:30.287Z"},"accession":"S-EPMC10915654","cross_references":{"pubmed":["35771783"],"doi":["10.1109/TNNLS.2022.3185742"]}}