Complex materials science problems such as glass formation must consider large system sizes that are many orders of magnitude too large to be solved by first-principles calculations. The successful applica-tion of machine learning (ML) in various other fields suggests that ML could be useful to address complex problems in materials science. To test its efficacy, we attempt to predict bulk metallic glass formation using ML. Surprisingly, we find that a recently developed ML model based on 201 alloy features con-structed using simple combinations of 31 elemental features is indistinguishable from models that are based on unphysical features. The 201ML-model performs better than the unphysical model only when significant separation of training and testing data is achieved. However, it performs significantly worse than a human-learning based three-feature model. The limited performance of the 201ML-model origi-nates from the inability to accurately represent alloy features through elemental features, showing that physical insights about mixing behavior are required to develop predictable ML models.(c) 2022 The Authors. Published by Elsevier Ltd on behalf of Acta Materialia Inc. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )