2019/09/30 by Guang-He Lee, Tommi Jaakkola, Lee, Guang-He +2 · 1 voice · 1 citation
Computer Science · Materials Science · Mathematics · #Computational Drug Discovery Methods #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Materials Science #Neural Networks and Applications #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1909.13488
openalex publication_date 2019/09/30 · arxiv published 2019/09/30 · arxiv updated 2020/05/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We show how neural models can be used to realize piece-wise constant functions such as decision trees. The proposed architecture, which we call locally constant networks, builds on ReLU networks that are piece-wise linear and hence their associated gradients with respect to the inputs are locally constant. We formally establish the equivalence between the classes of locally constant networks and decision trees. Moreover, we highlight several advantageous properties of locally constant networks, including how they realize decision trees with parameter sharing across branching / leaves. Indeed, only M neurons suffice to implicitly model an oblique decision tree with 2M leaf nodes. The neural representation also enables us to adopt many tools developed for deep networks (e.g., DropConnect (Wan et al., 2013)) while implicitly training decision trees. We demonstrate that our method outperforms alternative techniques for training oblique decision trees in the context of molecular property classification and regression tasks.