2022/11/07 by Rafael Frongillo, Dhamma Kimpara, Frongillo, Rafael +3 · 1 citation
Economics, Econometrics and Finance · #Complex Systems and Time Series Analysis #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sports Analytics and Performance
paper · pdf · doi:10.48550/arxiv.2211.03761
openalex publication_date 2022/11/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We initiate the study of proper losses for evaluating generative models in the discrete setting. Unlike traditional proper losses, we treat both the generative model and the target distribution as black-boxes, only assuming ability to draw i.i.d. samples. We define a loss to be black-box proper if the generative distribution that minimizes expected loss is equal to the target distribution. Using techniques from statistical estimation theory, we give a general construction and characterization of black-box proper losses: they must take a polynomial form, and the number of draws from the model and target distribution must exceed the degree of the polynomial. The characterization rules out a loss whose expectation is the cross-entropy between the target distribution and the model. By extending the construction to arbitrary sampling schemes such as Poisson sampling, however, we show that one can construct such a loss.