Publications

Learning guarantee of reward modeling using deep neural networks

Published in submited, 2025

In this work, we study the learning theory of reward modeling with pairwise comparison data using deep neural networks. We establish a novel non-asymptotic regret bound for deep reward estimators in a non-parametric setting, which depends explicitly on the network architecture. Furthermore, to underscore the critical importance of clear human beliefs, we introduce a margin-type condition that assumes the conditional winning probability of the optimal action in pairwise comparisons is significantly distanced from 1/2. This condition enables a sharper regret bound, which substantiates the empirical efficiency of Reinforcement Learning from Human Feedback and highlights clear human beliefs in its success. Notably, this improvement stems from high-quality pairwise comparison data implied by the margin-type condition, is independent of the specific estimators used, and thus applies to various learning algorithms and models.

Recommended citation: Luo, Y., Ge, Y., Han, R. & Shen, G. (2025). Learning Guarantee of Reward Modeling Using Deep Neural Networks. arXiv preprint arXiv:2505.06601.

Adaptive debiased SGD in high-dimensional GLMs with streaming data

Published in submitted, 2024

Online statistical inference refers to an inferential method that updates the model parameters as data is sequentially available. This approach has broad applications in network security, quantitative finance, and recommendation systems. In contrast to offline techniques where the entire dataset is used for training, online learning algorithms are anticipated to offer computational efficiency while still delivering statistical results comparable to their offline counterparts.

Recommended citation: Han, R., Luo, L., Luo, Y., Lin, Y., & Huang, J. (2024). Adaptive debiased SGD in high-dimensional GLMs with streaming data. To submit.