Alexander von Rohr

selected publications

1. NeurIPS_2021

   Local policy search with Bayesian optimization

   Sarah Müller*, Alexander von Rohr*, and Sebastian Trimpe

   In Advances in Neural Information Processing Systems 2021

   Abs arXiv Bib HTML PDF Code

   Reinforcement learning (RL) aims to find an optimal policy by interaction with an environment. Consequently, learning complex behavior requires a vast number of samples, which can be prohibitive in practice. Nevertheless, instead of systematically reasoning and actively choosing informative samples, policy gradients for local search are often obtained from random perturbations. These random samples yield high variance estimates and hence are sub-optimal in terms of sample complexity. Actively selecting informative samples is at the core of Bayesian optimization, which constructs a probabilistic surrogate of the objective from past samples to reason about informative subsequent ones. In this paper, we propose to join both worlds. We develop an algorithm utilizing a probabilistic model of the objective function and its gradient. Based on the model, the algorithm decides where to query a noisy zeroth-order oracle to improve the gradient estimates. The resulting algorithm is a novel type of policy search method, which we compare to existing black-box algorithms. The comparison reveals improved sample complexity and reduced variance in extensive empirical evaluations on synthetic objectives. Further, we highlight the benefits of active sampling on popular RL benchmarks.

   @inproceedings{mueller2021local,
     author = {M\"{u}ller*, Sarah and {von Rohr*}, Alexander and Trimpe, Sebastian},
     title = {Local policy search with Bayesian optimization},
     booktitle = {Advances in Neural Information Processing Systems},
     pages = {20708--20720},
     year = {2021},
     editor = {Ranzato, M. and Beygelzimer, A. and Dauphin, Y. and Liang, P.S. and Vaughan, J. Wortman},
     publisher = {Curran Associates, Inc.},
     volume = {34},
   }

2. TMLR

   Discovering Model Structure of Dynamical Systems with Combinatorial Bayesian Optimization

   Lucas Rath, Alexander von Rohr, Andreas Schultze, Sebastian Trimpe, and Burkhard Corves

   Transactions on Machine Learning Research 2024

   Abs Bib HTML Code

   Deciding on a model structure is a fundamental problem in machine learning. In this paper we consider the problem of building a data-based model for dynamical systems from a library of discrete components. In addition to optimizing performance, we consider crash and inequality constraints that arise from additional requirements, such as real-time capability and model complexity. We address this task of model structure selection with a focus on dynamical systems and propose to search over potential model structures efficiently using a constrained combinatorial Bayesian Optimization (BO) algorithm. We propose expressive surrogate models suited for combinatorial domains and an acquisition function that can handle inequality and crash constraints. We provide simulated benchmark problems within the domain of equation discovery of nonlinear dynamical systems. Our method outperforms the state-of-the-art in constrained combinatorial optimization of black-box functions and has a favorable computational overhead compared to other BO methods. As a real-world application example, we apply our method to optimize the configuration of an electric vehicle’s digital twin while ensuring its real-time capability for the use in one of the world’s largest driving simulators.

   @article{rath2024discovering,
     title = {Discovering Model Structure of Dynamical Systems with Combinatorial Bayesian Optimization},
     author = {Rath, Lucas and {von Rohr}, Alexander and Schultze, Andreas and Trimpe, Sebastian and Corves, Burkhard},
     journal = {Transactions on Machine Learning Research},
     issn = {2835-8856},
     year = {2024},
   }