Yaming GUO

2025

1. ICML

   

   Towards Lifelong Model Editing via Simulating Ideal Editor

   Yaming Guo, Siyang Guo, Hengshu Zhu, and 1 more author

   In Proceedings of the 42nd International Conference on Machine Learning, 13–19 jul 2025

   Abs Bib HTML PDF Code

   Model editing plays a crucial role in the cost-effective development of large language models, and the challenge of evolving knowledge facilitates its sequential extension, namely lifelong model editing. However, progress on standard and lifelong editing has historically followed separate tracks, overlooking the potential of generalizing standard methods to lifelong scenarios. By establishing this bridge, we can provide robust baselines in lifelong scenarios and ensure that lifelong editing benefits from the ongoing advancements in standard editing technologies. In response, this paper proposes a general framework, Simulating Ideal Editor (SimIE), which restores the strong performance of parameter-modifying methods from standard model editing in a lifelong context. SimIE formulates the ideal parameter shift as the minimum-norm solution to a linear system, constructed using the Moore-Penrose inverse, and subsequently enables recursive updates by truncating the limiting expression of the Moore-Penrose inverse under two mild assumptions. Theoretically, we demonstrate that if either assumption is not met, the solution provided by SimIE remains near-optimal in a statistical sense or stable against perturbations introduced by the sequential editing, but a trade-off between optimality and stability arises when both assumptions fail. Extensive experiments validate the effectiveness of SimIE, which allows standard algorithms to achieve performance comparable to specialized lifelong model editing methods. Our code is available at https://github.com/YamingGuo98/SimIE.

   @inproceedings{pmlr-v267-guo25c,
     title = {Towards Lifelong Model Editing via Simulating Ideal Editor},
     author = {Guo, Yaming and Guo, Siyang and Zhu, Hengshu and Sun, Ying},
     booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
     pages = {20793--20824},
     year = {2025},
     editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry},
     volume = {267},
     series = {Proceedings of Machine Learning Research},
     month = {13--19 Jul},
     publisher = {PMLR},
     url = {https://proceedings.mlr.press/v267/guo25c.html},
   }

2024

1. NeurIPS

   

   OT4P: Unlocking Effective Orthogonal Group Path for Permutation Relaxation

   Yaming Guo, Chen Zhu, Hengshu Zhu, and 1 more author

   In Advances in Neural Information Processing Systems, 2024

   Abs DOI Bib HTML PDF Code

   Optimization over permutations is typically an NP-hard problem that arises extensively in ranking, matching, tracking, etc. Birkhoff polytope-based relaxation methods have made significant advancements, particularly in penalty-free optimization and probabilistic inference. Relaxation onto the orthogonal group offers unique potential advantages such as a lower representation dimension and preservation of inner products; however, equally effective approaches remain unexplored. To bridge the gap, we present a temperature-controlled differentiable transformation that maps unconstrained vector space to the orthogonal group, where the temperature, in the limit, concentrates orthogonal matrices near permutation matrices. This transformation naturally implements a parameterization for the relaxation of permutation matrices, allowing for gradient-based optimization of problems involving permutations. Additionally, by deriving a re-parameterized gradient estimator, this transformation also provides efficient stochastic optimization over the latent permutations. Extensive experiments involving the optimization over permutation matrices validate the effectiveness of the proposed method.

   @inproceedings{NEURIPS2024_d3ee2816,
     title = {OT4P: Unlocking Effective Orthogonal Group Path for Permutation Relaxation},
     author = {Guo, Yaming and Zhu, Chen and Zhu, Hengshu and Wu, Tieru},
     booktitle = {Advances in Neural Information Processing Systems},
     doi = {10.52202/079017-3709},
     editor = {Globerson, A. and Mackey, L. and Belgrave, D. and Fan, A. and Paquet, U. and Tomczak, J. and Zhang, C.},
     volume = {37},
     pages = {116838--116869},
     publisher = {Curran Associates, Inc.},
     year = {2024},
     url = {https://proceedings.neurips.cc/paper_files/paper/2024/file/d3ee2816ae19c9e689c3352397c93a22-Paper-Conference.pdf},
   }

2023

1. ICML

   

   Out-of-Distribution Generalization of Federated Learning via Implicit Invariant Relationships

   Yaming Guo, Kai Guo, Xiaofeng Cao, and 2 more authors

   In Proceedings of the 40th International Conference on Machine Learning, 23–29 jul 2023

   Abs Bib HTML PDF Code

   Out-of-distribution generalization is challenging for non-participating clients of federated learning under distribution shifts. A proven strategy is to explore those invariant relationships between input and target variables, working equally well for non-participating clients. However, learning invariant relationships is often in an explicit manner from data, representation, and distribution, which violates the federated principles of privacy-preserving and limited communication. In this paper, we propose FedIIR, which implicitly learns invariant relationships from parameter for out-of-distribution generalization, adhering to the above principles. Specifically, we utilize the prediction disagreement to quantify invariant relationships and implicitly reduce it through inter-client gradient alignment. Theoretically, we demonstrate the range of non-participating clients to which FedIIR is expected to generalize and present the convergence results for FedIIR in the massively distributed with limited communication. Extensive experiments show that FedIIR significantly outperforms relevant baselines in terms of out-of-distribution generalization of federated learning.

   @inproceedings{pmlr-v202-guo23b,
     title = {Out-of-Distribution Generalization of Federated Learning via Implicit Invariant Relationships},
     author = {Guo, Yaming and Guo, Kai and Cao, Xiaofeng and Wu, Tieru and Chang, Yi},
     booktitle = {Proceedings of the 40th International Conference on Machine Learning},
     pages = {11905--11933},
     year = {2023},
     editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan},
     volume = {202},
     series = {Proceedings of Machine Learning Research},
     month = {23--29 Jul},
     publisher = {PMLR},
     url = {https://proceedings.mlr.press/v202/guo23b.html},
   }