2023 ICML High-throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, and 10 more authors HTML PDF ArXiv Is This Loss Informative? Speeding Up Textual Inversion with Deterministic Objective Evaluation Anton Voronov*, Mikhail Khoroshikh*, Artem Babenko, and Max Ryabinin* ArXiv HTML PDF ICML SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin*, Tim Dettmers*, Michael Diskin, and Alexander Borzunov ArXiv HTML PDF 2022 ArXiv BLOOM: A 176B-Parameter Open-Access Multilingual Language Model Teven Le Scao, Angela Fan, Christopher Akiki, Elizabeth-Jane Pavlick, and 386 more authors ArXiv HTML PDF WBRC Petals: Collaborative Inference and Fine-tuning of Large Models Alexander Borzunov*, Dmitry Baranchuk*, Tim Dettmers*, Max Ryabinin*, and 4 more authors NeurIPS Workshop on Broadening Research Collaborations HTML PDF EMNLP RuCoLA: Russian Corpus of Linguistic Acceptability Vladislav Mikhailov*, Tatiana Shamardina*, Max Ryabinin*, Alena Pestova, and 2 more authors Empirical Methods in Natural Language Processing (EMNLP) HTML PDF NeurIPS Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees Aleksandr Beznosikov, Peter Richtárik, Michael Diskin, Max Ryabinin, and 1 more author Neural Information Processing Systems HTML PDF ICML Secure Distributed Training at Scale Eduard Gorbunov*, Alexander Borzunov*, Michael Diskin, and Max Ryabinin International Conference on Machine Learning HTML PDF NeurIPS Demo Training Transformers Together Alexander Borzunov*, Max Ryabinin*, Tim Dettmers*, Quentin Lhoest*, and 4 more authors NeurIPS 2021 Competitions and Demonstrations Track HTML PDF 2021 NeurIPS Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets Max Ryabinin*, Andrey Malinin*, and Mark Gales Neural Information Processing Systems HTML PDF NeurIPS Distributed Deep Learning In Open Collaborations Michael Diskin*, Alexey Bukhtiyarov*, Max Ryabinin*, Lucile Saulnier, and 12 more authors Neural Information Processing Systems HTML PDF ACL Findings It’s All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning Alexey Tikhonov*, and Max Ryabinin* Findings of the Association for Computational Linguistics HTML PDF NeurIPS Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin*, Eduard Gorbunov*, Vsevolod Plokhotnyuk, and Gennady Pekhimenko Neural Information Processing Systems HTML PDF 2020 EMNLP Embedding Words in Non-Vector Space with Unsupervised Graph Learning Max Ryabinin, Sergei Popov, Liudmila Prokhorenkova, and Elena Voita Empirical Methods in Natural Language Processing (EMNLP) HTML PDF NeurIPS Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts Max Ryabinin, and Anton Gusev Neural Information Processing Systems HTML PDF