Google Scholar. " />

Publications

* denotes equal contribution. An up-to-date list is available on Google Scholar.

2023

  1. High-throughput Generative Inference of Large Language Models with a Single GPU
    Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, and 10 more authors
  2. Is This Loss Informative? Speeding Up Textual Inversion with Deterministic Objective Evaluation
    Anton Voronov*, Mikhail Khoroshikh*, Artem Babenko, and Max Ryabinin*
    ArXiv
  3. SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
    Max Ryabinin*, Tim Dettmers*, Michael Diskin, and Alexander Borzunov
    ArXiv

2022

  1. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
    Teven Le Scao, Angela Fan, Christopher Akiki, Elizabeth-Jane Pavlick, and 386 more authors
    ArXiv
  2. WBRC
    Petals: Collaborative Inference and Fine-tuning of Large Models
    Alexander Borzunov*, Dmitry Baranchuk*, Tim Dettmers*, Max Ryabinin*, and 4 more authors
    NeurIPS Workshop on Broadening Research Collaborations
  3. RuCoLA: Russian Corpus of Linguistic Acceptability
    Vladislav Mikhailov*, Tatiana Shamardina*, Max Ryabinin*, Alena Pestova, and 2 more authors
    Empirical Methods in Natural Language Processing (EMNLP)
  4. Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees
    Aleksandr Beznosikov, Peter Richtárik, Michael Diskin, Max Ryabinin, and 1 more author
    Neural Information Processing Systems
  5. Secure Distributed Training at Scale
    Eduard Gorbunov*, Alexander Borzunov*, Michael Diskin, and Max Ryabinin
    International Conference on Machine Learning
  6. NeurIPS Demo
    Training Transformers Together
    Alexander Borzunov*, Max Ryabinin*, Tim Dettmers*, Quentin Lhoest*, and 4 more authors
    NeurIPS 2021 Competitions and Demonstrations Track

2021

  1. Scaling Ensemble Distribution Distillation to Many Classes with Proxy Targets
    Max Ryabinin*, Andrey Malinin*, and Mark Gales
    Neural Information Processing Systems
  2. Distributed Deep Learning In Open Collaborations
    Michael Diskin*, Alexey Bukhtiyarov*, Max Ryabinin*, Lucile Saulnier, and 12 more authors
    Neural Information Processing Systems
  3. ACL Findings
    It’s All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning
    Alexey Tikhonov*, and Max Ryabinin*
    Findings of the Association for Computational Linguistics
  4. Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices
    Max Ryabinin*, Eduard Gorbunov*, Vsevolod Plokhotnyuk, and Gennady Pekhimenko
    Neural Information Processing Systems

2020

  1. Embedding Words in Non-Vector Space with Unsupervised Graph Learning
    Max Ryabinin, Sergei Popov, Liudmila Prokhorenkova, and Elena Voita
    Empirical Methods in Natural Language Processing (EMNLP)
  2. Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
    Max Ryabinin, and Anton Gusev
    Neural Information Processing Systems