title: README
emoji: 💻
colorFrom: indigo
colorTo: purple
sdk: static
pinned: false
Model/Data associated with Paper:
Training Software Engineering Agents and Verifiers with SWE-Gym
Jiayi Pan*,1, Xingyao Wang*,2, Graham Neubig3, Navdeep Jaitly4, Heng Ji2, Alane Suhr^,1, Yizhe Zhang^,4
1UC Berkeley, 2UIUC, 3CMU, 4Apple
*Equal contribution, ^Equal supervision
💻 Code • 📃 Paper • 🤗 Data & Models
We present SWE-Gym, the first environment for training real-world software engineering agents. We use it to train strong LM agents that achieve state-of-the-art open results on SWE-Bench, with early, promising scaling characteristics as we increase training and inference-time compute.
Progress in agents for software engineering has been limited by the lack of training environments that both include rigorous verification for reinforcement learning and cover the expansive tasks encountered in real-world repository-level engineering.
We introduce SWE-Gym: An Open Environment for Training Software Engineering Agents & Verifiers. Our baselines achieve new open SOTA - 32%/26% on SWE-Bench Verified/Lite, with promising scaling trends.
SWE-Gym enables scalable improvements for software engineering agents at both training and inference time. Our current results is primarity bottlenecked by training and inference compute, rather than the size of our environment.
Reproducing Results
Please refer to our Github Repo for more details: See docs/OpenHands.md and docs/MoatlessTools.md for instructions on reproducing results with our training and inference-time results for OpenHands and MoatlessTools agents.
📚 Citation
@misc{pan2024swegym,
title={Training Software Engineering Agents and Verifiers with {SWE-Gym}},
author={Pan, Jiayi and Wang, Xingyao and Neubig, Graham and Jaitly, Navdeep and Ji, Heng and Suhr, Alane and Zhang, Yizhe},
year={2024},
}