Discussion about this post

User's avatar
Mahir Abbas's avatar

SWE-bench is not a good benchmark. It's been contaminated as well. DeepSWE is much better, although it hasn't been run on a lot of models yet.

No posts

Ready for more?