Bot/Scraping/Crawler Traffic on Sports-Reference.com Sites

April 26, 2022

Sports Reference is primarily dependent on ad revenue, so we must ensure that actual people using web browsers have the best possible experience when using this site. Unfortunately, non-human traffic, ie bots, crawlers, scrapers, can overwhelm our servers with the number of requests they send us in a short amount of time. Therefore we are implementing rate limiting on the site. We will attempt to keep this page up to date with our current settings.

Currently we will block users sending requests to:

• FBref.com faster than one request every three seconds.
• Baseball-Reference.com faster than one request every second.

If you find yourself blocked, you should just wait a half day and then have access returned after a modest period of time.

Why Not Provide an API?

Most of our data comes from third parties who sell the data to us. As part of our agreements with them we can not provide the data available as a download on our site. We are aware that an API would mitigate some issues, but it's not our business model. If you want to get data for a low price, we suggest NatStat.com.

For more information on data use of SR sites, see our Data Use page.