500 Unable To Connect

[cd34]

If I had to guess, it looks like you have reverse dns lookups turned on, their bot/linkchecker possibly doesn't have reverse dns and has a short timeout. Your site looks up the DNS of the visiting bot, takes 27 seconds to time out, their bot/linkchecker times out after 5 or so seconds, and, invalid connection.

You might check your apache config to turn off dns lookups:

HostnameLookups Off

or, check your .htaccess for something like

deny from *.domain.com
or
deny from 12.23.*.*

The latter case doesn't ban just an IP block and forces a reverse DNS lookup for each query since 12.23.domain.com could be valid and would match your rules.

If you want to block just those IPs

deny from 12.23.

would do that.

or

deny from 12.23.0.0/16