good question

[Agent]

It removes control over my site and presentation which I enjoy having.

People will not see the amount of traffic hitting their gallery pages. They will only see their click throughs in the sponsor aff. area. The URI string may not show accurately in some their stats as the refferer because of your page's (unmasked) dynamic nature. The webmasters will not get SE value which a URI hardlink to their gallery provides. If the sponsor goes under/fucks affiliates over and the webmaster wants to pull links as far as I know they can't unless you do it for them. You'll get hit by autosubmitters eventually. The shitheads who blindly autosubmit will see you hotlinking might fire off some nasty emails your way.

The consistant presentation is a good idea, some people will like that. But when you throw other people's resources into the mix I dont see it flying well.