maam@feddit.uk to Linux@programming.dev · 4 months agoLinux traffic has grown 22.4% in PH this yearfeddit.ukimagemessage-square41linkfedilinkarrow-up15arrow-down10
arrow-up15arrow-down1imageLinux traffic has grown 22.4% in PH this yearfeddit.ukmaam@feddit.uk to Linux@programming.dev · 4 months agomessage-square41linkfedilink
minus-squaremaj@piefed.sociallinkfedilinkEnglisharrow-up0·4 months agoAI agents and web scrapers surely
minus-squarerealitaetsverlust@piefed.ziplinkfedilinkEnglisharrow-up1·4 months agoI would expect AI scrapers to fake a windows user-agent tbh.
minus-squareWhelks_chance@lemmy.worldlinkfedilinkarrow-up0·4 months agoWhy? Isn’t it likely that it’ll just report e.g. “python-requests” for the user agent, and it’s up to the server side to decide what that means?
minus-squarerealitaetsverlust@piefed.ziplinkfedilinkEnglisharrow-up1·4 months agoThat user agent would make them easily identifiable and therefore blockable. It’s more likely that they are trying to hide as a legitimate user.
AI agents and web scrapers surely
I would expect AI scrapers to fake a windows user-agent tbh.
Why? Isn’t it likely that it’ll just report e.g. “python-requests” for the user agent, and it’s up to the server side to decide what that means?
That user agent would make them easily identifiable and therefore blockable. It’s more likely that they are trying to hide as a legitimate user.