DEV Community

64BitDev
64BitDev

Posted on • Originally published at linkedin.com

Robots Exclusion Protocol finally penned

#website #robot #webcrawler #html

After 25years out in field and available in 500M+ websites, Robots Exclusion Protocol finally drafted to added to #IETF- https://www.ietf.org/id/draft-rep-wg-topic-00.txt

Top comments (0)

Read next

falselight profile image

Currency Plugins/Widgets for WordPress

YURII DE. -

abhishekgurjar profile image

Build a Flip Coin Website

Abhishek Gurjar -

abhishekgurjar profile image

Build a Pomodoro Timer Website

Abhishek Gurjar -

mubeen780 profile image

Create a Static Website with Amazon Simple Storage Service (Amazon S3) with Public Access

Mubeen Babar -