DEV Community

64BitDev
64BitDev

Posted on • Originally published at linkedin.com

Robots Exclusion Protocol finally penned

#website #robot #webcrawler #html

After 25years out in field and available in 500M+ websites, Robots Exclusion Protocol finally drafted to added to #IETF- https://www.ietf.org/id/draft-rep-wg-topic-00.txt

Top comments (0)

Read next

shuhikari profile image

6 Coisas que você precisa saber ao escrever commits

Édipo Hikari -

priya2422 profile image

Navigating the Directed Maze: Finding the Minimum Spanning Tree

Priya Pandey -

mrdprasad profile image

How to Deploy a Static Website on GitHub Pages: A Step-by-Step Guide

MrDPrasad -

sandr0p profile image

Structuring Your Space: Designing a Visual Grid

sandr0-p -