add steps to disallow pages with robots.txt#3370

oraNod wants to merge 1 commit intoansible:develansible/ansible-documentation:develfrom

oraNod:add-archiving-infooraNod/ansible-documentation:add-archiving-infoCopy head branch name to clipboard