Dears, I had included a redirect to a robots-txt file and it worked fine. Had to replace the file by a new version and now it is not found anymore (get a 404). I followed the same steps and the file name and location should be fine (see enclosed print screens). Any idea what is happening?
thanks!
kind regards
Mark
Hi! This is a bit strange … try, to use the full url in the To field on the redirect rule.
Old To value → /images/0/5592125/robots.txt
New To value → https://cdn1.site-media.eu/images/0/5592125/robots.txt
See if it works.
Hope this helps!
Hi Lucian, thanks for your help - I tried t out, but no change unfortunately…
I deleted the redirect for the time being, but hope some info will pop-up here on what could be the cause, as I don’t see any reason for this not working.
kind regards
Mark
Hi @Mark_Fosseprez ! If you want to hide some pages from the search engine, you can use this option: SEO with Sitejet - Sitejet Help
However, this option will just add noindex,nofollow meta into your page header.
<meta name="robots" content="noindex,nofollow" />
It will not add the url to the robots.txt file, and it will not remove the url from the sitemap.xml
P.S. Maybe the redirect has a bug, or something.
Probably your browser “remembered” (cached) the old redirect (301 means “Moved Permanently”), so you are still redirected to the old URL that you probably deleted. This is a browser behavior and not related to Sitejet. Read more in this article
On my end, the link https://www.chantaldujardin.be/robots.txt works properly. Since search engines do not cache in a way your browser does, they will find your updated file. You can probably do as well if you test it in an incognito tab or in a different browser or device.
Thanks @Lucian_Dinu, @AndyG , for your help. Anyd, your suggestions was correct indeed, so problem solved (at least with the redirect for robots.txt - the strange ways of operating of browsers won’t probably change 
).
kind rgerads
Mark