question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Why can't I add this HTTP header to my blobs?

See original GitHub issue

I just learned that Googlebot-Image has been indexing images at my site, presumably because a robots.txt entry is only honored for LOCAL images … those served from the site’s domain. Images that are in Azure Storage (served by the Azure CDN) are on a different domain, thus Google indexes the images.

To stop this, Googlebot-Image needs to see this header on the blobs:

X-Robots-Tag: noindex

However, when I try to add this header (as a user-defined header) in CloudBerry Explorer for Azure Blob Storage, the header cannot be added.

So, what’s up with this? Are you preventing me from adding the header? If so, why? If there is no good reason to stop me from adding the header, can you change your policy and allow its use? … or do you have any other ideas?

Issue Analytics

  • State:closed
  • Created 8 years ago
  • Comments:16 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
benaadamscommented, Feb 16, 2016

@GuardRex don’t forget items in the root directory of storage are served from a container called $root which you’ll have to create.

Google Image bot does respect robots.txt https://support.google.com/webmasters/answer/35308?hl=en

0reactions
firephantomassasincommented, Aug 5, 2023

@guardrex i’m also facing the same scenario. I want to add X-Robots-Tag = noindex in the blob storage http header. How did you resolve your issue?

Read more comments on GitHub >

github_iconTop Results From Across the Web

The Specified blob does not exist - when setting http header
The reason you are getting this error is because blobClient.SetHttpHeaders(blobhttpheader); is making an HTTP request to set the blob ...
Read more >
An HTTP header that's mandatory for this request is not ...
Hi all,. I keep getting this error message when trying to put data to Azure Datalake Storage Gen2 through REST API. I think...
Read more >
Setting and retrieving properties and metadata for Blob ...
Containers and blobs support custom metadata, represented as HTTP headers. Metadata headers can be set on a request that creates a new ...
Read more >
Cloud Platform Integration with MS Azure - missing ...
I'm using Azure SAS - Shared Access Signature and PUT parameter - which works when testing with POSTMAN and adding the header key...
Read more >
Http header for assets - English Community - 4D Forum
I wonder what is the most optimal way to add http headers to web assets responses. Just like .htaccess in Apache. For example...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found