Uncaught (in promise) Error: Unexpected close tag ; Chrome User-Agent?

I am currently seeing inconsistent behavior across browsers using rss-parser.

I am testing with this RSS feed: http://www.marketwatch.com/rss/topstories

I do get the same error for all RSS feeds I have tried on the same domain.

Here is the relevant code, Vue-flavored:

const CORS_PROXY = 'https://cors-anywhere.herokuapp.com/'

export default {
...
  mounted() {
    require('../../node_modules/rss-parser/dist/rss-parser.js')
    this.getFeeds()
  },

  methods: {
    async getFeeds() {
      const parser = new RSSParser({
        headers: {
          'User-Agent': 'rss-parser',
          'Cache-Control': 'no-cache',
          'Pragma': 'no-cache'
        },
        defaultRSS: 2.0,
        xml2js: {
          strict: true
        }
      })
      const feedRequests = this.$page.frontmatter.feeds.map(feed => {
        return parser.parseURL(CORS_PROXY + feed)
      })
      this.feeds = await Promise.all(feedRequests)
    }
  }
}

In Firefox, this will make a request that returns a valid RSS XML response; in Chrome, this is returning HTML, which makes rss-parser return the following error:

Uncaught (in promise) Error: Unexpected close tag
Line: 9
Column: 7
Char: >
    at error (rss-parser.js?e48f:12624)
    at strictFail (rss-parser.js?e48f:12648)
    at closeTag (rss-parser.js?e48f:12834)
    at SAXParser.write (rss-parser.js?e48f:13397)
    at Parser.parseString (rss-parser.js?e48f:11945)
    at Parser.eval [as parseString] (rss-parser.js?e48f:11620)
    at eval (rss-parser.js?e48f:8448)
    at new Promise (<anonymous>)
    at Parser.parseString (rss-parser.js?e48f:8447)
    at exports.IncomingMessage.eval (rss-parser.js?e48f:8520)

I can change all of the header attributes using the headers option, but User-Agent is the only one that will not change on Chrome (Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36). I suspect this has something to do with what I’m seeing.

Is there any way I can work around this? I will provide more information if necessary.

[rbren]

Yes, this is an issue with the folks serving the RSS feeds. They see that the request is coming from a browser (by checking the User-Agent header) and decide to show an HTML page instead of RSS. Unfortunately you can’t change the User-Agent header due to browser security.

The only solution here is to create a proxy that lets you set the User-Agent header, e.g. by setting an X-Fake-User-Agent header.

[seevee]

Thanks for the reply @rbren, I figured as much.

At least this serves as a good example of why User Agent sniffing is nasty business.