Cannot use http proxy with http puppeteer_download_host, proxy return 502.

Steps to reproduce

Tell us about your environment:

• Puppeteer version: puppeteer@1.14.0
• Platform / OS version: mac OS 10.14.4 (18E226)
• URLs (if applicable):
• Node.js version: v8.11.3

What steps will reproduce the problem?

Please include code that reproduces the issue.

1.setup a http puppeteer_download_host in .npmrc (say: puppeteer_download_host=http://xxx.com/mirrors) 2.config http_proxy to npm: http_proxy=http://127.0.0.1:xxx 3.npm i puppeteer

What is the expected result? puppeteer download and install chrome.

What happens instead?

ERROR: Failed to download Chromium r641577! Set "PUPPETEER_SKIP_CHROMIUM_DOWNLOAD" env variable to skip download.
Error: Download failed: server returned code 502. URL: http://xxxx.com/mirrors/chromium-browser-snapshots/Mac/641577/chrome-mac.zip
    at response (/Users/unightsun/workspace/git/MobileAssistCrawler/crawl-page/MarketPageAnalysisNode/node_modules/puppeteer/lib/BrowserFetcher.js:232:21)
    at ClientRequest.requestCallback (/Users/unightsun/workspace/git/MobileAssistCrawler/crawl-page/MarketPageAnalysisNode/node_modules/puppeteer/lib/BrowserFetcher.js:288:7)
    at Object.onceWrapper (events.js:315:30)
    at emitOne (events.js:116:13)
    at ClientRequest.emit (events.js:211:7)
    at HTTPParser.parserOnIncomingClient (_http_client.js:543:21)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:112:17)
    at Socket.socketOnData (_http_client.js:440:20)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
  -- ASYNC --
    at BrowserFetcher.<anonymous> (/Users/unightsun/workspace/git/MobileAssistCrawler/crawl-page/MarketPageAnalysisNode/node_modules/puppeteer/lib/helper.js:110:27)
    at Object.<anonymous> (/Users/unightsun/workspace/git/MobileAssistCrawler/crawl-page/MarketPageAnalysisNode/node_modules/puppeteer/install.js:64:16)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:191:16)
    at bootstrap_node.js:612:3

I chase the code across install.js, found that puppeteer use ‘https-proxy-agent’ as default ProxyAgent, but it use 443 as it’s defaultPort: /https-proxy-agent/index.js:59:

 this.defaultPort = 443;

_http_client.js:135:

  var defaultPort = options.defaultPort ||
                    this.agent && this.agent.defaultPort;
  var port = options.port = options.port || defaultPort || 80;

/puppeteer/lib/BrowserFetcher.js:269:
function httpRequest gives neither defaultPort or port.

so proxy use 443 port for http url:

CONNECT xxx.com:443 HTTP/1.1
Host: xxx.com:443
Connection: close

I must specified port in url to make it works:

puppeteer_download_host=http://xxx.com:80/mirrors

so consider specified port in BrowserFetcher.js:273:

+++ options.port = options.port || (options.protocol === 'https:' ? 443 : 80);

[hiroppy]

@aslushnikov currently, google is down. I think the cause is this. fyi https://downdetector.com/status/google

So, maybe not the same issue… sorry.

[stale[bot]]

We are closing this issue. If the issue still persists in the latest version of Puppeteer, please reopen the issue and update the description. We will try our best to accomodate it!