unable to get local issuer certificate

Question, Bug, or Feature?
Type: Bug

Enter Task Name: DownloadSecureFileV1 / InstallSSHKeyV0

Environment

• TFS on-premises

  • If using TFS on-premises, provide the version:

About Azure DevOps Server Version Dev17.M153.3

• Agent - Hosted or Private: private Linux agent running Azure Pipelines agent v2.158

Issue Description

Both tasks fail with infamous ‘unable to get local issuer certificate’. server certificate is added to system-wide cert store - agent does not complain when connecting to the queue nor git complains when fetching sources. I established that the source of an error is NodeJS bundled with Azure pipelines agent. For some reason it does not fetch system-wide SSL certificates, it seems to use cert store bundled with NodeJS only. It does not help that Azure Pipelines agent is bundled with NodeJS v6.10.3 - some additional SSL/TLS related configuration options were introduced in 6.11 and more recently in NodeJS v7 and above.

I found few workarounds that do work, but nothing that could last.

1. export NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt export SSL_CERT_DIR=/etc/ssl/certs and run agent interactively with ./run.sh. This option still did not work for agent configured as service despite adding exports above to user profile

2. add the following at the beginning of bin/AgentService.js: process.env.NODE_EXTRA_CA_CERTS=‘/etc/ssl/certs/ca-certificates.crt’; process.env.SSL_CERT_DIR=‘/etc/ssl/certs’;

This will only last as long the agent will not be updated / reinstalled from server.

I am looking for a proper fix.

Task logs

##[debug]Evaluating condition for step: 'Install an SSH key for tfsa'
##[debug]Evaluating: succeeded()
##[debug]Evaluating succeeded:
##[debug]=> True
##[debug]Result: True
##[section]Starting: Install an SSH key for tfsa
==============================================================================
Task         : Install SSH key
Description  : Install an SSH key prior to a build or deployment
Version      : 0.151.2
Author       : Microsoft Corporation
Help         : [More information](https://go.microsoft.com/fwlink/?linkid=875267)
==============================================================================
##[debug]agent.TempDirectory=/home/tfs-build-agent/agent2/_work/_temp
##[debug]loading inputs and endpoints
##[debug]loading INPUT_HOSTNAME
##[debug]loading INPUT_SSHPUBLICKEY
##[debug]loading INPUT_SSHKEYSECUREFILE
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading SECUREFILE_TICKET_7e2886f1-72fd-4665-9534-59f2641a1447
##[debug]loaded 7
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]sshPublicKey=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQ7dWn3DIdrnB5gam7mqu1h+WwcJEggXSzBL/vO61xynyqfG+v8Kzs/N1yMCTQqeUzbU6LqJImljfsrOoXNtM168vCG73WP+g+7pvziRPpLB5WKZlD8qV9wSZZYpxnd+rYI+5ZP84cS8PmimjXAaGKOEbI335rBboAdIpUNvoefJrGmb7+YzUozW3zQMcKLc0YoRc4GTYnZCh3B7AW8i4VTVE5gEXmYincY124uaNy2fOAXOtH2kQck1zGpdu8aasVeLqxhCx6hWPFHabsaJ2f4LCIKxNKmsHTDLrTR0PIB0D9U940h4TPYAyrqi9CByNcauaibUxykvu+UUnSDJ2V [e-mail]
##[debug]hostName=|1|RyX1HGab+gSa29Q99VApWbD5XPE=|ROzy+vq75uqQc+p9/MWisspu3xw= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZeVJ4ejjhE5QgjIz1iB8imXEsYpbVEzVJ2JEEtpG6nahg3mN8CEyv0gtSGt9HoPtiAMPieIcvGLVGmCEfOGJ3utifFrJAysVp2h1zlnt7ttwpF+X+Di30+XUZy7goyBUx2ZnICNKM+aPByQAPs2ohR8hXX0ErM/KsfjrCJ2rOQrkdRmI6OPr+HMudrHiUU7EfhW9zlj10pY1JIJfmuqR/h90zFNYxedkxUJnDh72RuelT+EvN9Y8bY/7KZt61FHYQmx8iE459ULdnXoRr0BhgZ2oH8m+bUlJYDmFpcQ2Dy5BYBqO3WNUyb75uYDU4wLOYXHzgigVoYQc5xsS/6Q/ |1|d/9td4eq+52WDdMl271Y02P+po0=|jtBAGpjXEvJCX6wMAd+aZEl9GYI= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZeVJ4ejjhE5QgjIz1iB8imXEsYpbVEzVJ2JEEtpG6nahg3mN8CEyv0gtSGt9HoPtiAMPieIcvGLVGmCEfOGJ3utifFrJAysVp2h1zlnt7ttwpF+X+Di30+XUZy7goyBUx2ZnICNKM+aPByQAPs2ohR8hXX0ErM/KsfjrCJ2rOQrkdRmI6OPr+HMudrHiUU7EfhW9zlj10pY1JIJfmuqR/h90zFNYxedkxUJnDh72RuelT+EvN9Y8bY/7KZt61FHYQmx8iE459ULdnXoRr0BhgZ2oH8m+bUlJYDmFpcQ2Dy5BYBqO3WNUyb75uYDU4wLOYXHzgigVoYQc5xsS/6Q/ |1|VJWwLf/mldc2naUNSJdVuR+MwxCob5qZVR8MgT+jfiU= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9ZeVJ4ejjhE5QgjIz1iB8imXEsYpbVEzVJ2JEEtpG6nahg3mN8CEyv0gtSGt9HoPtiAMPieIcvGLVGmCEfOGJ3utifFrJAysVp2h1zlnt7ttwpF+X+Di30+XUZy7goyBUx2ZnICNKM+aPByQAPs2ohR8hXX0ErM/KsfjrCJ2rOQrkdRmI6OPr+HMudrHiUU7EfhW9zlj10pY1JIJfmuqR/h90zFNYxedkxUJnDh72RuelT+EvN9Y8bY/7KZt61FHYQmx8iE459ULdnXoRr0BhgZ2oH8m+bUlJYDmFpcQ2Dy5BYBqO3WNUyb75uYDU4wLOYXHzgigVoYQc5xsS/6Q/
##[debug]sshPassphrase=null
##[debug]check path : /home/tfs-build-agent/agent2/_work/_tasks/InstallSSHKey_5c9af2eb-5fc5-42dc-9b91-dc234a8c4400/0.151.2/task.json
##[debug]adding resource file: /home/tfs-build-agent/agent2/_work/_tasks/InstallSSHKey_5c9af2eb-5fc5-42dc-9b91-dc234a8c4400/0.151.2/task.json
##[debug]system.culture=en-US
##[debug]sshKeySecureFile=7e2886f1-72fd-4665-9534-59f2641a1447
##[debug]System.TeamFoundationCollectionUri=https://tfsb.abb.com/tfs/EPBP/
##[debug]SYSTEMVSSCONNECTION auth param ACCESSTOKEN = ***
##[debug]Agent.ProxyUrl=undefined
##[debug]secure file name for id 7e2886f1-72fd-4665-9534-59f2641a1447 = id_rsa_tfsa
##[debug]Agent.TempDirectory=/home/tfs-build-agent/agent2/_work/_temp
##[debug]Absolute path for pathSegments: /home/tfs-build-agent/agent2/_work/_temp,id_rsa_tfsa = /home/tfs-build-agent/agent2/_work/_temp/id_rsa_tfsa
##[debug]Downloading secure file contents to: /home/tfs-build-agent/agent2/_work/_temp/id_rsa_tfsa
##[debug]task result: Failed
##[error]Error: unable to get local issuer certificate
##[debug]Processed: ##vso[task.issue type=error;]Error: unable to get local issuer certificate
##[debug]Processed: ##vso[task.complete result=Failed;]Error: unable to get local issuer certificate
##[debug]secure file name for id 7e2886f1-72fd-4665-9534-59f2641a1447 = id_rsa_tfsa
##[debug]Agent.TempDirectory=/home/tfs-build-agent/agent2/_work/_temp
##[debug]Absolute path for pathSegments: /home/tfs-build-agent/agent2/_work/_temp,id_rsa_tfsa = /home/tfs-build-agent/agent2/_work/_temp/id_rsa_tfsa
##[debug]Deleting secure file at: /home/tfs-build-agent/agent2/_work/_temp/id_rsa_tfsa
##[debug]rm -rf /home/tfs-build-agent/agent2/_work/_temp/id_rsa_tfsa
##[debug]removing file
##[debug]End

Error logs

##[error]Error: unable to get local issuer certificate

[InternetPseudonym]

lorem ipsum dolor sit amet.

[anatolybolshakov]

Hi everyone, reopened this to investigate further