URL reader for techdocs not working while pulling from the Azure DevOps

Expected Behavior

When clicking on the READ DOCS link that is generated using my simple example, the techdocs are properly displayed.

Current Behavior

The following error appears in the logs after clicking on the READ DOCS:

│ 2021-05-18T09:11:29.893Z techdocs info Step 2 of 3: Generating docs for entity Component:default/space type=plugin                                                                                     ││ 2021-05-18T09:11:30.408Z techdocs error Build failed with error: DEBUG   -  Loading configuration file: /tmp/backstage-3wIBEu/mkdocs.yml,ERROR   -  Config value: 'docs_dir'. Error: The path /tmp/bac ││ kstage-3wIBEu/docs isn't an existing directory.                                                                                                                                                        
││ DEBUG   -  Config value: 'config_file_path' = '/tmp/backstage-3wIBEu/mkdocs.yml'                                                                                                                       
││ DEBUG   -  Config value: 'site_name' = 'space-docs'                                                                                                                                                    
││ DEBUG   -  Config value: 'nav' = [{'Home': 'index.md'}, {'Test': 'index2.md'}],DEBUG   -  Config value: 'pages' = None                                                                       
││ DEBUG   -  Config value: 'site_url' = ''                                                                                                                                                               
││ DEBUG   -  Config value: 'site_description' = None                                                                                                                                                     
││ DEBUG   -  Config value: 'site_author' = None                                                                                                                                                          
││ DEBUG   -  Config value: 'theme' = {'name': 'mkdocs'}                                                                                                                                                  
││ DEBUG   -  Config value: 'docs_dir' = 'docs'                                                                                                                                                           
││ DEBUG   -  Config value: 'site_dir' = '/tmp/techdocs-tmp-LA7GwR'                                                                                                                                       
││ DEBUG   -  Config value: 'copyright' = None                                                                                                                                                            
││ DEBUG   -  Config value: 'google_analytics' = None                                                                                                                                                     
││ DEBUG   -  Config value: 'dev_addr' = Address(host='127.0.0.1', port=8000)                                                                                                                             
││ DEBUG   -  Config value: 'use_directory_urls' = True,DEBUG   -  Config value: 'repo_url' = ''                                                                                                          
││ DEBUG   -  Config value: 'repo_name' = None                                                                                                                                                            
││ DEBUG   -  Config value: 'edit_uri' = None                                                                                                                                                             
││ DEBUG   -  Config value: 'extra_css' = [],DEBUG   -  Config value: 'extra_javascript' = []                                                                                                             
││ DEBUG   -  Config value: 'extra_templates' = [],DEBUG   -  Config value: 'markdown_extensions' = ['toc', 'tables', 'fenced_code'],DEBUG   -  Config value: 'mdx_configs' = None                        
││ DEBUG   -  Config value: 'strict' = False,DEBUG   -  Config value: 'remote_branch' = 'gh-pages'                                                                                                        
││ DEBUG   -  Config value: 'remote_name' = 'origin'                                                                                                                                                      
││ DEBUG   -  Config value: 'extra' = {},DEBUG   -  Config value: 'plugins' = PluginCollection([('techdocs-core', <src.core.TechDocsCore object at 0x7fe178a43b00>)]),Aborted with 1 Configuration Errors ││ ! type=plugin

I’m fetching the project from Azure DevOps and it seems there is some problem with “unpacking” the docs dir? This is how the repo looks like:

.
├── README.md
├── backstagenlp
│   ├── __init__.py
│   └── backstagenlp.py
├── cookiecutter.json
├── docs
│   ├── index.md
│   └── index2.md
├── mkdocs.yml

And the mkdocs.yml

site_name: 'space-docs'

nav:
  - Home: index.md
  - Test: index2.md

plugins:
  - techdocs-core

Now, when the repo is fetched, this is how it looks inside the container:

|-- README.md
|-- __init__.py
|-- backstagenlp.py
|-- cookiecutter.json
|-- index.md
|-- index2.md
|-- mkdocs.yml

Somehow all the directories are flattened but the techdocs plugin doesn’t seem to like it: Config value: 'docs_dir'. Error: The path /tmp/backstage-3wIBEu/docs isn't an existing directory

This is the annotation definition from catalog-info:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: space
  annotations:
    backstage.io/techdocs-ref: url:https://dev.azure.com/org/project/_git/backstagenlp

Possible Solution

Steps to Reproduce

1. Create the app using yarn backstage-create-app
2. Add the above example to your app-config.yaml
3. Use local setup:

techdocs:
  builder: 'local' # Alternatives - 'external'
  generators:
    techdocs: 'local' # Alternatives - 'docker'
  publisher:
    type: 'local' # Alternatives - 'googleGcs' or 'awsS3' or 'azureBlobStorage' or 'openStackSwift'. Read documentation for using alternatives.

1. Build docker image and start it.

Context

See above.

Your Environment

• NodeJS Version (v12): v14
• Operating System and Version (e.g. Ubuntu 14.04): using default Dockerfile, image running on AKS
• Used master branch for building, latest commit https://github.com/backstage/backstage/commit/f3f09dda341ff21fc1d55d088ee6698a638feb93

[benjdlambert]

Think this has been fixed with last weeks release. Closing and feel free to reopen if the problem persists after updating.

[awanlin]

@nadworny @OrkoHunter Pretty sure you are running into the issue I described over here: #4359