PySpark setting for launch.json has bad default values

Environment data

• VS Code version: 1.27.2
• Extension version (available under the Extensions sidebar): 2018.8.0
• OS and version: Mac OS High Sierra 10.13.6
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): direnv
• Relevant/affected Python packages and their versions: XXX

Actual behavior

I followed the steps on this page to setup IntelliSense for pyspark. I have a standalone spark downloaded and I wanted to try and integrate it with VSCode. I have already set up my SPARK_HOME and I ran the command “Update Workspace PySpark Libraries”.

However I was still not able to get IntelliSense running as expected for PySpark. Turns out what happened was that the environment variable referencing was incorrect:

Expected behavior

The reference to SPARK_HOME should be ${env:SPARK_HOME} instead of ${env.SPARK_HOME} as specified in VSCode documentation

Steps to reproduce:

1. Download a standalone Spark binary from https://spark.apache.org/downloads.html
2. Set SPARK_HOME as an environment variable in your shell (e.g bash_profile etc.)
3. Open VSCode in a python workspace
4. Run “Update Workspace PySpark Libraries” from Cmd+P
5. Create a .py file and add in from pyspark import SparkContext
6. Autocompletion fails to find information about SparkContext

I know that I could have just installed pyspark using pip install, which does work as expected, but I was just trying out this setup which took me quite a while to realize the mistake

[brettcannon]

@btruhand PRs are always welcome, but realize you are actually looking at Don’s personal copy of the extension code and not the code in this repo. 😄

[btruhand]

If there is a better way to do it in the team’s opinion then yeah probably documentation update.

Else I was thinking something needs to be changed so that ${env.SPARK_HOME} is made to be ${env:SPARK_HOME} instead