question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Detect POSIX-style paths on Windows

See original GitHub issue

I have a git binary installed on Windows (the MSYS2 variant, specifically) that does not understand Windows-style paths. When trying to install software using setuptools_scm, this leads to cryptic errors such as the following:

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/nmigen
$ python setup.py develop
Traceback (most recent call last):
  File "setup.py", line 10, in <module>
    setup(
  File "C:/msys64/mingw64/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "C:/msys64/mingw64/lib/python3.8/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "C:/msys64/mingw64/lib/python3.8/site-packages/setuptools/dist.py", line 447, in __init__
    _Distribution.__init__(self, {
  File "C:/msys64/mingw64/lib/python3.8/distutils/dist.py", line 292, in __init__
    self.finalize_options()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/setuptools/dist.py", line 740, in finalize_options
    ep.load()(self)
  File "C:/msys64/mingw64/lib/python3.8/site-packages/setuptools/dist.py", line 747, in _finalize_setup_keywords
    ep.load()(self, ep.name, value)
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/integration.py", line 17, in version_keyword
    dist.metadata.version = _get_version(config)
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/__init__.py", line 147, in _get_version
    parsed_version = _do_parse(config)
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/__init__.py", line 103, in _do_parse
    version = _version_from_entrypoints(config) or _version_from_entrypoints(
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/__init__.py", line 63, in _version_from_entrypoints
    version = _call_entrypoint_fn(root, config, ep.load())
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/__init__.py", line 44, in _call_entrypoint_fn
    return fn(root, config=config)
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/git.py", line 98, in parse
    wd = GitWorkdir.from_potential_worktree(config.absolute_root)
  File "c:/msys64/home/william/projects/fpga/nmigen/.eggs/setuptools_scm-3.5.0-py3.8.egg/setuptools_scm/git.py", line 33, in from_potential_worktree
    if not samefile(real_wd, wd):
  File "C:\msys64\mingw64/lib/python3.8/genericpath.py", line 100, in samefile
    s1 = os.stat(f1)
FileNotFoundError: [WinError 3] The system cannot find the path specified: '/home/William/Projects/FPGA/nmigen'

The error comes from the path git returns from the rev-parse command:

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/nmigen
$ git rev-parse --show-toplevel
/home/William/Projects/FPGA/nmigen

One potential solution is to pipe the output of git rev-parse through cygpath is we detect that the git being run is MSYS2 (or otherwise doesn’t understand Windows-style paths), as unfortunately conversion between POSIX and Windows path is an involved problem:

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/nmigen
$ git rev-parse --show-toplevel | cygpath -w -f -
C:\msys64\home\William\Projects\FPGA\nmigen

The following workaround allows me to successfully run the egg-info, develop, and install successfully (with warning: no previously-included files found matching '.travis.yaml'). Obviously this patch isn’t acceptable as-is, and this may not be the correct repo to bring this issue up, depending on the scope of supported git variants. However, I figure it’s worth a shot.

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/py-packages/setuptools_scm
$ git diff
diff --git a/src/setuptools_scm/file_finder_git.py b/src/setuptools_scm/file_finder_git.py
index 8b81d2c..f68c91a 100644
--- a/src/setuptools_scm/file_finder_git.py
+++ b/src/setuptools_scm/file_finder_git.py
@@ -17,6 +17,13 @@ def _git_toplevel(path):
                 universal_newlines=True,
                 stderr=devnull,
             )
+
+            out = subprocess.check_output(
+                ["cygpath", "-w", out],
+                cwd=(path or "."),
+                universal_newlines=True,
+                stderr=devnull,
+            )
         trace("find files toplevel", out)
         return os.path.normcase(os.path.realpath(out.strip()))
     except subprocess.CalledProcessError:
diff --git a/src/setuptools_scm/git.py b/src/setuptools_scm/git.py
index afefa34..43cc6c5 100644
--- a/src/setuptools_scm/git.py
+++ b/src/setuptools_scm/git.py
@@ -27,6 +27,9 @@ class GitWorkdir(object):
     @classmethod
     def from_potential_worktree(cls, wd):
         real_wd, _, ret = do_ex("git rev-parse --show-toplevel", wd)
+        if ret:
+            return
+        real_wd, _, ret = do_ex("cygpath -w {}".format(real_wd), wd)
         if ret:
             return
         trace("real root", real_wd)

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/py-packages/setuptools_scm
$

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:13 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
RonnyPfannschmidtcommented, Jan 31, 2021

Can you open a pr for more detailed discussion

I believe it looks okish enough

0reactions
lazkacommented, Jan 31, 2021

We’ve patched this now in MSYS2 by calling git in a way so it outputs relative paths: https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python-setuptools-scm/0001-git-Use-show-prefix-instead-of-show-toplevel-for-fin.patch

If this is something that’s considered acceptable I can try to upstream it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Confusing paths in Windows vs POSIX - CloudAppie
We are using the CLI to determine the validity of SPFx packages. These packages contain JSON files with relative paths to other files...
Read more >
Path.Posix - Hackage
This library provides a well-typed representation of paths in a filesystem directory tree. Note: This module is for working with Posix style paths....
Read more >
How to run OS-agnostic Jest test files that check paths?
In your tests you then always write POSIX-style paths /path/to/thing , and path takes care of providing the appropriate path separator for ...
Read more >
[Q]Windows PATH is not working.. - Cygwin
Is there anybody doing OK with Windows path? Cygwin documentation says: Cygwin supports both Win32- and POSIX-style paths, where directory ...
Read more >
Python 3's pathlib Module: Taming the File System
Counting Files; Display a Directory Tree; Find the Last Modified File ... A little tip for dealing with Windows paths: on Windows, the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found