question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Move impala imports out of global namespace into tests where its used

See original GitHub issue

Should IbisTestEnv be using impala when its doing file system testing?

In Ibis filesystems means actually hdfs, and hdfs is only supported with Impala afaik. I thought on moving it inside the backend, but it’s part of the public API, so users may be using it, so it requires more thought, discussion, and probably a deprecation warning first.

In any case, we should be calling pytest with not hdfs, and that should prevent test_filesystems from running. But I guess that import is in the global namespace, and that’s causing the issue. Not sure why this is just failing in this PR. If you want to move that import inside the test/tests where it’s being used (assuming it’s not too many) that would be good. Probably better in a separate PR.

Thanks for all the work with this @matthewmturner, really happy to have you fixing all this technical debt.

_Originally posted by @datapythonista in https://github.com/ibis-project/ibis/pull/2435#issuecomment-707612410_

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
datapythonistacommented, Oct 14, 2020

Would the idea be to move the global ENV = IbisTestEnv() and from ibis.impala.tests.conftest import IbisTestEnv into an __init__ method for each of those classes with a variable like self.ENV = IbisTestEnv() and then update references to cls.ENV to self.ENV?

I think should be as easy as replacing this:

cls.ENV = ENV

By

from ibis.impala.tests.conftest import IbisTestEnv
cls.ENV = IbisTestEnv()

And then removing the import and the initialization of IbisTestEnv from the global namespace.

0reactions
matthewmturnercommented, Oct 21, 2020

closed via #2470

Read more comments on GitHub >

github_iconTop Results From Across the Web

Add or remove imported namespaces (Visual Basic)
Use the Imports statement to use a namespace in a single source code file. To add an imported namespace. In Solution Explorer, double-click...
Read more >
Porting SQL from other database systems to Impala
Impala can use data files produced from many different sources, such as Pig, Hive, or MapReduce jobs. The fast import mechanisms of LOAD...
Read more >
Using and importing modules and namespaces
A key point with modules is that they produce separate namespaces. A namespace (also called a scope) is simply the domain of control...
Read more >
python - Change of a global variable gets lost when importing ...
If it's imported once, it will execute the module level code, but it will not execute again if it's imported again. Hence you...
Read more >
Tcl Built-In Commands - namespace manual page
The namespace import command has snapshot semantics: that is, only requested commands that are currently defined in the exporting namespace are imported. In...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found