question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Masking passwords with empty connection passwords make some logs unreadable in 2.1.0

See original GitHub issue

Discovered in this Slack conversation.

When you have connections with empty passwords masking logs masks all the character breaks:

[2021-05-23 04:00:23,309] {{logging_mixin.py:104}} WARNING - ***-***-***-*** ***L***o***g***g***i***n***g*** ***e***r***r***o***r*** ***-***-***-***
[2021-05-23 04:00:23,309] {{logging_mixin.py:104}} WARNING - ***T***r***a***c***e***b***a***c***k*** ***(***m***o***s***t*** ***r***e***c***e***n***t*** ***c***a***l***l*** ***l***a***s***t***)***:***
[2021-05-23 04:00:23,309] {{logging_mixin.py:104}} WARNING - *** *** ***F***i***l***e*** ***"***/***u***s***r***/***l***o***c***a***l***/***l***i***b***/***p***y***t***h***o***n***3***.***8***/***l***o***g***g***i***n***g***/***_***_***i***n***i***t***_***_***.***p***y***"***,*** ***l***i***n***e*** ***1***0***8***1***,*** ***i***n*** ***e***m***i***t***
*** *** *** *** ***m***s***g*** ***=*** ***s***e***l***f***.***f***o***r***m***a***t***(***r***e***c***o***r***d***)***

Until this is fixed, an easy workaround is to disable masking via disabling sensitive connection masking in configuration:

[core]
hide_sensitive_var_conn_fields = False

or vial env variable:

AIRFLOW__CORE__HIDE_SENSITIVE_VAR_CONN_FIELDS="False"

This is only happening if the task accesses the connection that has empty password. However there are a number of cases where such an empty password might be “legitimate” - for example in google provider you might authenticate using env variable or workload identity and connection will contain an empty password then.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
ashbcommented, May 25, 2021

Oh, I think I know – I think they had something like this in their extra field:

{"password": ""}

That would do it.

0reactions
potiukcommented, May 25, 2021

Why don’t we do a bit "smarter’ formatting. I know it’s adding complexity but I think it could work better if for short passwords (1-3 letters) we only replace whole words. That would require a regexp matching (so lilkely slower with [\W^]PASS[\W$] or smth. but it would handle all the cases where we have password:ab or ab. Again - not very strong on it the empty case handles most problems, the “short password” is mostly a test/edge case., just trying to see if we can brainstorm on finding simple yet effective protection.

My current thinking. When I consider edge cases, for sure it makes very little sense to replace 1 character passwords (there is no point in that, really) so we could exclude 1-character passwords immediately (and they are sometimes - quite often even used in test systems when you need non-empty password but you have no minimum length). Two characters are not likely to be good passwords neither for test nor for any real use so maybe we should not worry about those.

How about excluding all passwords with length <=1 ? That might protect against VAST majority of cases where log output will be weird.

Read more comments on GitHub >

github_iconTop Results From Across the Web

java - Connection properties like username and password are ...
Connection properties like username and password are getting printed in the Logs without masking. How to mask the password?(in Hibernate 2.1.6).
Read more >
Masking sensitive data - Apache Airflow
Airflow will by default mask Connection passwords and sensitive Variables and keys from a Connection's extra (JSON) field when they appear in Task...
Read more >
Security - ArchWiki - Arch Linux
As a rule, do not pick insecure passwords just because secure ones are harder to remember. Passwords are a balancing act. It is...
Read more >
Disable masking of usernames - Jenkins Jira
Is there a way to disable or configure the password masking feature? ... which result in logs that are totally mangled and unreadable,...
Read more >
Security Guide Red Hat Enterprise Linux 7
The nullok option, which allows users to log in with a blank password if the password field in the /etc/shadow file is empty,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found