question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Feature][config] improve the config parse logic

See original GitHub issue

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

This sub issue focus on solving the parse logic

  • Change path token separator from “.” to “->”".
  • Added parseObjectForseatunnel() to parse config objects in input, filter, output for seatunnel.

👉 Change 1 : Change path token separator from “.” to “->”".

config always let the key as path if the key contains ., like following

sink {
  a.b.c.Console {}
}

equivalent to

sink {
  a {
    b {
      c {
        Console {}
      }
    }
  }
}

In fact, it’s not recommend to make these changes, because JSON keys should be enclosed in double quotes, so we can revert these changes.

👉 Change 2 : parseObjectForseatunnel().

The main login of parseObjectForseatunnel is to parse config objects in sources, transforms, sinks for seatunnel.

We can move these changes out of config, and move them to ConfigBuilder

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
RickyHuocommented, Dec 30, 2021

I have saw the draft PR. The seatunnel-config has been replaced with typesafe config.

The reason why we develop seatunnel-config base on typesafe config is that the plugin order of parsed Config is not the same as our configuration when transform section has multiply plugins.

The origin issue is #29, I’m sorry I forget config test at that moment.

We should test those demo:

source {
	
}

transform {
	
	sql {
		sql = "sql1"
	}

	sql {
		sql = "sql2"
	}

	sql {
		sql = "sql3"
	}
}

sink {
	
}

source {
	
}

transform {
	
	sql {
		sql = "sql1"
	}

	json {
		source_field = "xxx"
	}

	sql {
		sql = "sql3"
	}
}

sink {
	
}

1reaction
simon824commented, Dec 30, 2021

Great job ! I’m ok with these two changes, and I will test more cases later.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Feature: config-directory.d/ pattern support · Issue #324 · node ...
Hi! As we have seeing with many Unix packages sometimes configuration files grow complex, long and difficult to handle. Many projects out there...
Read more >
Feature Engagement
FeatureConfig ; EventConfig; Comparator; Boolean; SessionRateImpact ... The client-side backend is feature agnostic and has no special logic for any specific ...
Read more >
Feature Configuration Guide for Cisco Unified ...
Use the following procedure to configure device mobility group is a logical grouping of sites with similar dialing patterns.
Read more >
Windows OS Optimization Essentials Part 3 - Nutanix
As opposed to what we saw previously with Active Setup and the Microsoft Store, ... The logic behind a ForEach is ForEach ($<item>...
Read more >
History — DeepCTR 0.9.3 documentation
Improve compatibility with tensorflow 2.x . ... Simplify VarLenSparseFeat,support setting weight_normalization. ... Simplify the input logic.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found