question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[vie] Enhancements

See original GitHub issue

I notice a few obvious problems with Vietnamese:

  1. As far as I can tell, nearly all Vietnamese words have three phonetic pronunciations: “Hà Nội”, "Huế, and “Hồ Chí Minh City”. We should just add dialects to languages.json
  2. As is well-known, the Vietnamese Roman orthography puts a space between every syllable. So our data set is really only monosyllabic Vietnamese words! We could solve this by adding a flag (--no-skip-space) here, or we could make a language specific-extractor, I suppose.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:11

github_iconTop GitHub Comments

1reaction
kylebgormancommented, Mar 27, 2020

I guess some reasons to skip them include:

  1. There’s no general guidance for why multiword expressions might be entered in Wiktionary in the first place. Sometimes they’re common compounds or idioms, but occasionally there are even entire phrases.
  2. There’s no reason to collect multiword expressions for pronunciation purposes, which is our goal here.

You’re right that that function controls it, but I think we probably want it to be scriptable so users can control whether they skip, and we can set it on a per-language basis for the big scrape. So I think the obvious thing to do is to add a flag (in cli.py) and then make sure it’s passed to _skip_word; then, disable that for Chinese and Vietnamese.

Potentially we could also make the rules about what gets skipped more expressive—maybe you could pass in a regular expression and if it’s matched, the word is skipped—but that’s probably premature.

0reactions
lfashbycommented, Apr 1, 2020

Scraping for a Vietnamese dialect from the three that Kyle added in #135 returns results for all three current dialects (when phonetic=True). Scraping for one of the additional dialects that Yeonju has added returns the same results as scraping for one of the first three dialects added in #135 (when phonetic=True). As far as I can tell this is due to how we define _DIALECT_XPATH_SELECTOR_TEMPLATE in config.py and the inconsistent HTML used on pages like cầm. It looks like Vietnamese dialect information (for the three dialects covered in #135 ) is stored within <i> elments, while the dialect information for the dialects that Yeonju has added is stored as text in <span class="ib-content qualifier-content"> elements - but not as text in the <a> elements within those span elements that we usually target (which is why we get nothing when scraping for one of the dialects Yeonju found while phonetic=False).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Managing Vendor Price Increases on Minor Enhancements
At VIE Healthcare®, we categorize enhancements into either: ... While enhancements are minor, vendors often raise their prices by 30% or more.
Read more >
Lip correction and enhancement at Vie Aesthetics - YouTube
Be smart when it comes to having your lips enhanced with fillers. So many people have had to learn this the hard way....
Read more >
Vie Air Collection Fan Series, 20 Inch, Charcoal Gray
The 3 speed control and tiling fan head allows for easy air flow management. This floor fan is ideal for living room or...
Read more >
How it works - Viesus
Viesus intelligently analyzes images and applies only necessary enhancements, rather subtle than exaggerated.
Read more >
Semantic content enhancements with Drupal, Apache Stanbol ...
Next, we integrated VIE.js libraries and widgets into Drupal 7 such that one can easily use Apache Stanbol content enhancement capabilities from within ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found