Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[issue] special character changes

See original GitHub issue

When some special characters used in a file, it’s been displayed correctly in the SH operator.

specialcharacter.sql SELECT 'àáâãäåèéêëìíîïòóôõöùúûüñýÿç' AS MatchFirstName

test.dig

timezone: UTC
 
_export:
  sh:
    shell: ["powershell.exe", "-"]

+disp_current_date:
  sh>: type c:\dev\test\test\specialcharacter.sql

Output

2019-03-12 19:01:30 -0700 [INFO] (0019@[0:default]+test+disp_current_date): sh>: type c:\dev\test\test\specialcharacter.sql
SELECT 'A A¡A¢A£A☼A¥A"AcAªA«A¬A-ArA_A²A3A'AµA¶A1AºA»A¼A±A½A¿A§' AS MatchFirstName

Issue Analytics

State:
Created 5 years ago
Comments:7 (4 by maintainers)

Top GitHub Comments

1reaction

mikoto2000commented, Mar 13, 2019

@kulmam92

Both PowerShell seem to require chcp 65001.

open by user
open by digdag.

Please try this.

chcp 65001 before run digdag
change sh> command to chcp 65001; type -Encoding UTF8 c:\dev\test\test\specialcharacter.sql

specialcharacter.sql

SELECT 'àáâãäåèéêëìíîïòóôõöùúûüñýÿç' AS MatchFirstName

test.dig

timezone: UTC

_export:
  sh:
    shell: ["powershell.exe", "-"]

+disp_current_date:
  sh>: chcp 65001; type -Encoding UTF8 c:\Users\mikoto\project\tmp\digdag\specialcharacter.sql

my operation log(I tried v0.9.35)

PS C:\Users\mikoto\project\tmp\digdag> chcp 65001
Active code page: 65001

PS C:\Users\mikoto\project\tmp\digdag> ..\..\..\bin\digdag.bat run .\test.dig --rerun
2019-03-14 00:50:14 +0900: Digdag v0.9.35
2019-03-14 00:50:16 +0900 [WARN] (main): Reusing the last session time 2019-03-13T00:00:00+00:00.
2019-03-14 00:50:16 +0900 [INFO] (main): Using session C:\Users\mikoto\project\tmp\digdag\.digdag\status\20190313T000000+0000.
2019-03-14 00:50:16 +0900 [INFO] (main): Starting a new session project id=1 workflow name=test session_time=2019-03-13T00:00:00+00:00
2019-03-14 00:50:17 +0900 [INFO] (0027@[0:default]+test+disp_current_date): sh>: chcp 65001; type -Encoding UTF8 c:\Users\mikoto\project\tmp\digdag\specialcharacter.sql
Active code page: 65001
SELECT 'àáâãäåèéêëìíîïòóôõöùúûüñýÿç' AS MatchFirstName

Success. Task state is saved at C:\Users\mikoto\project\tmp\digdag\.digdag\status\20190313T000000+0000 directory.
  * Use --session <daily | hourly | "yyyy-MM-dd[ HH:mm:ss]"> to not reuse the last session time.
  * Use --rerun, --start +NAME, or --goal +NAME argument to rerun skipped tasks.

1reaction

chezoucommented, Mar 13, 2019

If you want to show UTF-8 characters on your terminal, you have to set chcp 65001 before running the workflow. https://superuser.com/questions/237081/whats-the-code-page-of-utf-8

Top Results From Across the Web

Word changes Font after inserting special Characters

Created on August 31, 2020. Word changes Font after inserting special Characters. I have the same problem that I found here:.

Character set, encoding and special character issues - IBM

Character set, encoding and special character issues. This section describes how to resolve common character set and encoding issues. The IBM Dashboard ...

🚀 Preventing issues with special characters and page ...

When different components exchange data with one another, issues can arise with the character encoding used for text.

5 Ways to Fix Windows 10 Keyboard Special Characters Not ...

Let's find out the answer to what causes the issue and how to fix keyboard symbols changed issue in Windows 10. 1. Restart...

How to deal with special characters in CSV - CSV Loader

Problems with special characters and encoding usually occur when you open the CSV in another program (i.e., not a regular text editor).