sqlglot seems to remove adjacent comments

The following line,

Select * from Table /*comment 1*/
/*comment 2*/

becomes,

Select * from Table /*comment 2*/

after we run the parser,

expression_tree = parse_one(query)

This is an unexpected behaviour.

[mpf82]

Again, this is just my preference, but I’d say yes, for “normal” mode, side-by-side should be used, so SELECT * FROM table /*comment 1*/ /*comment 2*/ would look the same after parsing and printing.

For pretty mode, it’s probably easiest to also output the comments using /* … */ notation, that way you do not have to worry about multi-line strings inside comments, the difference to normal mode being, that each comment has a line-break prepended.

Let’s talk code.

Initial statement: SELECT * FROM tbl /*line1\nline2\nline3*/ /*another comment*/ where 1=1 -- comment at the end

Current output (10.0.8) (“another comment” being ignored/swallowed)

SELECT * FROM tbl /* line1
line2
line3 */ WHERE 1 = 1 /* comment at the end */

Suggestion normal mode:

SELECT * FROM tbl /* line1
line2
line3 */ /* another comment */ WHERE 1 = 1 /* comment at the end */

Suggestion pretty mode:

SELECT * FROM tbl
/* line1
line2
line3 */
/* another comment */
WHERE 1 = 1
/* comment at the end */

To extend on the idea, you could also remember if the original comment was a block or inline comment, however this would require either a Comment class where you store the inline/block attribute, or you could make an educated guess, based on whether or not the comment includes a \n.

I guess would be the best way, as (in pretty mode) it could print very much the comment as it originally was, but it’s the most work.

For normal mode, using always block comments in order is probably best, so

-- comment 1
-- comment 2
-- comment 3
SELECT * FROM foo

would print as

/* comment 1 */ /* comment 2 */ /* comment 3 */ SELECT * FROM foo

with the above mentioned changes, in pretty mode, the output could be the same as the input, as it would remember the inline attribute on the comment object.

---

By the way, it seems currently comments are not supported “everywhere”, e.g.

SELECT /* a */ * FROM /* b */ tbl WHERE /* c */ 1=2"

parses to

(SELECT expressions:
  (STAR ), from:
  (FROM expressions:
    (TABLE this:
      (IDENTIFIER this: tbl, quoted: False))), where:     
  (WHERE this:
    (EQ this:
      (LITERAL this: 1, is_string: False), expression:    
      (LITERAL this: 2, is_string: False))), comment:  a )

and prints as

/* a */ SELECT * FROM tbl WHERE 1 = 2

I didn’t have time to look at the code to figure out if there’s a reason for this behaviour, just wanted to mention it.

[GeorgeSittas]

Hello SudarshanVS,

The logic is different if you place the comment on the same line as the previous token, as opposed to placing it on a new line. That’s why we get different outputs. However, you have a point, since the tokenizer is expected to not overwrite the previous token’s comment if any.

I’ll post a fix so that the two cases are consistent and we always keep the first comment.