Indentation after CROSS APPLY is messed up with dialect T-SQL

Search before asking

• I searched the issues and found no similar issues.

What Happened

When formatting a query containing a CROSS APPLY expression, all lines after the CROSS APPLY expression are wrongly indented by an additional indentation.

Expected Behaviour

SELECT
    table1.col,
    table2.col
FROM table1
CROSS APPLY (
    VALUES ((1), (2))
) AS table2(col)
-- Expected: no additional indentation after running `sqlfluff fix`
INNER JOIN table3
    ON table1.col = table3.col;

Observed Behaviour

SELECT
    table1.col,
    table2.col
FROM table1
CROSS APPLY (
    VALUES ((1), (2))
) AS table2(col)
-- Additional indentation after formatting via `sqlfluff fix`!
    INNER JOIN table3
        ON table1.col = table3.col;

How to reproduce

Format the above query with sqlfluff fix.

Dialect

T-SQL

Version

• sqlfluff, version 1.2.1 (installed via homebrew)

Configuration

[sqlfluff]
dialect = tsql

Are you willing to work on and submit a PR to address the issue?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project’s Code of Conduct

[tunetheweb]

Awesome, thanks for your extensive answer @tunetheweb! Regarding tests: where can I add tests related to formatting? It seems like the SQL files in dialects/... are only useful for testing the parser.

You can add test cases in test/fixtures/rules/std_rule_cases/L003.yml

@tunetheweb, I’m not exactly sure why this happens, but simply moving the grammar for the “join type” to a named grammar and referencing it (i.e. your change nr. 1), causes tests to fail: test/fixtures/dialects/ansi/select_many_join.yml cannot be parsed anymore. I’m not sure why this happens? I thought that the reference is just replaced with the grammar at runtime.

Ah my fault. You need to move the optional=True,:

+++ b/src/sqlfluff/dialects/dialect_ansi.py
@@ -546,6 +546,18 @@ ansi_dialect.add(
         "FILTER", Bracketed(Sequence("WHERE", Ref("ExpressionSegment")))
     ),
     FrameClauseUnitGrammar=OneOf("ROWS", "RANGE"),
+    PreJoinKeywordsGrammar=OneOf(
+        "CROSS",
+        "INNER",
+        Sequence(
+            OneOf(
+                "FULL",
+                "LEFT",
+                "RIGHT",
+            ),
+            Ref.keyword("OUTER", optional=True),
+        ),
+    ),
     # It's as a sequence to allow to parametrize that in Postgres dialect with LATERAL
     JoinKeywordsGrammar=Sequence("JOIN"),
     # NATURAL joins are not supported in all dialects (e.g. not in Bigquery
@@ -1468,19 +1480,7 @@ class JoinClauseSegment(BaseSegment):
     match_grammar: Matchable = OneOf(
         # NB These qualifiers are optional
         Sequence(
-            OneOf(
-                "CROSS",
-                "INNER",
-                Sequence(
-                    OneOf(
-                        "FULL",
-                        "LEFT",
-                        "RIGHT",
-                    ),
-                    Ref.keyword("OUTER", optional=True),
-                ),
-                optional=True,
-            ),
+            Ref("PreJoinKeywordsGrammar", optional=True,),

If you Ref something it needs to match something unless you make it optional.

[borchero]

@tunetheweb, I’m not exactly sure why this happens, but simply moving the grammar for the “join type” to a named grammar and referencing it (i.e. your change nr. 1), causes tests to fail: test/fixtures/dialects/ansi/select_many_join.yml cannot be parsed anymore. I’m not sure why this happens? I thought that the reference is just replaced with the grammar at runtime.