feast materialize runs into BigQuery syntax error

Expected Behavior

No syntax error

Current Behavior

running feast materialize using BigQuerySource creates a query that has syntax errors

Steps to reproduce

1. Run feast materialize
2. Check the output and inspect the query as part of the job output

 google.api_core.exceptions.BadRequest: 400 Syntax error: Expected ")" but got "," at [4:58]

Query with invalid syntax (missing parenthesis)

SELECT InvoiceNo, Quantity, InvoiceDate,
FROM (
SELECT InvoiceNo, Quantity, InvoiceDate, ,
ROW_NUMBER() OVER(PARTITION BY InvoiceNo ORDER BY InvoiceDate DESC,  DESC) AS _feast_row
FROM `feast-feature-store:data.ecommerce`
WHERE InvoiceDate BETWEEN TIMESTAMP('2011-03-24 00:00:00+00:00') AND TIMESTAMP('2011-08-02 00:00:00+
7 00:00')
)
WHERE _feast_row = 1

The feature definition does not contain any extraordinary

from datetime import timedelta
from feast import BigQuerySource, Entity, Feature, FeatureView, ValueType


transactions = Entity(
    name="transactions",
    join_key="InvoiceNo",
    value_type=ValueType.INT64,
)


transactions_source= BigQuerySource(
    table_ref="feast-feature-store:data.ecommerce",
    event_timestamp_column="InvoiceDate",
)


transactions_view = FeatureView(
    name="transactions",
    entities=["transactions"],
    ttl=timedelta(weeks=52),
    features=[
        Feature(name="Quantity", dtype=ValueType.INT64)
    ],
    input=transactions_source,
    tags={"team": "driver_performance"},
)

Specifications

• Version: 0.10
• Platform: GCP
• Subsystem:

Possible Solution

[woop]

(In which way is feast using it?)

Whenever you have duplicated rows in your source (meaning a row with the same event_timestamp and InvoiceNo) then we deduplicate by taking the latest row based on created_timestamp. The created_timestamp is when the row was written to the source.

[SaschaHeyer]

confirmed working after upgrading to 0.10.1 lovely ❤️

Materializing feature view transactions from 2011-03-24 00:00:00+01:00 to 2011-08-02 00:00:00+02:00 done!