`to_gbq` fails to create empty table with correct schema

I want to create an empty table that has a specific schema, which in this specific case I’m inferring from a dask dataframe. The table gets created on BigQuery, however it shows with no schema.

Minimal reproducible example:

import pandas_gbq
import dask.dataframe as dd
import random 

#generate some data
def gen_data(size=10):
    records = [
        {
            "name": random.choice(["fred", "wilma", "barney", "betty"]),
            "number": random.randint(0, 100),
            "idx": i,
        }
        for i in range(size)
    ]
    return records

test= gen_data(size=50)
df = pd.DataFrame(test)

ddf = dd.from_pandas(df, chunksize=10)

meta = ddf._meta.dtypes.to_dict()
schema = [{"name":k, "type": v} for k, v in meta.items()]

#the schema looks like this
# [{'name': 'name', 'type': dtype('O')},
# {'name': 'number', 'type': dtype('int64')},
#  {'name': 'idx', 'type': dtype('int64')}]

destination_table="my_dataset.test_table"
project_id="my_project"

pandas_gbq.to_gbq(dataframe=pd.DataFrame(), destination_table=destination_table, 
          project_id=project_id, if_exists="append", table_schema=schema)

But when I check the schema on the table created I get:

Environment details

• Programming language: python 3.3.10
• OS: MacOS Big Sur 11.4
• Package version: pandas-gbq 0.15.0 (pypi)
• pandas version: 1.3.1
• Dask version: 2021.07.2

[jimfulton]

You want something like:

schema = [
    {'name': 'name', 'type': 'STRING'},
    {'name': 'number', 'type': 'int64'},
    {'name': 'idx', 'type': 'int64'},
]
pandas_gbq.to_gbq(
    pd.DataFrame({field['name']: [] for field in schema}),
    destination_table=destination_table, 
    project_id=project_id,
    if_exists="append",
    table_schema=schema)

[tswast]

Thanks for the report! I agree this looks like a bug.