question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Empty file when sending an XLSX or BytesIO

See original GitHub issue

First Check

  • I added a very descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn’t find it.
  • I searched the FastAPI documentation, with the integrated search.
  • I already searched in Google “How to X in FastAPI” and didn’t find any information.
  • I already read and followed all the tutorial in the docs and didn’t find an answer.
  • I already checked if it is not related to FastAPI but to Pydantic.
  • I already checked if it is not related to FastAPI but to Swagger UI.
  • I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

  • I commit to help with one of those options 👆

Example Code

from datetime import datetime
import uvicorn
import asyncio
from io import BytesIO as IO
from io import StringIO as SIO
import pandas as pd
import os
from fastapi import BackgroundTasks, Body, File, UploadFile, Form,FastAPI, Depends
from fastapi_mail import FastMail, MessageSchema, ConnectionConfig
from dotenv import load_dotenv
from starlette.responses import JSONResponse
from typing import Any, Dict
from pydantic import EmailStr, BaseModel
import numpy as np
load_dotenv('.env')

class Envs:
    MAIL_USERNAME = os.getenv('MAIL_USERNAME',"")
    MAIL_PASSWORD = os.getenv('MAIL_PASSWORD',"")
    MAIL_FROM = os.getenv('MAIL_FROM',"")
    MAIL_PORT = int(os.getenv('MAIL_PORT',587))
    MAIL_SERVER = os.getenv('MAIL_SERVER',"smtp.office365.com")
    MAIL_FROM_NAME = os.getenv('MAIN_FROM_NAME','')
    MAIL_USE_TLS = os.getenv('MAIL_USE_TLS',True)
    MAIL_USE_SSL = os.getenv('MAIL_USE_SSL',False)

conf = ConnectionConfig(
    MAIL_USERNAME=Envs.MAIL_USERNAME,
    MAIL_PASSWORD=Envs.MAIL_PASSWORD,
    MAIL_FROM=Envs.MAIL_FROM,
    MAIL_PORT=Envs.MAIL_PORT,
    MAIL_SERVER=Envs.MAIL_SERVER,
    MAIL_FROM_NAME=Envs.MAIL_FROM_NAME,
    MAIL_TLS=Envs.MAIL_USE_TLS,
    MAIL_SSL=Envs.MAIL_USE_SSL,
    USE_CREDENTIALS=True,
    
)

mailing= FastAPI(title="Envio de correos", description="En este proyecto seres capaz de enviar correos facilmente", version="1.1"

)

@mailing.post("/dayli") 
async def alarmas_diaria():
    #obtengo la data    
    
    data= pd.DataFrame(np.random.randn(100, 2))
    xls_data = get_xls_data(data)
    csv_data = get_csv_data(data) 
    #creo una tarea para enviar el corroe
    respuesta= await send_email_async_file({'email':"jdsuarez@logyca.com"},{'file':xls_data})
    #await send_email_async_file({'email':"jdsuarez@logyca.com"},{'file':csv_data})
    
    #asyncio.create_task(dayli_report.send_reports(day=fecha.day))
    return(respuesta)

def get_xls_data(df):
    output = IO()
    with pd.ExcelWriter(output,  engine='openpyxl', options={'strings_to_numbers': False}) as writer:
        df = df.replace(np.nan, '', regex=True)
        df = df.astype(str)
        df.reset_index(drop=True).to_excel(writer, 'AlarmaDiaria', index=False)
        writer.save()
        output.seek(0)
        return output

def get_csv_data(df):
    s_buf = SIO()
    df.to_csv(s_buf,index=False, header=False)
    s_buf.seek(0)
    return s_buf

class fileSchema(BaseModel):
    file:bytes = File(...)


class EmailSchema(BaseModel):    
    email: EmailStr

async def send_email_async_file(email: EmailSchema, file: fileSchema=File(...)) -> JSONResponse:
    #
    file_upload=UploadFile(filename="alarmasDiariasfil.xlsx",file=file.get("file"),content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
    #file_uploadCSV=UploadFile(filename="alarmasDiariasfil.csv",file=file.get("file"),content_type='text/csv')
    
    message = MessageSchema(
        subject="prueba",
        body="alarmas diarias prueba",
        subtype='html',
        recipients=[email.get("email")],
        #attachments=[file]  if send file direct response fastapi_mail.errors.WrongFile: attachments field type incorrect, must be UploadFile or path
        attachments=[file_upload]        
        #attachments=[file_uploadCSV]        
    )
    fm = FastMail(conf)    
    await fm.send_message(message)
    return JSONResponse(status_code=200, content={"message": "email has been sent"})

Description

I am trying to send an xlmx file by mail but when converting it into a BytesIO object the final file is empty as if when executing its read () function it would not find the data. since it is in getvalue (), if I send it a stringIO with a csv it works because the data is in read ()

Get the data and transform it into stringIO or BytesIO

I call the function to send emails and I send the data, if I pass directly the Binary or string it tells me that it only accepts Uploadfile or path. that’s why I build an object of type Uploadfile and pass it to attachments

I add the csv method that if you attach the file with data. replace xls_data with csv_data

Operating System

Windows

Operating System Details

WIndows 10

FastAPI Version

0.67.0

Python Version

3.7

Additional Context

No response

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
sm-Fifteencommented, Aug 5, 2021

You might want to test your Pandas options by saving your dataframe to a local file instead of a BytesIO buffer. As far as I can tell, the snippet I posted above should be sending the whole file, so you may want to compare a locally-saved file with the output of your API to see if there’s any noticeable differences between the files. If the local file doesn’t work either, then it’s probably not a FastAPI issue.

1reaction
sm-Fifteencommented, Aug 4, 2021

I believe ExcelWriter does some finalization work when exiting the context manager, so it’s probably not a good idea to seek on the buffer while inside it.

def get_xls_data(df):
    output = IO()
    with pd.ExcelWriter(output,  engine='openpyxl', options={'strings_to_numbers': False}) as writer:
        df = df.replace(np.nan, '', regex=True)
        df = df.astype(str)
        df.reset_index(drop=True).to_excel(writer, 'AlarmaDiaria', index=False)
        writer.save()

    output.seek(0)
    return output
Read more comments on GitHub >

github_iconTop Results From Across the Web

Developers - Empty file when sending an XLSX or BytesIO -
First Check. [X] I added a very descriptive title to this issue. [X] I used the GitHub search to find a similar issue...
Read more >
Discord sending blank text file with BytesIO - Stack Overflow
Save this question. Show activity on this post. I'm trying to send a text file with discord.py, but when I send the file,...
Read more >
pandas.ExcelWriter — pandas 1.5.2 documentation
Class for writing DataFrame objects into excel sheets. Default is to use: xlwt for xls files. xlsxwriter for xlsx files if xlsxwriter is...
Read more >
Request Files - FastAPI
This is because uploaded files are sent as "form data". Import File ¶. Import File and UploadFile from fastapi : from fastapi import...
Read more >
TDD Excel File Download API with Flask | by Yifeng Hou
Recently, I have just “TDDed” a file download feature in python ... So let's create an empty Flask app first. ... from io...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found