Swagger UI uploads empty CSV (latin1 encoded)
Swagger UI uploads empty CSV if it is latin1 encoded (csv has danish characters like ø, Ø, Æ), works alright with standard utf-8 encoded CSV.
Here is swagger json:
{ "swagger": "2.0", "info": { "version": "2.2", "title": "searcher" }, "paths": { "/search": { "post": { "summary": "Search endpoint uploading files manually", "tags": [ "Calls" ], "consumes": [ "multipart/form-data" ], "produces": [ "text/csv" ], "parameters": [ { "name": "reference_file", "in": "formData", "type": "file", "required": true, "description": "CSV file containing reference data" }, { "name": "query_file", "in": "formData", "type": "file", "required": true, "description": "CSV file containing query data" }, { "name": "config_file", "in": "formData", "type": "file", "required": true, "description": "JSON file containing configuration data" } ], "responses": { "200": { "description": "Successful response", "schema": { "type": "string", "example": "type,similarity,id,name,surname,age,gender,region,date_payment,amount,id_reference,name_reference,surname_reference,age_reference,gender_reference,region_reference\nsimilar_id,0.9999997,1771594086,Molly,Forsberg,50,male,Stockholm,43888,398.82,1771594087,Molly,Forsberg,50,male,Stockholm\nfeature_search,1.0000001,3519854485,Isak,Ek,25,male,Gotland,44797,819.62,3040728485,Isak,Ek,25,male,Gotland\nmatch,1,6919199375,William,Lundverg,35,male,Dalarna,43882,398.82,6919199375,William,Lundberg,35,male,Dalarna\n" } } } }, "x-swagger-router-controller": "app" } } }
here is the python code that tries to read a file identified with 'query_file' in wagger json:
from flask import request @app.route('/search', methods=['POST']) def seas_query() -> Response: try: filename = 'query_file' df = pd.read_csv(request.files[filename], sep = ',', dtype = 'object') except UnicodeDecodeError: print('Unable to read {fname}, trying with latin1 encoding.'.format(fname=request.files[filename])) file_obj = request.files[filename] file_obj.save('downloaded_'+file_obj.filename) # even the downloaded file from above code is empty df = pd.read_csv('downloaded_'+file_obj.filename, delimiter=";", encoding='latin1') except Exception as e: print('Unable to read file')
the exception I receive is:
df = pd.read_csv('downloaded_'+file_obj.filename, delimiter=";", encoding='latin1')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ore\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\pandas\util\_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
...
...
...
File "pandas\_libs\parsers.pyx", line 554, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file
The csv has data inside it. Also, loads data fine when I use utf-8 encoded csv or any other csv that doesn't contain danish letters.