Swagger UI uploads empty CSV (latin1 encoded)
Swagger UI uploads empty CSV if it is latin1 encoded (csv has danish characters like ø, Ø, Æ), works alright with standard utf-8 encoded CSV.
Here is swagger json:
{
"swagger": "2.0",
"info": {
"version": "2.2",
"title": "searcher"
},
"paths": {
"/search": {
"post": {
"summary": "Search endpoint uploading files manually",
"tags": [
"Calls"
],
"consumes": [
"multipart/form-data"
],
"produces": [
"text/csv"
],
"parameters": [
{
"name": "reference_file",
"in": "formData",
"type": "file",
"required": true,
"description": "CSV file containing reference data"
},
{
"name": "query_file",
"in": "formData",
"type": "file",
"required": true,
"description": "CSV file containing query data"
},
{
"name": "config_file",
"in": "formData",
"type": "file",
"required": true,
"description": "JSON file containing configuration data"
}
],
"responses": {
"200": {
"description": "Successful response",
"schema": {
"type": "string",
"example": "type,similarity,id,name,surname,age,gender,region,date_payment,amount,id_reference,name_reference,surname_reference,age_reference,gender_reference,region_reference\nsimilar_id,0.9999997,1771594086,Molly,Forsberg,50,male,Stockholm,43888,398.82,1771594087,Molly,Forsberg,50,male,Stockholm\nfeature_search,1.0000001,3519854485,Isak,Ek,25,male,Gotland,44797,819.62,3040728485,Isak,Ek,25,male,Gotland\nmatch,1,6919199375,William,Lundverg,35,male,Dalarna,43882,398.82,6919199375,William,Lundberg,35,male,Dalarna\n"
}
}
}
},
"x-swagger-router-controller": "app"
}
}
}
here is the python code that tries to read a file identified with 'query_file' in wagger json:
from flask import request @app.route('/search', methods=['POST']) def seas_query() -> Response: try: filename = 'query_file' df = pd.read_csv(request.files[filename], sep = ',', dtype = 'object') except UnicodeDecodeError: print('Unable to read {fname}, trying with latin1 encoding.'.format(fname=request.files[filename])) file_obj = request.files[filename] file_obj.save('downloaded_'+file_obj.filename) # even the downloaded file from above code is empty df = pd.read_csv('downloaded_'+file_obj.filename, delimiter=";", encoding='latin1') except Exception as e: print('Unable to read file')
the exception I receive is:
df = pd.read_csv('downloaded_'+file_obj.filename, delimiter=";", encoding='latin1')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ore\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\pandas\util\_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
...
...
...
File "pandas\_libs\parsers.pyx", line 554, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file
The csv has data inside it. Also, loads data fine when I use utf-8 encoded csv or any other csv that doesn't contain danish letters.