Forum Discussion

erehoba's avatar
erehoba
Occasional Visitor
10 months ago

Swagger UI uploads empty CSV (latin1 encoded)

Swagger UI uploads empty CSV if it is latin1 encoded (csv has danish characters like ø, Ø, Æ), works alright with standard utf-8 encoded CSV.

Here is swagger json:

{
  "swagger": "2.0",
  "info": {
    "version": "2.2",
    "title": "searcher"
  },
  "paths": {
    "/search": {
      "post": {
        "summary": "Search endpoint uploading files manually",
        "tags": [
          "Calls"
        ],
        "consumes": [
          "multipart/form-data"
        ],
        "produces": [
          "text/csv"
        ],
        "parameters": [
          {
            "name": "reference_file",
            "in": "formData",
            "type": "file",
            "required": true,
            "description": "CSV file containing reference data"
          },
          {
            "name": "query_file",
            "in": "formData",
            "type": "file",
            "required": true,
            "description": "CSV file containing query data"
          },
          {
            "name": "config_file",
            "in": "formData",
            "type": "file",
            "required": true,
            "description": "JSON file containing configuration data"
          }
        ],
        "responses": {
          "200": {
            "description": "Successful response",
            "schema": {
              "type": "string",
              "example": "type,similarity,id,name,surname,age,gender,region,date_payment,amount,id_reference,name_reference,surname_reference,age_reference,gender_reference,region_reference\nsimilar_id,0.9999997,1771594086,Molly,Forsberg,50,male,Stockholm,43888,398.82,1771594087,Molly,Forsberg,50,male,Stockholm\nfeature_search,1.0000001,3519854485,Isak,Ek,25,male,Gotland,44797,819.62,3040728485,Isak,Ek,25,male,Gotland\nmatch,1,6919199375,William,Lundverg,35,male,Dalarna,43882,398.82,6919199375,William,Lundberg,35,male,Dalarna\n"
            }
          }
        }
      },
      "x-swagger-router-controller": "app"
    }
  }
}


here is the python code that tries to read a file identified with 'query_file' in wagger json:

from flask import request
@app.route('/search', methods=['POST'])
def seas_query() -> Response:
    try:
        filename = 'query_file'
        df = pd.read_csv(request.files[filename], sep = ',', dtype = 'object')
    except UnicodeDecodeError:
        print('Unable to read {fname}, trying with latin1 encoding.'.format(fname=request.files[filename]))

        file_obj = request.files[filename]
        file_obj.save('downloaded_'+file_obj.filename)
        # even the downloaded file from above code is empty
        df = pd.read_csv('downloaded_'+file_obj.filename, delimiter=";", encoding='latin1')
    except Exception as e:
        print('Unable to read file')


the exception I receive is:

 

df = pd.read_csv('downloaded_'+file_obj.filename, delimiter=";", encoding='latin1')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ore\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\pandas\util\_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
...
...
...
File "pandas\_libs\parsers.pyx", line 554, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

The csv has data inside it. Also, loads data fine when I use utf-8 encoded csv or any other csv that doesn't contain danish letters.

No RepliesBe the first to reply