How can response status code make sense for multiple media types for error status codes

If we follow the OAS3 spec for Response here we can see that each response status code can have multiple media types and each media type inturn has a schema particular to it. For example below, we can see 200 has a binary stream response but 400 has 2 media-types:application/json, application/xml.

So is client expected to request accept-type header with all the media-types mentioned below. How can we have specific media-type for 400 response code, or basically how we can convey to the REST Service to respond with media type as application/xml when its a 400 bad request if 200 is returning a binary stream.

Does this OAS3 response multiple media-type make sense for Client/Server Errors. If yes then whats the accept-type set for expecting, say "application/xml" for 400 bad request

        '200':
          description: Document content from the Archive
          content:
            application/octet-stream:
              schema:
                type: string
                format: binary
        '400':
          description: One or more input parameters are invalid
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  errors:
                    $ref: "#/components/schemas/Error_400"
            application/xml:
              schema:
                type: "object"
                properties:
                  errors:
                    $ref: "#/components/schemas/Error_400"
        '403':
          description: Consumer doesn't have access to the document
          content:
            application/xml:
              schema:
                type: "object"
                properties:
                  errors:
                    $ref: "#/components/schemas/Error_403"