Swagger throwing 401 with valid bearer token

Created a .NET Web API with swagger (Swashbuckle v5.6.0) and enabled apikey support

EnableApiKeySupport("apiKey", "header");

I got the definitions loaded but on providing strict HTTPS direct and with valid bearer token it throws 401 exceptions.  The endpoints are working perfectly from fiddler or postman or from the application.  What am I missing here?