Debugging Chronicles: Missing Lambda Invocation

TLDR;
Why would only one Lambda out of many not be executed even if the Gateway API is properly configured and executed?

Make sure every single fetch you make to the GatewayAPI passes the Cognito Authorization Token in the Headers.

I am working on a FullStack project where we recently added Cognito Authorizers on top of the Lambda functions we are invoking through GatewayAPI.

We use serverless framework and configuration is quite simple

functions:
  create-user-lambda:
    handler: create.handler
    events:
     - http:
          path: users
          method: post
          cors: true
          authorizer:
            type: COGNITO_USER_POOLS
            authorizerId:
              Ref: ApiGatewayAuthorizer
  get-user-lambda:
    handler: get.handler
    events:
     - http:
          path: users/{id}
          method: get
          cors: true
          authorizer:
            type: COGNITO_USER_POOLS
            authorizerId:
              Ref: ApiGatewayAuthorizer

  my-special-lambda:
    handler: publish.handler
    events:
     - http:
          path: publish
          method: post
          cors: true
          authorizer:
            type: COGNITO_USER_POOLS
            authorizerId:
              Ref: ApiGatewayAuthorizer

After a few days - and a bunch of other PullRequests - we noticed that only one endpoint of the API was not working when invoked by our React App.

I was sure functionality was ok: Unit Tests were all working and also the Integration Test was successful at updating the DB. Since the test invokes directly our logic bypassing the handler I needed to check the behaviour of the Lambda itself when invoked by the client.

I used serverless-offline plugin to start my serverless stack locally and then configured the frontend running on localhost:3000 to invoke the API on localhost:3001. All good.

I also tried using ngrok to expose to my running local version of the lambda to the outside world and switcheroo chrome plugin to redirect the call to the Gateway API to the Ngrok URL, so that I could test directly on the version of the App we had on QA. Again no problem.

It was obvious that there was something wrong at the Gateway API / Cognito level, especially because as awesome the serverless-offline plugin is, the local version of the stack differs in many aspects (aws credentials, roles and authtokens) from the real version on AWS.

This theory was confirmed by the fact that the logs of the Lambda were completely empty. ( you can check them in the Cloudwatch UI console or just by sls logs -f YOUR_FUNCTION -s STAGE/ENV ): that meant that the Lambda was never triggered, but since the OPTIONS handshake made by the browser just before the POST was successful, that meant that the GateWay API was properly configured...

It must have something to do with the Authorizer but the configuration was exactly the same of all other Lambdas... I quickly removed the Authorizer only on that Lambda and redeploy. Of course the Endpoint started working again as expected.
What could be wrong?!?!

I checked again the requests headers and content in the network panel of Chrome Dev tools.
The working ones looked like this:

The broken one looked like this:

It's evident that the broken lambda is not sending the Authorization Token!
The misleading thing in this was that the Request Headers on Localhost ( using serverless-offline ) were exactly the same, but endpoint was working. So checking it wasnt immediately a smell..

Then i remembererd that the frontend is using ReactAdmin and in order to have its dataprovider working (which automagically invokes the endpoints for each Resource / section of the App ) with Amplify we had to somehow override its httpClient to push the jwt token to every fetch.
Since our special Lambda was not really following the strict structure of the ReactAdmin - it was an external method bound to a button but not to a specific UI grid, we were directly using fetch there...

import { Auth } from "aws-amplify";
import restDataProvider from "ra-data-simple-rest";
import { fetchUtils } from "react-admin";
const httpClient = async (url, options = {}) => {
    options.user = {
        authenticated: true,
        token: (await Auth.currentSession()).idToken.jwtToken,
    };
    return fetchUtils.fetchJson(url, options);
};
const dataProvider = restDataProvider(process.env.REACT_APP_API_URL, httpClient);

I just added export in the const httpClient declaration and replaced fetch' withhttpClient` in that only place where we used it directly ( and not the dataProvider from ReactAdmin)
Tadaa!!.