First, I understand that the connection pool with RDS is managed on RDS Proxy.
Therefore, I changed the endpoint toward RDS in the AWS Lambda function to the endpoint of RDS Proxy.
From what I understand, because the connection pool is managed by RDS Proxy, when I use the SHOW PROCESSLIST command in the DB, only the connection information for the corresponding pool should be displayed.