In order to create a personal evaluation function with catboost for binary classification, I used the example mentioned here: /questions/65462220/how-to-create-custom-eval-metric-for-catboost