Error in stacking with spatial resampling: response is not a factor at two levels but ‘family = Binomial()
I’ve coded a graph that performs stacking using the mlr3 package. In the first step, I tuned the parameters of the level 0 learners, and in the final step, I used the predictions from the tuned level 0 learners to obtain the predictions of an ensemble learner. However, when I ran my code, I received this error message:
How to simultaneously perform feature selection and hyperparameter tuning using multiple performance measures?
I am exploring the possibility of simultaneously performing feature selection and hyperparameter tuning using multiple performance measures. I am building ecological niche models, and it is advised to use multiple performance measures rather than just one. I came across this post, but it only uses a single performance measure. One of the comments mentioned: “For each specific feature subset, tune the hyperparameters of the learner in the inner resampling loop, and after you tune them, evaluate them in the middle resampling loop to select the optimal feature subset.”
In the mlr3
book, an example is given for multi-objective tuning:
Error with ‘classif.svm’ learner while tuning parameters : <simpleError: '’ does not support param sets with dependencies!>
I am tuning parameters for multiple learners using parallel computing. However, I am encountering this error message for the learner ‘classif.svm
‘:
Spatial resampling in stacking pipelines
I’ve coded a graph that performs stacking using the mlr3
package. The original code can be found here using a reproducible example. In summary, in a first step, I tuned the parameters of the level 0 learners, and in a final step, I used the predictions from the tuned level 0 learners to obtain the predictions of an ensemble learner (i.e., the averaged predictions of the level 0 learners). For the final step, I used mlr3pipelines::LearnerClassifAvg
as follows:
Parallelization of stacked learner tuning using the ti() function from the mlr3 package
I am trying to integrate parallel computing into the stacked learner tuning using the ti()
function from the mlr3tuning package. I have included two levels: level 0 allows tuning of 9 learners, and level 1 calculates the average of the predictions from the 9 learners. I included 9 learners: classif.gausspr, classif.glmnet, classif.rpart, classif.kknn, classif.svm, classif.gbm, classif.xgboost, classif.ranger, classif.nnet
. In the mlr3
book, the example given to illustrate parallel tuning does not apply to stacked learners. Therefore, I am not sure if I am doing things correctly, especially since my script takes a long time to run (currently, I cannot get the results from my script). I have included the main code of my script below.
Stacked learner tuning and parallel computing using the ti() function from the mlr3 package
I’m trying to tune a stacked learner using the mlr3
package. My original code can be found in this post, where I used the auto_tuner
function. In the new version, I removed the prediction threshold tuning po("threshold")
, but most importantly, I used the ti
function to be able to consider multiple performance metrics: