I’m new in MLOps and trying to figure out how to work with label feature in data. I read that for the uniformity of the data it is necessary to use the same schema for both the training and validation sets.
My question is: How can I mark the label as optional for the validation set? If I received data from users that will not have a label feature, how do I compare the schemas of the new data (without label) with the original schema that contains the label by example_validator?
I know that it can be done by tfdv.get_feature(schema, 'labels').not_in_environment.append('SERVING'), but as far as I know it is not solution for production pipeline.
Another thoughts: using preprocessing function in Transform delete label feature from validation set, but I don’t really understand how context.pipeline.stage works. Example: