Combining paths from in hydra configurations

  Kiến thức lập trình

I am using hydra for configuration management.

When my configuration files contain parts of paths (folder names, file names), what is the most convenient way to create a full path to a given file? As an example:

I have a config.yaml that contains some folder names:

paths:
  data:
    base: data # top-level data folder in my project
    external: external # data as loaded from other sources
    raw: raw # data as in external, but stored as binaries

dataset:

My configuration-files in dataset contain file names, for example

files:
  train: train_data
  test: test_data

Finally, I have defined my own subclasses of DictConfig for better type hinting in my editor.

What I would like:

>>> cfg.dataset.files.train
Path("base/raw/train_data")

But currently, I always have to write

>>> Path(cfg.paths.data.base)/cfg.paths.data.raw/cfg.dataset.files.train
Path("base/raw/train_data")

Is there a hydra-based solution? Or should I rather create the full filepaths in my `DictConfig` subclasses? Or should I completely abandon storing filepaths in configs?

LEAVE A COMMENT