TL;DR: How is the generator expression $<TARGET_RUNTIME_DLLS:..> able to (recursively?) iterate over all linked targets at generation time and can I replicate the behaviour inside of cmake?

Project background

I am working on a medium sized C++ project in cmake. The project is closed source (buh) and consists of a few core shared libraries with their dependencies, Unit and Integration tests with additional test framework dependencies, a collection of samples with additional GUI- and misc. dependencies and a self-built licensing tool.

All of this is cmake based, as we release for multiple Linux distributions and on Windows with Release and Debug configurations on all platforms. Our build is modular and I try to keep up with modern cmake styles, so everything is target based.

For the (admittedly quite complex) dependency management we use Conan and it’s cmake integration with CMakeDeps and CMakeToolchain. With our own conanfile.py recipe we are able to turn off certain parts of the project from Conan, so that the required dependencies are no longer built/checked out (e.g. do not install gtest, if Testing are disabled).

We also CPack to create shipable packages/installers for our releases.

Dependencies

As already mentioned, most of our dependencies are open source and handled by Conan or at least come with their own cmake package config, so all our dependencies have their own targets. Most are thankfully built as static libs, but a few require us to build them as shared libraries. You can probably already see the problems arising.

Dependencies with their own package config work fine, they define a SHARED IMPORTED target for themselves with the correct paths written plainly in their IMPORTED_LOCATION or IMPORTED_LOCATION_<CONFIG>.

The conan built packages are a bit more complicated. They define (depending on the library) multiple internal targets per lib as SHARED IMPORTED, often with cryptic, platform specific names and then define the “official”/”public” INTERFACE IMPORTED targets (e.g. log4cplus::log4cplus) that links against the correct internal target(s) The problem is, that all relevant properties (INTERFACE_LINK_LIBRARIES, IMPORTED_LOCATION, etc.) use generator expressions to determine the correct target/filepath (e.g. “$<$<CONFIG:Release>:lib/liblog4cplus.so.2.1.0>”), so they are not known at configuration time

Dependencies in Runtime output folder

For our daily work with the project we need to run our built executable (e.g. Unit Tests) from the build output directory using a debugger. Thankfully on linux the built executable know where the shared objects are that the linker linked them with, but on Windows we need the dll in a path environment (in the case of system libraries, that is fine) or copied to the runtime output directory. A post-build command using the $<TARGET_RUNTIME_DLLS:…> genex is able to resolve all transitive dependency targets through some kind of magic, even if it goes through multiple INTERFACE IMPORTED targets that then determine the correct dll to copy to the output folder.

Dependencies in Relocatable Test packages

For some of our Test, we have to copy the built output and run it on a different machine that may have some test software installed while other development tools are deliberately not installed.

For Windows, we can just copy the runtime build output folder, as it contains all needed dlls already, but on linux the linked 3rd party libs are obviously missing. We already set the RPATH of our executables (build to bin) to “../lib” so it will look for all missing shared objects in the install folders own lib first. However, for Linux there is no “$<TARGET_RUNTIME_SO:..>” generator expression or similar.

How can I copy my targets transitive dependencies of shared libraries to my CMAKE_LIBRARY_OUTPUT directory in a post build step on Linux?

In a first naive approach I wrote a cmake function that would walk through all (INTERFACE_)LINK_LIBRARIES of my target and, depending on their TYPE, IMPORTED attribute and properties recursively go through the dependency tree returning all (IMPORTED_)LOCATIONs that could then be copied to the runtime/library folder.

However, this obviously breaks if any of these properties use generator expressions, which they sadly do. I am not aware of any way to achieve the same “get-all-transitive-dependencies” using only generator expressions, as there are no “while/foreach/call_function” structures in generator expressions

Dependencies in Install step

Because we use CPack with components to pack our build artifacts into installable packages for different platforms (deb & rpm packages, zip & tar.gz archives and NSIS installer) and CPack uses cmake install steps to create these packages, I also need to specify these dependencies in one or multiple additional install steps per target. The problem here is similar to the relocatable test packages: I can use $<TARGET_RUNTIME_DLLS:…> on Windows to pack just dependencies known to cmake which is exactly what I want, but I have no way to replicate a similar dependency resolution on Linux.

Example CMakeLists.txt

My own projects CMakeLists.txt:

cmake_minimum_required(VERSION 3.26)

project(MyProject LANGUAGES CXX C)
include(GNUInstallDirs)

find_package(SamplePackage CONFIG REQUIRED)

add_library(MyLibrary SHARED)
target_sources(MyLibrary 
    PRIVATE "main.cpp"
    PUBLIC FILE_SET header_public 
        TYPE HEADERS 
        BASE_DIRS "include"
        FILES "include/MyLibrary/header.h"
)

target_link_libraries(MyLibrary PRIVATE sample_package::sample_package)

## On windows, i can do:
# Post build 
set(_HAVE_FILES_TO_COPY $<BOOL:$<TARGET_RUNTIME_DLLS:MyLibrary>>)
set(_COPY_FILES_COMMAND ${CMAKE_COMMAND} -E copy_if_different $<TARGET_RUNTIME_DLLS:MyLibrary> $<TARGET_FILE_DIR:MyLibrary>)
add_custom_command(TARGET MyLibrary POST_BUILD
    COMMAND "$<${_HAVE_FILES_TO_COPY}:${_COPY_FILES_COMMAND}>"
    COMMAND_EXPAND_LISTS
)
# Install command
install(FILES $<TARGET_RUNTIME_DLLS:MyLibrary>
    DESTINATION ${CMAKE_INSTALL_BINDIR}
    COMPONENT RuntimeDependencies
)
# But how to achieve the same on Linux without TARGET_RUNTIME_DLLS?

# Own lib install commands
install(TARGETS MyLibrary 
    EXPORT ${PROJECT_NAME}Targets
    RUNTIME
        DESTINATION ${CMAKE_INSTALL_BINDIR}
        COMPONENT Runtime
    LIBRARY 
        DESTINATION ${CMAKE_INSTALL_LIBDIR}
        COMPONENT Runtime
    ARCHIVE 
        DESTINATION ${CMAKE_INSTALL_LIBDIR}
        COMPONENT Development
    INCLUDES 
        DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
        COMPONENT Development
    FILE_SET header_public
        DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
        COMPONENT Development
)

install(EXPORT ${PROJECT_NAME}Targets
    FILE ${PROJECT_NAME}Targets.cmake
    NAMESPACE ${PROJECT_NAME}::
    DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}
    COMPONENT Development
)

#CPack logic omitted

Auto Generated SamplePackageConfig.cmake by Conan, simplified as much as possible for readability. Conan usually generates seperate files per platform, instead I opted for if(WIN32)...else()#Linux... sections

add_library(sample_package::sample_package INTERFACE IMPORTED)

add_library(SamplePackage::CoreLib INTERFACE IMPORTED)

add_library(SamplePackage::OtherLib SHARED IMPORTED)
add_library(CONAN_LIBS::samplepackage_CoreLib_2_1d_Debug_TARGET SHARED IMPORTED) # //<- internal target name that can change on platform, package version, conan version, configuration and potentially even compiler


if(WIN32)
    set_target_properties(CONAN_LIBS::samplepackage_CoreLib_2_1d_Debug_TARGET
        PROPERTIES 
            IMPORTED_LOCATION_DEBUG "$<$<CONFIG:Debug>:/absolute/path/to/bin/corelib.dll>"
            IMPORTED_IMPLIB_DEBUG "/absolute/path/to/lib/corelib.lib"
    )
    set_target_properties(SamplePackage::OtherLib
        PROPERTIES 
            IMPORTED_LOCATION_DEBUG "/absolute/path/to/bin/otherlib.dll"
            IMPORTED_IMPLIB_DEBUG "/absolute/path/to/lib/otherlib.lib"
    )
else()
    set_target_properties(CONAN_LIBS::samplepackage_CoreLib_2_1d_Debug_TARGET
        PROPERTIES 
            IMPORTED_LOCATION_DEBUG "$<$<CONFIG:Debug>:/absolute/path/to/lib/libcorelib.so>"
    )
    set_target_properties(SamplePackage::OtherLib
        PROPERTIES 
            IMPORTED_LOCATION_DEBUG "/absolute/path/to/lib/libotherlib.so"
    )
endif()

set_property(TARGET SamplePackage::CoreLib APPEND PROPERTY INTERFACE_LINK_LIBRARIES
    $<$<CONFIG:Debug>:CONAN_LIBS::samplepackage_CoreLib_2_1d_Debug_TARGET>
)
set_property(TARGET sample_package::sample_package APPEND PROPERTY INTERFACE_LINK_LIBRARIES SamplePackage::CoreLib)
set_property(TARGET sample_package::sample_package APPEND PROPERTY INTERFACE_LINK_LIBRARIES SamplePackage::OtherLib)

Not working, already tried

Apart from trying to create my own monster function to traverse the dependency tree of my targets during cmakes configuration time (which failes due to generator expressions used on target properties) I also tried a few other things:

1. file(GET_RUNTIME_DEPENDENCIES …): returns a billion system libraries from all PATH locations for all possible compilation settings resulting in gigabytes of unwanted libs, that are present on every machine anyway.
2. install(IMPORTED_RUNTIME_ARTIFACTS …): Needs to know the exact SHARED IMPORTED library targets, cannot determine transitive dependencies
3. install(TARGETS <targets..> RUNTIME_DEPENDENCIES …): same as 1.
4. BundleUtilities’ fixup_bundle(…): Does only work with executables, also includes system libs and seems to be intended for MacOS (?). Could not get this to work properly.
5. include(InstallRequiredSystemLibraries): As the name suggests, this includes only the generic system libs.
6. using $<GENEX_EVAL:…>: Maybe I did not use it right, but it was unable to resolve the generator expressions for configuration time functions.