ctypes “crashes” while trying to return pointer

I am currently trying to get a pointer element back from .dll library (compiled using cygwin and gcc – Source code is in C). But when executing the C function inside of python. The programm just “crashes” without any errors. (I am running Windows 11 Pro 23H2)

— Python Code —

import win32api, win32con, ctypes

path = r"PATH_TO_DLL_FILE" (Path is absolute)

dll_handle = win32api.LoadLibraryEx(path, 0, win32con.LOAD_WITH_ALTERED_SEARCH_PATH)
lib = ctypes.CDLL(path, handle=dll_handle)

lib.getArray.restype = ctypes.POINTER(ctypes.c_int)

print("TEST")

result = lib.getArray()

print("NICE")

print(type(result))
print(result)

— Output using python 3.12.4 —

TEST

— C File —

#include <stdlib.h>
#include <stdio.h>

int* getArray() {
    int* arr = (int*)malloc(10 * sizeof(int));

    if (arr == NULL) {
        return NULL; // Handle allocation failure
    }

    for (int i = 0; i < 10; i++) {
        arr[i] = i;
        printf("%dn", arr[i]);
    }

    return arr;
}

void freeMem(int *arr) {
    free(arr);
}

int main(int argc, char const *argv[]) {
    return 0;
}

— Cmd for compiling —

gcc -shared -o test.dll test.c

I try everything I could find online to it (which was sadly not that much, but I think that is also because calling me a hobby-c-programmer would be to much. I primarily use python, but opted to use c for the speed (The finall code is for a microcontroller)). But what I can tell you is that importing the library just with ctypes.CDLL didnt work for me (absolute or relativ path) and only the approach with LoadLibraryEx did. But generell any function which doesnt return a pointer seems to be working just fine.

(The current C Code with the pointer compiled to a .exe file then being run works also fine)

If anyone could help, it would greatly appreciated 🙂

functions in a C library intended to called by an external process need to be marked as exportable in source code so that they are placed into the .export table within the new library. Also, a main() function is optional when creating a .dll in C, so you can write your functions like this in the .c file:

#include <stdio.h>
#include <stdlib.h>

__declspec(dllexport) int* getArray() {
   int* arr = malloc(10 * sizeof(int));//casting is unnecessary in C 
   ...
   return arr
}
__declspec(dllexport) void freeMem(int* arr) {
    ...
}

From a command prompt in Windows:
gcc -shared -o test.dll test.c (minimal)
gcc -shared -v -Wl,-v -o mydll.dll mydll.c (more from compiler and linker)

I am not strong with Python, but the steps are essentially as follows:

import ctypes
//load dll
dll_handle = ctypes.CDLL('./test.dll')

//declars return and argument types
dll_handle.getArray.restype = ctypes.POINTER(ctypes.c_int)
dll_handle.freeMem.argtypes = [ctypes.POINTER(ctypes.c_int)]

//get array
result = dll_handle.getArray()

//do something with data
for i in range...

//free memory
dll_handle.freeMem(result)

Listing [Python.Docs]: ctypes – A foreign function library for Python.

Regarding the .dll not being able to load, here are some pointers:

[SO]: Discover missing module using command-line (“DLL load failed” error) (@CristiFati’s answer)
[SO]: Load a DLL with dependencies in Python (@CristiFati’s answer)
[SO]: Can’t import dll module in Python (@CristiFati’s answer)

I assume that you’re using Python for Win (installed from [Python]: Download the latest version for Windows or Microsoft Store). That is being built using VStudio ([Python.Wiki]: Windows Compilers). Mixing VStudio and Cygwin (or MSYS2, or other such environments) in the same process is unsupported (Undefined Behavior).

I prepared a (complex) example.

dll00.c:

#include <stdio.h>
#include <stdlib.h>

#if defined(_WIN32)
#  define DLL00_EXPORT_API __declspec(dllexport)
#else
#  define DLL00_EXPORT_API
#endif


#if defined(__cplusplus)
extern "C" {
#endif

DLL00_EXPORT_API int* getArray(int dim);
DLL00_EXPORT_API void freeArray(int *pArr);
DLL00_EXPORT_API void test();

#if defined(__cplusplus)
}
#endif


int* getArray(int dim)
{
    int* arr = (int*)malloc(dim * sizeof(int));
    if (arr == NULL) {
        return NULL; // Handle allocation failure
    }
    for (int i = 0; i < dim; i++) {
        arr[i] = i;
        //printf("C - %dn", arr[i]);
    }
    return arr;
}


void freeArray(int *pArr)
{
    free(pArr);
}


void test()
{
    int i = 0;
    i++;
    //printf("testn");
}

code00.py:

#!/usr/bin/env python

import ctypes as cts
import os
import sys


IntPtr = cts.POINTER(cts.c_int)

DLL_NAME_PAT = "./libdll00_{:s}.dll"


def main(*argv):
    if not argv:
        print("An argument (library build suffix) must be provided")
        sys.exit(-1)
    plat = argv[0].lower()
    dll_name = DLL_NAME_PAT.format(plat)
    uname = getattr(os, "uname", lambda: ("win32",))
    print("uname:", uname()[0])
    print("Load dll:", os.path.basename(dll_name))
    dll = cts.CDLL(dll_name, winmode=0)
    getArray = dll.getArray
    getArray.argtypes = (cts.c_int,)
    getArray.restype = IntPtr
    freeArray = dll.freeArray
    freeArray.argtypes = (IntPtr,)
    freeArray.restype = None
    test = dll.test
    test.argtypes = ()
    test.restype = None

    dim = 10
    test()
    print("Start...")
    ptr = getArray(dim)
    print("Pointer:", ptr)
    if ptr:
        for i in range(dim):
            print(ptr[i], end=" ")
    print()
    freeArray(ptr)


if __name__ == "__main__":
    print(
        "Python {:s} {:03d}bit on {:s}n".format(
            " ".join(elem.strip() for elem in sys.version.split("n")),
            64 if sys.maxsize > 0x100000000 else 32,
            sys.platform,
        )
    )
    rc = main(*sys.argv[1:])
    print("nDone.n")
    sys.exit(rc)

I’ll be using 4 systems (Cygwin, MSYS2, MinGW32, VStudio) to build the .dll and 3 consoles (Cygwin, MSYS2, Cmd) to run.

Output:

Cygwin:

[cfati@cfati-5510-0:/cygdrive/e/Work/Dev/StackExchange/StackOverflow/q078975409]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[064bit prompt]> uname
CYGWIN_NT-10.0-19045
[064bit prompt]> ls
code00.py  dll00.c  main00.c
[064bit prompt]>
[064bit prompt]> gcc -fPIC -shared -o libdll00_cyg.dll dll00.c
[064bit prompt]>
[064bit prompt]> python ./code00.py cyg
Python 3.9.16 (main, Mar  8 2023, 22:47:22) [GCC 11.3.0] 064bit on cygwin

uname: CYGWIN_NT-10.0-19045
Load dll: libdll00_cyg.dll
Start...
Pointer: <__main__.LP_c_int object at 0x6fffffe0cdc0>
0 1 2 3 4 5 6 7 8 9

Done.

MSYS2:

[cfati@cfati-5510-0:/e/Work/Dev/StackExchange/StackOverflow/q078975409]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[064bit prompt]> uname
MSYS_NT-10.0-19045
[064bit prompt]> ls
code00.py  dll00.c  libdll00_cyg.dll  main00.c
[064bit prompt]>
[064bit prompt]> gcc -fPIC -shared -o libdll00_msys.dll dll00.c
[064bit prompt]>
[064bit prompt]> python ./code00.py msys
Python 3.12.5 (main, Aug  9 2024, 15:03:05) [GCC 13.3.0] 064bit on cygwin

uname: MSYS_NT-10.0-19045
Load dll: libdll00_msys.dll
Start...
Pointer: <__main__.LP_c_int object at 0x6fffffdc6f50>
0 1 2 3 4 5 6 7 8 9

Done.

Cmd:

[cfati@CFATI-5510-0:e:WorkDevStackExchangeStackOverflowq078975409]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###

[prompt]> dir /b
code00.py
dll00.c
libdll00_cyg.dll
libdll00_msys.dll
main00.c

[prompt]> :: Setup VStudio
[prompt]> "c:Installpc064MicrosoftVisualStudioCommunity2022VCAuxiliaryBuildvcvarsall.bat" x64 > nul

[prompt]> set _PATH=%PATH%

[prompt]> :: Add Cygwin (cygwin1.dll), MSYS2 (msys-2.0.dll) and MinGW32 (GCC executable and libraries) to PATH
[prompt]> set PATH=%_PATH%;c:Installpc064CygwinCygwinVersionbin;f:Installpc064MSYS2MSYS2Versionusrbin;f:InstallQtQtToolsmingw730_64bin

[prompt]> "f:InstallQtQtToolsmingw730_64bingcc.exe" -fPIC -shared -o libdll00_mgw.dll dll00.c

[prompt]> cl /nologo /MD /DDLL dll00.c  /link /NOLOGO /DLL /OUT:libdll00_vs.dll
dll00.c
   Creating library libdll00_vs.lib and object libdll00_vs.exp

[prompt]>
[prompt]> dir /b
code00.py
dll00.c
dll00.obj
libdll00_cyg.dll
libdll00_mgw.dll
libdll00_msys.dll
libdll00_vs.dll
libdll00_vs.exp
libdll00_vs.lib
main00.c

[prompt]>
[prompt]> "e:WorkDevVEnvspy_pc064_03.12_test0Scriptspython.exe" code00.py cyg
Python 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] 064bit on win32

uname: win32
Load dll: libdll00_cyg.dll
Start...
Traceback (most recent call last):
  File "e:WorkDevStackExchangeStackOverflowq078975409code00.py", line 53, in <module>
    rc = main(*sys.argv[1:])
         ^^^^^^^^^^^^^^^^^^^
  File "e:WorkDevStackExchangeStackOverflowq078975409code00.py", line 36, in main
    ptr = getArray(dim)
          ^^^^^^^^^^^^^
OSError: exception: access violation writing 0x0000000000000000

[prompt]>
[prompt]> "e:WorkDevVEnvspy_pc064_03.12_test0Scriptspython.exe" code00.py msys
Python 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] 064bit on win32

uname: win32
Load dll: libdll00_msys.dll
Start...
Traceback (most recent call last):
  File "e:WorkDevStackExchangeStackOverflowq078975409code00.py", line 53, in <module>
    rc = main(*sys.argv[1:])
         ^^^^^^^^^^^^^^^^^^^
  File "e:WorkDevStackExchangeStackOverflowq078975409code00.py", line 36, in main
    ptr = getArray(dim)
          ^^^^^^^^^^^^^
OSError: exception: access violation writing 0x0000000000000000

[prompt]>
[prompt]> "e:WorkDevVEnvspy_pc064_03.12_test0Scriptspython.exe" code00.py mgw
Python 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] 064bit on win32

uname: win32
Load dll: libdll00_mgw.dll
Start...
Pointer: <__main__.LP_c_long object at 0x000001FF0B903450>
0 1 2 3 4 5 6 7 8 9

Done.


[prompt]>
[prompt]> "e:WorkDevVEnvspy_pc064_03.12_test0Scriptspython.exe" code00.py vs
Python 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] 064bit on win32

uname: win32
Load dll: libdll00_vs.dll
Start...
Pointer: <__main__.LP_c_long object at 0x0000019691E83450>
0 1 2 3 4 5 6 7 8 9

Done.

Notes:

In the above (Cmd) output, for Cygwin and MSYS2, I’m getting Access Violation (SegFault). For older Python versions I got (depending on runs) following results:
1. Crashing (last output: Start…)
2. Freezing (similar to the above but the Python process was hanging, eating a full (logical) CPU – had to kill it from Task Manager)
3. Working
which is a clear Undefined Behavior indicator
Each system works independently (MinGW32 also works from (Win) Python)
Apparently, the problem occurs when a C library function is called from the .dll (as seen, when test function doesn’t contain the printf call, it works, uncommenting the line, will trigger a crash)
I didn’t test all the combinations because I simply don’t care about them

This isn’t related to Python. The same thing can be achieved using the following program (main00.c):

#include <stdio.h>
#include <Windows.h>


typedef void (*TestPtr)();
typedef int* (*GetArrayPtr)(int);
typedef void (*FreeArrayPtr)(int*);


int main(int argc, char **argv)
{
    if (argc <= 1) {
        printf("An argument (library build suffix) must be providedn");
        return -1;
    }
    char dll_name[0xFF] = "./libdll00_";
    strcat(dll_name, argv[1]);
    strcat(dll_name, ".dll");
    HMODULE hmod = LoadLibrary(dll_name);
    if (hmod == NULL) {
        printf("Could not load module: %dn", GetLastError());
        return -2;
    }
    TestPtr test = (TestPtr)(GetProcAddress(hmod, "test"));
    GetArrayPtr getArray = (GetArrayPtr)(GetProcAddress(hmod, "getArray"));
    FreeArrayPtr freeArray = (FreeArrayPtr)(GetProcAddress(hmod, "freeArray"));
    if ((test == NULL) || (getArray == NULL) || (freeArray == NULL)) {
        printf("Could not load one of the functionsn");
        FreeLibrary(hmod);
        return -3;
    }
    test();
    printf("Start...n");
    int dim = 10;
    int *arr = getArray(10);
    printf("Pointer: %pn", arr);
    if (arr) {
        for(int i = 0; i < dim; ++i) {
            printf("%d  ", arr[i]);
        }
        printf("n");
        freeArray(arr);
    }
    FreeLibrary(hmod);
    printf("nDone.nn");
    return 0;
}

Build it (from Cmd) using the command: cl /nologo /MD /W0 main00.c /link /NOLOGO /OUT:main_vs.exe, and run it with the same arguments like the Python script to get same results (which I’m not going to post here)

This is actually a duplicate of [SO]: Paradoxon: silent crash on Python’s ctypes.CDLL when importing, but not when running directly – how is this possible? (@CristiFati’s answer), but I didn’t mark it as such because that way I couldn’t have included:

Debug results
.dll load info (from the beginning)

Conclusion

Chose one technology and stick with it. Do not mix and match. Either:

Install VStudio (there is a (free) Community edition [MS.VStudio]: Meet the Visual Studio family), or MinGW32 as an alternative to build the .dll to be loaded in (Win) Python
Stay inside the Cygwin (or MSYS2, …) environment

Filed under: Kiến thức lập trình - @ 18:15

Thẻ: pythoncpython-3.xpointersctypes

Thiết kế website giá rẻ

Danh mục

ctypes “crashes” while trying to return pointer

Conclusion