EsoErik

Tuesday, June 24, 2014

 

The Dread Error: unresolved external symbol __imp_PyModule_Create2 referenced in [blah]

On Windows, if you go through the immense pain of making a real debug Python build, you will find that absolutely no extensions that require the Python C API can build.  IE, a Python debug build is totally hosed and basically worthless.  All builds will fail with:

unresolved external symbol __imp_PyModule_Create2 referenced in [blah]

Googling for this error yields one hit: this post.  Terrific!!!!! My conclusion is that nobody anywhere has run a Python debug build even once on Windows.  Not a single time, not ever.  This is extremely disturbing for many reasons.  At any rate, I solved this problem the first time around by, get this: building every single Python .c file into my Python extension.

I have since found a better solution.  The part Python leaves out of Debug builds in order to be a gigantic jerk is python3.dll (not to be confused with python34.dll or somesuch, mind you).  This is some sort of "stable ABI" thing that I'm sure is really great for keeping backwards binary compatibility, I thing I do not want or care for at this moment.  But, we have to have it, or extensions won't build.  One may try to build it from the PCBuild.sln file.  This didn't work for me.  Instead, I:

D:
cd Python\CPython\PC
nmake /f python3.mak MACHINE=x64 OutDir=D:\Python\CPython\PCbuild\amd64\
 
After that, I copied the new and shiny python3.dll and .lib files outputted to D:\Python\CPython\PCbuild\amd64 from there to D:\Python\CPython\libs where extensions for some reason insist upon looking.

From there, you need to make sure that all extensions link against that python3.lib file.  Which they don't do by default.  Somehow, things building against the release build know to link against that... perhaps it is normally a dependency of python34.dll but not python34_d.dll or some other horrible #pragma windows garbage.

For the record, the stock advice, "just use the official CPython .pdb files," is worthless to me.  I have to actually step through what the interpreter is really doing in order to solve difficult foreign function interface issues.  Line by line, accurately, with correct stack information.  *this frame optimized out* *this frame optimized out* *this frame optimized out* *this frame optimized out* *this frame optimized out* *this frame optimized out* *this frame optimized out* *this frame optimized out*  is not informative.  I don't have time for half the things I ought to be doing as it is, damn it.  Why must anything related to Python extension building and, heaven help you, debugging, be so hugely time wasting?

PS: command line for linking numpy against debug python, since distutils doesn't work:
icl -O3 -fp=strict -openmp -QxHost /Qstd=c99 build\temp.win-amd64-3.4-pydebug\Debug\numpy\core\src\dummymodule.obj
-LD:\Python\CPython\libs -LD:\Python\CPython\PCbuild\amd64 -Lbuild\temp.win-amd64-3.4-pydebug 
D:\Python\CPython\PCbuild\amd64\python34_d.lib \-o build\lib.win-amd64-3.4-pydebug\numpy\core\_dummy.pyd 
-link -nodefaultlib:python34.lib -dll D:\Python\CPython\PCbuild\amd64\python3.lib

Also, I needed to modify numpy/distutils/ccompiler.py near line 102:

   for src_name in source_filenames:
        base, ext = os.path.splitext(os.path.normpath(src_name))
        base = os.path.splitdrive(base)[1] # Chop off the drive
        base = base[os.path.isabs(base):]  # If abs, chop off leading /
        if base.startswith('..'):
            # Resolve starting relative path components, middle ones
            # (if any) have been handled by os.path.normpath above.
            i = base.rfind('..')+2
            d = base[:i]
            d = os.path.basename(os.path.abspath(d))
            base = d + base[i:]
        if ext not in self.src_extensions:
            raise UnknownFileError("unknown file type '%s' (from '%s')" % (ext, src_name))
        if strip_dir:
            base = os.path.basename(base)
        if self.obj_extension == '.o':
            self.obj_extension = '.obj'
            print('distutils unbreakage: replacing .o with .obj')

        obj_name = os.path.join(output_dir, base + self.obj_extension)
        obj_names.append(obj_name)
    return obj_names


Apparently, fundamental design aspects of distutils completely prevent it from supporting .obj object filenames on Windows, or something.  Presumably that MKL windows binaries guy also hacks up his numpy sources in order to do a build.

Archives

July 2009   August 2009   September 2009   October 2009   November 2009   December 2009   January 2010   September 2010   December 2010   January 2011   February 2011   April 2011   June 2011   August 2011   February 2012   June 2012   July 2012   August 2012   October 2012   November 2012   January 2014   April 2014   June 2014   August 2014   September 2014   October 2014   January 2015   March 2015   April 2015   June 2015   November 2015   December 2015   January 2016   June 2016   August 2016   January 2017   March 2017   April 2018   April 2019   June 2019   January 2020  

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]