dm.xmlsec.binding broken?

[navneil]

I can't get dm.xmlsec.binding to work on my fedora host. It just core dumps. Has anyone gotten this to work. As far as I can tell, I have everything installed cleanly. I find dm.xmlsec.binding to be very unreliable and modified python-saml to use the xmlsec1 binary instead.

[pitbulk]

@navneil , Why do you say dm.xmlsec.binding is unreliable?

dm.xmlsec.binding is used in dm.zope.saml2. After review many libraries, I choosedm.xmlsec.binding.
It has documentation https://pypi.python.org/pypi/dm.xmlsec.binding/1.3.1 and is maintained.

What would you propose?

[navneil]

I can't get dm.xmlsec.binding to work on different linux distributions. Even after installing cleanly, it core dumps on import dm.xmlsec.binding. What distribution are you using?

Also, have you thought of using the xmlsec1 binaries instead of the python bindings.

[pitbulk]

I tested it on Redhat and Ubuntu. I dislike the use of xmlsec1 binaries execution.
How are you installing the python-saml library? Do you use pip?

Can you take a look on this Travis trace? https://travis-ci.org/onelogin/python-saml As you see is installed as expected.

Also, take a look on the installation section of: https://pypi.python.org/pypi/dm.xmlsec.binding/1.3.1

What you get after those commands?

sudo yum install python-setuptools python-setuptools-devel python-pip
pip install python-saml

[cjsteinke]

Hi, I found this during a google search and I am having the same problem.
I get a segmentation fault when I try to load the module as well.

This is on RHEL6.6, I followed the instructions from the Travis trace (with modifications for RHEL) and ensured all the necessary packages are installed.

In order to get past the M2Crypto failures I needed to download the package from the python web site and then apply the fedora_setup from the git clone of M2Crypto (which is an older version than the one available from python pypi)because it's NOT included with the download from pypi.

I also needed to install, which isn't mentioned anywhere, xmlsec1-openssl, xmlsec1-openssl-devel, libtool-ltdl and libtool-ltdl-devel (added the -devel packages to be on the safe side) Thi

Here is output from the coverage run

root@gandalf # coverage run --source=src/onelogin/saml2 --rcfile=tests/coverage.rc setup.py test
running test
running egg_info
writing requirements to src/python_saml.egg-info/requires.txt
writing src/python_saml.egg-info/PKG-INFO
writing top-level names to src/python_saml.egg-info/top_level.txt
writing dependency_links to src/python_saml.egg-info/dependency_links.txt
reading manifest file 'src/python_saml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
writing manifest file 'src/python_saml.egg-info/SOURCES.txt'
running build_ext
Segmentation fault (core dumped)

Here is my GDB run:

root@gandalf # gdb python2.7
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /opt/rh/python27/root/usr/bin/python2.7...(no debugging symbols found)...done.
Missing separate debuginfos, use: debuginfo-install python27-python-2.7.5-10.el6.x86_64
(gdb) run
Starting program: /opt/rh/python27/root/usr/bin/python2.7 
[Thread debugging using libthread_db enabled]
Python 2.7.5 (default, Dec  3 2013, 08:35:16) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dm.xmlsec.binding

Program received signal SIGSEGV, Segmentation fault.
__pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>) at src/lxml/lxml.etree.c:26833
26833   src/lxml/lxml.etree.c: No such file or directory.
    in src/lxml/lxml.etree.c

Any thoughts?

Thanks for any help.

[cjsteinke]

For shiz and giggles I tried on Fedora Core 20 and received the same results.

[root@sauron python-saml]#  coverage run --source=src/onelogin/saml2 --rcfile=tests/coverage.rc setup.py test
running test
running egg_info
writing requirements to src/python_saml.egg-info/requires.txt
writing src/python_saml.egg-info/PKG-INFO
writing top-level names to src/python_saml.egg-info/top_level.txt
writing dependency_links to src/python_saml.egg-info/dependency_links.txt
reading manifest file 'src/python_saml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
writing manifest file 'src/python_saml.egg-info/SOURCES.txt'
running build_ext
Segmentation fault

[pitbulk]

Hi @cjsteinke and @navneil , Sorry to see that you experiencing problems installing python-saml

@cjsteinke , I install M2Crypto using pypi: https://pypi.python.org/pypi/M2Crypto (openssl and swig are dependences from this library).
You can read in the documentation of python-saml, at https://github.com/onelogin/python-saml#dependences you can see libxmlsec1-dev, and libxmlsec1-openssl is a dependece of this package.

I googled the error and it seems a bug with the lxml library (lxml is a dependece of dm.xmlsec.binding): https://bugs.launchpad.net/lxml/+bug/928333

[pitbulk]

Hi @navneil and @cjsteinke , sorry to see that python-saml is not working on Fedora.

In python-saml I install M2Crypto using pypi (https://pypi.python.org/pypi/M2Crypto). This library requires openssl and swig.

I googled the error and it seems a bug on the lxml library (a dependence of dm.xmlsec.binding)

[pitbulk]

I asked a friend to try get it working on Fedora.

@cjsteinke can you try to use the lxml library? Maybe an issue that this library was not compiled correctly. Did you install the python-dev, libxml2-dev and libxslt-dev libraries?

In Ubuntu, when building 'dm.xmlsec.binding' I see:

    building 'dm.xmlsec.binding._xmlsec' extension
    gcc -pthread -fno-strict-aliasing -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_SIZE_T=1 -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_CRYPTO=openssl -I/home/travis/virtualenv/python2.7.8/build/dm.xmlsec.binding/lxml-3.4.0-py2.7-linux-x86_64.egg/lxml -I/home/travis/virtualenv/python2.7.8/build/dm.xmlsec.binding/lxml-3.4.0-py2.7-linux-x86_64.egg/lxml/includes -Isrc -I/usr/include/libxml2 -I/usr/include/xmlsec1 -I/opt/python/2.7.8/include/python2.7 -c src/_xmlsec.c -o build/temp.linux-x86_64-2.7/src/_xmlsec.o
    gcc -pthread -shared -L/opt/python/2.7.8/lib -Wl,-rpath=/opt/python/2.7.8/lib build/temp.linux-x86_64-2.7/src/_xmlsec.o -L/usr/lib -L/usr/lib/x86_64-linux-gnu -L/opt/python/2.7.8/lib -lxml2 -lxmlsec1-openssl -lxmlsec1 -lxslt -lssl -lcrypto -lpython2.7 -o build/lib.linux-x86_64-2.7/dm/xmlsec/binding/_xmlsec.so

[pitbulk]

Can you try to execute this in order to know the versión installed:

import sys
from lxml import etree

print("%-20s: %s" % ('Python', sys.version_info))
print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION))
print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION))
print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION))
print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION))
print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION))

Also in the dm.xmlsec.binding documentation appears:

This package interfaces with lxml via its Cython interface (described in etreepublic.pxd).
Some operating system installations for lxml lack the respective files. In those cases,
you may need to download an lxml source distribution and let the environment variable
LXML_HOME point to its root.

don't know if maybe is missing this LXML_HOME path.

Have you tried to install lxml using pypi? https://pypi.python.org/pypi/lxml/3.4.0

[navneil]

I got the normal source but I still get dm.xmlsec.binding to crash:

Python 2.7.5 (default, Dec 3 2013, 08:35:16)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import dm.xmlsec.binding

Program received signal SIGSEGV, Segmentation fault.
__pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>)
at src/lxml/lxml.etree.c:26833
26833 if (!__pyx_t_1) break;

This is what I am using:

Python : sys.version_info(major=2, minor=7, micro=5, releaselevel='final', serial=0)
lxml.etree : (3, 4, 0, 0)
libxml used : (2, 7, 6)
libxml compiled : (2, 7, 6)
libxslt used : (1, 1, 26)
libxslt compiled : (1, 1, 26)

On Wed, Oct 29, 2014 at 1:55 AM, Sixto Martin notifications@github.com
wrote:

In /src/lxml there are only .pyx files, no .c

So you need to get "normal release sources" to have the C files
pre-compiled, e.g. from http://pypi.python.org/pypi/lxml. If you check
out from git, you need Cython.

http://stackoverflow.com/questions/14588404/build-lxml-pre-compiled-c-cython-files-missing-solved

—
Reply to this email directly or view it on GitHub
#30 (comment).

[pitbulk]

I sent a mail to the author of dm.xmlsec.binding (Dieter).
He was very kind helping me with some doubts with the library in the past. I hope he has a solution for this problem.

Sorry for the inconveniences.

[cjsteinke]

Output:

Python 2.7.5 (default, Dec  3 2013, 08:35:16) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from lxml import etree
>>> print("%-20s: %s" % ('Python', sys.version_info))
Python              : sys.version_info(major=2, minor=7, micro=5, releaselevel='final', serial=0)
>>> print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION))
lxml.etree          : (3, 4, 0, 0)
>>> print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION))
libxml used         : (2, 7, 6)
>>> print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION))
libxml compiled     : (2, 7, 6)
>>> print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION))
libxslt used        : (1, 1, 26)
>>> print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION))
libxslt compiled    : (1, 1, 26)

I was able to upgrade lxml, I had 2.2.3 installed and it upgraded it to 3.4.0.

I also downloaded lxml and pointed LXML_HOME to it's root and no luck.

Thanks for your help.

[pitbulk]

I found this resource:
http://nz-realme.readthedocs.org/en/latest/installation.html#libxml2-and-xmldsig
Can you try install that libxml2 as suggested?