LDAP geek

Lock table is out of available object entries

François — Wed, 31 Jan 2007 14:04:00 +0100

Today my Fedora Directory Server indices could not be updated anymore. Every entry creation requiring an index modification (like adding an inetOrgPerson entry, requiring modifying the index of uid attribute) would fail with "operation error". In the ns-slapd logs there was this message:

[31/Jan/2007:12:01:03 +0100] - libdb: Lock table is out of available object entries
[31/Jan/2007:12:01:03 +0100] - libdb: Lock table is out of available object entries
[31/Jan/2007:12:01:03 +0100] - idl_new.c BAD 50, err=12 Cannot allocate memory
[31/Jan/2007:12:01:03 +0100] - database index operation failed BAD 1130, err=12 Cannot allocate memory
[31/Jan/2007:12:01:03 +0100] - database index operation failed BAD 1140, err=12 Cannot allocate memory
[31/Jan/2007:12:01:03 +0100] - database index operation failed BAD 1230, err=12 Cannot allocate memory
[31/Jan/2007:12:01:03 +0100] - database index operation failed BAD 1030, err=12 Cannot allocate memory
[31/Jan/2007:12:01:03 +0100] - add: attempt to index 12 failed

I tried to modify the DB_CONFIG files since I was thinking of a Berkeley DB configuration issue. In fact the problem seemed to be linked with the presence of temporary data files in the db/ directory.

[root@fedoracore6 slapd-fedoracore6]# ls db/
__db.001        __db.004        DBVERSION       NetscapeRoot/
__db.002        __db.005        guardian        userRoot/
__db.003        DB_CONFIG       log.0000000001
[root@fedoracore6 slapd-fedoracore6]# rm db/log.0000000001
[root@fedoracore6 slapd-fedoracore6]# rm db/__db.00*

After a restart-slapd the problem was solved.

Enabling TLS on a SUN One Directory Server Using OpenSSL

François — Mon, 22 Jan 2007 17:37:00 +0100

I use OpenSSL here for the certificate authority. SUN DS generate a server certificate request, that is to be signed with OpenSSL (in my case).

In SUN One DS administration console, open your LDAP server console, then choose "Manage Certificates". Choose a server if not already done, then generate a server certificate request, using certificate information compatible with your C.A. information (locality, country, etc.).

Save it into a file and copy it in your OpenSSL demoCA directory under the name newreq.pem. Edit the file to remove unexpected lines that the SUN One console often add in the file.

Then, launch:

# /usr/lib/ssl/misc/CA.sh -sign

This creates a file named newcert.pem. Import it in the SUN One console by choosing "Install..." in the "Manage Certificates" tool.

TLS Certificate Authority on Ubuntu Edgy (for OpenLDAP)

François — Mon, 22 Jan 2007 09:30:00 +0100

Here are the steps to build a very simple CA with OpenSSL, to be used with OpenLDAP:

Disable private keys ciphering

OpenLDAP (AFAIK) needs private keys to be in cleartext. Copy the OpenSSL CA.sh file in your working directory:

# cp /usr/lib/ssl/misc/CA.sh CA-nodes.sh

then modify it to add -nodes in the -newcert) and -newreq) sections:

-newcert)
   # create a certificate
   $REQ -new -nodes -x509 -keyout newreq.pem -out newreq.pem $DAYS
   RET=$?
   echo "Certificate (and private key) is in newreq.pem"
   ;;
-newreq)
   # create a certificate request
   $REQ -new -nodes -keyout newreq.pem -out newreq.pem $DAYS
   RET=$?
   echo "Request (and private key) is in newreq.pem"
   ;;

Create the CA

# /usr/lib/ssl/misc/CA.sh -newca

Generate a server certificate request

# ./CA-nodes.sh -newreq

Be sure to choose the FQDN of your server as the common name of the certificate, the one that will be used from client applications.

Sign the certificate request

# ./CA-nodes.sh -sign

Use the newly created certificate files

The certificate authority file is ./demoCA/cacert.pem. The server certificate file is newcert.pem. The server certificate key file is newkey.pem.

# mkdir /etc/ldap/tls
# cp newcert.pem /etc/ldap/tls/slapd-cert.pem
# cp newkey.pem /etc/ldap/tls/slapd-key.pem
# cp ./demoCA/cacert.pem /etc/ldap/tls/

Modify OpenLDAP' slapd.conf file to have the following lines:

TLSCACertificateFile    /etc/ldap/tls/cacert.pem
TLSCertificateFile      /etc/ldap/tls/slapd-cert.pem
TLSCertificateKeyFile   /etc/ldap/tls/slapd-key.pem

You can then test your installation with:

ldapsearch -x -ZZ -h 'your_server_FQDN' -b '' -s base

LDAP XPCOM for linux

François — Wed, 06 Dec 2006 09:44:00 +0100

I compiled LDAP XPCOM for Linux (see Using LDAP XPCom with standard XULRunner).

mozldap.xpt and libmozldap_stub.so must be copied in the components directory of your xul app. libmozldap.so, libldap50.so and libprldap50.so must be copied in the libraries directory of your xul app.

Using LDAP XPCom with standard XULRunner

François — Tue, 05 Dec 2006 01:03:00 +0100

Now than we have built LDAP components, the aim is to use it in a xulapp with the standard XULRunner, which has not LDAP support enabled.

The first idea is to copy mozldap.dll and mozldap.xpt (which can be found in the components directory of your LDAP-enabled-xulrunner package) into the components directory of your xulapp. The native LDAP SDK dlls must also be copied. They can be found in the xulrunner root directory: nsldap32v50.dll and nsldappr32v50.dll.

The tests I have made show that the interfaces published by the component are read by xulrunner (example: nsILDAPConnection). You can see it by executing this js code:

if (Components.interfaces.nsILDAPConnection)
{
  jsdump("nsILDAPConnection available");
}

(I am using a simple jsdump function to log stuff in to the js console, as explained here: Debugging a XULRunner Application)

But the LDAP components can't be loaded. You can see it by executing this js code:

if (Components.classes["@mozilla.org/network/ldap-connection;1"])
{
  jsdump("ldap-connection available");
}

This means that the LDAP dlls can't be loaded. Actually, this is perfectly normal, since they are neither in the xulrunner.exe directory, nor in any directory of the system library path.

The solution is described here: Using Dependent Libraries In Extension Components.

As I did not want to have a stub dll named bsmedberg_stub :-) I changed the names. Here are the steps I followed:

Create a subdirectory extensions/ldapstub in the mozilla source code directory, containing two files, makefile.in and LDAPStubLoader.cpp. Those files are attached to this post.

Then change your .mozconfig to add this line:

ac_add_options --enable-extensions=ldapstub

Then, build xulrunner by running:

make -f client.mk build

at the root of your mozilla directory.

Repackage XULRunner by running:

make -C xpinstall/packager

and extract it somewhere.

Now, create a subdirectory named libraries in your xulapp directory.

in the components directory, copy the mozldap.xpt and ldapstub.dll, both found in the components directory of XULRunner.
in the libraries directory, copy the mozldap.dll, nsldap32v50.dll, and nsldappr32v50.dll files (mozldap.dll can be found in the components directory of XULRunner, while nsldap32v50.dll and nsldappr32v50.dll can be found directly in the XULRunner directory).

Delete your xulapp directory in your user profile, and restart your xulapp with xulrunner. The LDAP components should be available.

XULRunner with LDAP support for Linux

François — Mon, 04 Dec 2006 14:28:00 +0100

XULRunner for Linux, compiled with Gtk2 and XFT (and LDAP), on an Ubuntu Dapper Drake system (don't know if it will work on another system).

Compiling XulRunner with LDAP support on Windows

François — Sun, 03 Dec 2006 15:54:00 +0100

Here are the steps I followed to compile XULRunner on MS Windows (using MSVC++7.1) with LDAP support enabled.

Prerequisites

Cygwin

You need the Cygwin environment (a collection of GNU tools used by the build process). Download the Cygwin installer at ftp://ftp.osuosl.org/pub/cygwin/set.... During the installation, choose the "DOS" style line ending. Install the following packages:

ash -- UNIX-like command line interpreter shell (Base category)
coreutils -- GNU core utilities (includes fileutils, sh-utils, and textutils) (Base category)
diffutils -- file comparison utility (Base category)
findutils (Base category)
sed -- a search and replace language (Base category)
grep -- text search tool (Base category)
gawk -- pattern matching language (Base and Interpretors categories)
perl -- a scripting language used to control parts of the build (Interpreters category)
patchutils -- a small collection of programs that operate on patch files (Devel category)
libiconv -- GNU character set conversion library and utilities (Devel category)
unzip -- zip file extraction (Archive category)
zip -- zip file creation (Archive category)

XULRunner 1.8.0.4 needs make 3.80, which is not available in the latest Cygwin installer. You need to get it here: http://cygwin.paracoda.com/release/..., or from my local copy if the link is dead. To extract the archive, put it in your root cygwin directory, then launch a Cygwin window (from the Cygwin icon on your desktop or in the start menu). Then enter:

cd /
tar xjvf make-3.80-1.tar.bz2

This will install make 3.80 on your cygwin tree. You can check if everything is ok by running make -v:

$ make -v
GNU Make 3.80 
Copyright (C) 2002  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

There is a conflict with the Cygwin linker and the MSVC linker, so you have to rename the Cygwin linker, which can be found in %CYGWINBASE%\bin\link.exe. You can run in Cygwin:

mv /bin/link.exe /bin/link-bak.exe

GLib and libIDL for VC7.1

You need GLib and libIDL compiled with your compiler. I found them here: GLib for VC7.1 and libIDL for VC7.1. Extract them in a directory not included in cygwin. For example:

C:\moztools-vc71bin\

Make the DLLs executable, by running in Cygwin:

chmod 755 /cygdrive/c/moztools-vc71bin/bin/*

Netscape Wintools

Netscape's Wintools is a collection of tools needed to build Mozilla software on Windows. You can get it here: http://ftp.mozilla.org/pub/mozilla..... Extract the archive into a temporary directory, then install it using a Microsoft command-line window (run "cmd" in the run dialog of the start menu):

set MOZ_TOOLS=C:\moztools
cd YOURTEMPDIR\buildtools\windows
install.bat

Of course C:\moztools is my case, you can change it (it is created by the install.bat program if needed).

Get the Mozilla source

Get the xulrunner 1.8.0.4 source code here: ftp://ftp.mozilla.org/pub/mozilla.o.... Take care to extract it into a directory not containing spaces in its path.

Configure the build process

Once you have extracted the source tarball, you have to configure the build options. Create a file named .mozconfig at the root of the "mozilla" source directory, and containing:

. $topsrcdir/xulrunner/config/mozconfig
ac_add_options --enable-ldap
ac_add_options --disable-javaxpcom

--enable-ldap enables LDAP support, and --disable-javaxpcom disables Java XPCom bridge, since it needs a Java development environment, which I have not.

Then, create a file initializing your build environment, named for example env.cmd, and containing:

rem --- Set HOME so that cvs and ssh work correctly
rem --- cvs uses HOME to locate your .cvspass file, and ssh to locate your .ssh file
rem --- if you are using ssh, your HOME should match the home directory specified in /etc/passwd. See http://www.cygwin.com/faq/faq0.html.
set HOME=C:\home

rem --- Set VCVARS to wherever the MSVC vcvars.bat file is found
set VCVARS=C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\VCVARS32.BAT

rem --- Set MSSDK to wherever the MS SDK is installed
rem --- The separate SDK is only required for MSVC6
set MSSDK=C:\Program Files\Microsoft Platform SDK

rem --- Set MOZ_TOOLS to wherever you have the moztools packaged installed
set MOZ_TOOLS=C:\moztools

rem --- Set CYGWINBASE to wherever cygwin is installed
rem --- Do not use CYGWIN or else cygserver, cygrunsrv, and other cygwin services will not function properly.
rem --- The CYGWIN variable is also used to sort of change the behaviour of Cygwin.
set CYGWINBASE=C:\cygwin
rem --- Make sure Cygwin does not print out a DOS style path warning
set CYGWIN=nodosfilewarning

rem --- If you are using VC7 or VC7.1, you must set GLIB_PREFIX and LIBIDL_PREFIX
set GLIB_PREFIX=C:\moztools-vc71bin
set LIBIDL_PREFIX=C:\moztools-vc71bin

rem --- Prepend Cygwin path
rem --- This is necessary so that cygwin find is ahead of windows find.exe in the PATH, but cygwin link is  after MSVC link.exe.
rem PATH=%CYGWINBASE%\bin;%PATH%

rem --- Set MSVC environment vars
call "%VCVARS%"

rem --- Prepend SDK paths
rem --- Only required for MSVC6
set PATH=%MSSDK%\bin;%PATH%
set INCLUDE=%MSSDK%\include;%INCLUDE%
set LIB=%MSDDK%\lib;%LIB%

rem --- Add glib/libidl to build environment
rem --- Only needed for VC7 or 7.1
set PATH=%PATH%;%GLIB_PREFIX%;%GLIB_PREFIX%\bin
set INCLUDE=%GLIB_PREFIX%\include;%INCLUDE%
set LIB=%GLIB_PREFIX%\lib;%LIB%

rem -- moztools comes last after glib/libIDL
set PATH=%PATH%;%MOZ_TOOLS%\bin

rem --- Now the PATH variable contains:
rem MS-SDK; MSVC; Cygwin; Windows; glib/libIDL; Moztools

rem --- Typically the last thing the script does is launch a cygwin shell
rem --- WARNING: This may reset your carefully setup path! Check your /etc/profile and ~/.profile files.
bash --login -i

Launch a Cygwin window. In the Cygwin environment, your C hard disk can be found in the /cygdrive/c/ directory. Run your env.cmd script. Then move to your mozilla source directory. On my system it's:

cd /cygdrive/c/mozilla-src/mozilla/

Build

You can launch the build by running:

make -f client.mk build

XULRunner is now being built. You can experience xpidl.exe crashes. In my case, the reason was that I wasn't using the right Glib and libIDL DLL for my compiler. Also, don't forget to make your GLib and libIDL executable.

Package

Now that you have built XULRunner, you can package it by running in the mozilla directory:

make -C xpinstall/packager

This will create an archive of XULRunner in the dist/ directory. That you can install on another computer. Here is mine: http://techblog.beretti.org/public/....

Two strange errors using ADSI (solved, of course)

François — Tue, 24 Oct 2006 17:24:00 +0200

0x8000500C when getting an attribute value

=> delete %WINDOWS%\SchCache\*.sch file

server down when calling ExecuteSearch

=> invalid filter !

Visual Studio macro to build only the project containing the active file

François — Tue, 12 Sep 2006 11:46:00 +0200

Here is the macro VB code :

    Sub BuildCurrentProject()
        Dim MyProj As String
        Dim Solution As String
        Dim item As ProjectItem

        On Error Resume Next

        Solution = DTE.Solution.Properties.Item("Name").Value
        Projet = DTE.ActiveDocument.ProjectItem.ContainingProject.Name

        If Projet <> "" Then
            DTE.Windows.Item(DTE.ActiveDocument.Name).Activate()
            DTE.Windows.Item(Constants.vsWindowKindSolutionExplorer).Activate()
            DTE.ActiveWindow.Object.GetItem(Solution + "\" + Projet).Select(vsUISelectionType.vsUISelectionTypeSelect)
            DTE.ExecuteCommand("Build.BuildOnlyProject")
        End If

    End Sub

Be careful when using T2W or T2OLE

François — Tue, 05 Sep 2006 10:52:00 +0200

The problem

If you are using ADSI programming interface in your software in C++, you probably do things like :

USES_CONVERSION;
...
hr = pDSSearch->ExecuteSearch(...);
hr = pDSSearch->GetFirstRow(hSearch);
ADS_SEARCH_COLUMN scValue;
while (SUCCEEDED(hr) && hr != S_ADS_NOMORE_ROWS)
{
        pDSSearch->GetColumn(hSearch, T2W(_T("objectClass")), &scValue);
        ...
}

If you are searching under a node that has thousands of children, you will probably experience crashes, caused by stack overflow or memory corruption.

This is explained on a Code Guru article: ATL String: What's wrong with the USES_CONVERSION macros? How to avoid using them?

The problem is that USES_CONVERSION macros use _alloca, which allocate memory on the stack.

For example, on my system, this simple code crashes when m equals 44390:

#include "stdafx.h"

#include <atlbase.h>

int _tmain(int argc, _TCHAR* argv[])
{
        
        USES_CONVERSION;
        LPWSTR lpszTmp = NULL;
        for (int m=0;m<400000;m++)
        {
                printf("m: %d\n", m);
                lpszTmp = T2W(_T("objectClass"));
        }

        return 0;
}

So when you use these macros, check that they shall not be executed in a big loop.

The solution

If you want a safe way to convert strings from and to unicode using ATL, ATL 7.0 provides you with conversion classes, CT2W and CW2T.

CT2W pszAttribute(_T("objectClass"));
pDSSearch->GetColumn(hSearch, pszAttribute, &scValue);

This is a safe and simple way. Memory is not allocated on the stack anymore, the CT2W class allocates it for you and frees it when the instance is destroyed.

Using AjaxTerm on Ubuntu

François — Mon, 21 Aug 2006 10:58:00 +0200

AjaxTerm is a python web server offering a terminal emulator web application, which allows you to open a session on your server when the only way you can access it is by HTTP (for example, if a firewall prevents you from using SSH). Here is how to install it on Ubuntu.

Downloading and running AjaxTerm

Download the latest version on the AjaxTerm website. Then uncompress the archive and just run the ajaxterm.py executable file.

wget http://antony.lesuisse.org/qweb/files/Ajaxterm-0.9.tar.gz
tar zxvf Ajaxterm-0.9.tar.gz
cd Ajaxterm-0.9
./ajaxterm.py

AjaxTerm listen for new connection at http://localhost:8022/.

Enabling secure remote access

If you want to allow remote access, the most secure way is to use the HTTP proxy feature of Apache, and SSL.

Install Apache2 web server, then enable the proxy and ssl modules :

sudo cp /etc/apache2/mods-available/proxy.* /etc/apache2/mods-enabled/
sudo cp /etc/apache2/mods-available/ssl.* /etc/apache2/mods-enabled/

Initialize SSL on Apache :

sudo apache2-ssl-certificate

Then configure your new host :

sudo vi /etc/apache2/sites-availables/002-AjaxTerm

In this file, write :

Listen 443
NameVirtualHost *:443

<VirtualHost *:443>
   ServerName localhost
   SSLEngine On
   SSLCertificateKeyFile ssl/apache.pem
   SSLCertificateFile ssl/apache.pem

   ProxyRequests Off
   <Proxy *>
      Order deny,allow
      Allow from all
   </Proxy>
   ProxyPass /ajaxterm/ http://localhost:8022/
   ProxyPassReverse /ajaxterm/ http://localhost:8022/
</VirtualHost>

Replace 'localhost' in 'ServerName localhost' with the hostname of your web server. Enable this host :

sudo cp /etc/apache2/sites-availables/002-AjaxTerm /etc/apache2/sites-enabled/

Then restart Apache 2 to check everything is ok. Youre web terminal emulator is now ready. You can access it on https://hostname/ajaxterm/.

Remember than ajaxterm.py must be running every time you want to use it.

Configuring TLS on IBM Tivoli Directory Server 5.2

François — Fri, 18 Aug 2006 01:37:00 +0200

I had diffculties to configure TLS on IBM Tivoli Directory Server, because of the certificate manager provided with GSKit, that did not support .kdb files used by ITDS. Here is some information on how to configure GSKit to handle .kdb files, then how to use your newly created certificate with ITDS.

Configure GSKit to support .kdb files

Check that GSKit 7a is installed
Install the IBM JDK (for example, using the Eclipse development package)
Set the JAVA_HOME environment variable to this setup
Delete the following files in JAVA_HOME/jre if they exist :
- lib/ext/ibmjsse.jar
- lib/ext/gskikm.jar
- lib/ext/ibmjcaprovider.jar (the ‘a’ is important)
Check that the following files exist in JAVA_HOME/jre :
- lib/ext/ibmjceprovider.jar
- lib/ext/ibmpkcs.jar (or ibmpkcs11.jar)
- lib/ibmjcefw.jar
- lib/ext/ibmjcefips.jar (optional - for FIPS support)
- lib/security/local_policy.jar
- lib/security/US_export_policy.jar
- lib/ibmpkcs11.jar (or ibmpkcs.jar)
Edit JAVA_HOME/jre/lib/security/java.security to add IBM CMS and IBM JCE security providers, right after the SUN security provider. Examples :

security.provider.1=sun.security.provider.Sun
security.provider.2=com.ibm.spi.IBMCMSProvider
security.provider.3=com.ibm.crypto.provider.IBMJCE

security.provider.1=com.ibm.jsse.IBMJSSEProvider
security.provider.2=com.ibm.spi.IBMCMSProvider
security.provider.3=com.ibm.crypto.provider.IBMJCE
security.provider.2=com.ibm.crypto.provider.IBMJCE
security.provider.4=com.ibm.security.jgss.IBMJGSSProvider
security.provider.5=com.ibm.security.cert.IBMCertPath