using perl dynamic loading in scratchbox ======================================== Some perl modules are not written completely in perl, instead they use native libraries for some tasks. One such module is notorious XML::Parser::Expat, which is basically perl wrapper for libexpat library. Because XML::Parser::Expat has caused so much grief lately, it will be used as an example throughout this document. however, these instructions should work for any perl module. Scratchbox environment is an exotic mix of host and target binaries. Usually host and target architectures differ (thats the whole idea of scratchbox, ain't it?), and that causes problems with dynamic loading of perl modules. when one runs simple perl command like: perl -e 'use XML::Parser::Expat' following chain of events will happen: - perl is searched and will probably be found at "/scratchbox/tools/bin/perl", due to scratchbox's binary redirection mechanism. - (x86)perl gets loaded and it starts working. first thing it does is to load all the needed modules (XML::Parser::Expat in this case) - Next thing that gets loaded is Expat.so, which is native library containing c-bindings for perl to be able to use libexpat.so. - libexpat.so is loaded. +----------+ +----------+ +-----------+ | | | | | | | perl +----->| Expat.so +----->|libexpat.so| | | | | | | +----------+ +----------+ +-----------+ if all these three binaries are for the very same processor architecture everything should go smoothly and one-liner will return with no further output. Is this isn't the case, the shit will hit fan and one should see error message something like this: ------------------------- [sbox-ARM: /host_usr/perl] > perl -e 'use XML::Parser::Expat' Can't load '/host_usr/perl/auto/XML/Parser/Expat/Expat.so' for module XML::Parser::Expat: /usr/lib/libexpat.so.1: ELF file OS ABI invalid at /scratchbox/tools/lib/perl5/5.8.3/i686-linux-thread-multi/DynaLoader.pm line 229. at -e line 1 Compilation failed in require at -e line 1. BEGIN failed--compilation aborted at -e line 1. -------------------------- the crucial line here is to notice following statement: "ELF file OS ABI invalid", which means that Application Binary Interface is non compatible between binaries used. ie. one is trying to mix host and target binaries, which of course will not work as expected. answer to this problem is quite simple. one just have to ensure that all binaries are compiled for the HOST system. perl shipped with scratchbox is configured to search it's perl modules first from the "/host_usr/perl" directory. just place all the stuff there and things should go through smoothly. it is crucial that the directory structure under /host_usr/perl is correct, so one should use one of the perl-module packages provided or compile perl modules to right place using CPAN perl module. both methods will be explained in the rest of this document starting with using pre made package, because that is way simpler than using CPAN module. Using pre made package ---------------------- issue following commands INSIDE scratchbox. cd /host_usr tar -xzvf pre-made-perl-module-package.tar.gz DONE! that wasn't so hard. next you probably want to test if the setup works correctly. simply execute: perl -e 'use XML::Parser::Expat' Using CPAN module ----------------- CPAN is provided with perl by default. one can get to the interactive command prompt by issuing command: perl -MCPAN -e shell when one uses CPAN module for the first time, some questions about configuration will be made. the defaults should be fine to get started. at this prompt one can install modules by saying: install Module::Name help can be obtained by saying: ? modules composed purely from perl should install themselves with no problems at all but modules with c-bindings will require headers and libraries of the package they depend on. scratchbox's perl is configured so that the headers must be placed to /host_usr/perl/include and libraries to /host_user/perl/lib. there is two methods how one can obtain headers and libraries to the /host_usr/perl. simpler (may not work, though. depends on your HOST) is to just copy files from your HOST system to the scrathcbox. issue following commands OUTSIDE scratchbox: cp /usr/include/expat.h /scratchbox/users/$USER/host_usr/perl/include cp -a /usr/lib/libexpat.so* /scratchbox/users/$USER/host_usr/perl/lib start scratchbox session and run CPAN: cd /scratchbox/users/$USER ./run.sh perl -MCPAN -e shell install XML::Parser::Expat q perl -e 'use XML::Parser::Expat' More complex and complete way of getting needed files into place is to compile them with host-gcc. tip of the day -------------- one can view all installed perl modules by issuing following command OUTSIDE scratchbox: perldoc /scratchbox/users/$USER/host_usr/perl/perllocal for some reason perldoc wont work correctly inside scratchbox; the screen will get scrambled. this might get fixed in the future. tip for the night ----------------- use file command to view which architecture binaries are for. man file. feedback -------- please, help us to make this document more complete. all feedback is very welcome. one can mail corrections && suggestions to lauri.arimo@movial.fi help is also needed to make pre-made perl dev package as complete as possible. if you think that some module is missing from there, please, tell us about it.