Keystone Administration</TITDone. ETA NAME="GENERATOR" CONTENT="Modular DocBook HTML Stylesheet Version 1.79"></HEAD ><BODY CLASS="book" BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#0000FF" VLINK="#840084" ALINK="#0000FF" ><DIV CLASS="BOOK" ><A NAME="keystone" ></A ><DIV CLASS="TITLEPAGE" ><H1 CLASS="title" ><A NAME="AEN2" >Keystone Administration</A ></H1 ><H3 CLASS="author" ><A NAME="AEN4" ></A >Adam Dickmeiss</H3 ><H3 CLASS="author" ><A NAME="AEN7" ></A >Sebastian Hammer</H3 ><H3 CLASS="author" ><A NAME="AEN10" ></A >Mike Taylor</H3 ><H3 CLASS="author" ><A NAME="AEN13" ></A >Marc Cromme</H3 ><H3 CLASS="author" ><A NAME="AEN16" ></A >Anders Sønderberg Mortensen</H3 ><P CLASS="copyright" >Copyright © 2002, 2003, 2004, 2005 Index Data ApS</P ><DIV ><DIV CLASS="abstract" ><P ></P ><A NAME="AEN25" ></A ><P > <A HREF="http://www.indexdata.dk/keystone/" TARGET="_top" > The Keystone Library (TKL)</A > is a web application framework that eases the construction of library web portals and other web sites with semi-structured content. It is used to build literature web portals which are highly user customizable, and literature search engines based on the Z39.50 or SRW protocol. </P ><P > TKL consists of many modules and pluggable handlers used to add specialized functionality. Among the available modules are OAI harvesters and web harvesters. The existing handlers are specialized to integrate Z39.50 queries, SOAP queries, SQL queries and much more into the TKL framework. Custom designed pluggable components can be freely added. </P ><P > The Keystone Library is based on XML and XSLT technologies. It currently uses Sablotron as processing engine, and PHP as application glue language. Helper utilities are written in Perl and Tcl. </P ><P > This documentation describes the installation and programming interfaces of <A HREF="http://www.indexdata.dk/keystone/" TARGET="_top" > The Keystone Library (TKL)</A >. </P ><P > CVS id: $Id: tkl.xml,v 1.7 2005/06/23 10:03:58 adam Exp $ </P ><P ></P ></DIV ></DIV ><HR></DIV ><DIV CLASS="TOC" ><DL ><DT ><B >Table of Contents</B ></DT ><DT >1. <A HREF="#keystone-introduction" >Introduction</A ></DT ><DD ><DL ><DT >1.1. <A HREF="#intro-basics" >Basic principles</A ></DT ><DT >1.2. <A HREF="#intro-base" >The base portal Bibliotheca</A ></DT ></DL ></DD ><DT >2. <A HREF="#keystone-installation" >Installation</A ></DT ><DD ><DL ><DT >2.1. <A HREF="#debian" >Debian</A ></DT ><DT >2.2. <A HREF="#redhat" >Red Hat</A ></DT ><DT >2.3. <A HREF="#web-server" >Web Server</A ></DT ><DT >2.4. <A HREF="#expat" >XML parser</A ></DT ><DT >2.5. <A HREF="#dom" >DOM XML</A ></DT ><DT >2.6. <A HREF="#xslt" >Sablotron XSLT</A ></DT ><DT >2.7. <A HREF="#yaz" >Z39.50 Support</A ></DT ><DT >2.8. <A HREF="#php" >PHP</A ></DT ><DT >2.9. <A HREF="#install-php-yaz" >PHP/YAZ</A ></DT ><DT >2.10. <A HREF="#install-keystone-files" >Installing Keystone files</A ></DT ><DT >2.11. <A HREF="#enable-tkl1" >Enabling Keystone in Apache 1.3.X</A ></DT ><DT >2.12. <A HREF="#enable-tkl2" >Enabling Keystone in Apache 2</A ></DT ><DT >2.13. <A HREF="#installing-portal" >Installing the Sample Portal</A ></DT ><DT >2.14. <A HREF="#starting-zebra" >Starting the Zebra server</A ></DT ><DT >2.15. <A HREF="#testing-it" >Testing it</A ></DT ><DT >2.16. <A HREF="#virtual-hosting" >Virtual hosting</A ></DT ></DL ></DD ><DT >3. <A HREF="#keystone-content" >Content administration</A ></DT ><DD ><DL ><DT >3.1. <A HREF="#content-dir" >Directory structure</A ></DT ><DT >3.2. <A HREF="#content-docs" >Documents</A ></DT ><DT >3.3. <A HREF="#content-schemas" >Schemas</A ></DT ><DT >3.4. <A HREF="#content-users" >User Administration</A ></DT ><DT >3.5. <A HREF="#content-base" >The base portal Bibliotheca</A ></DT ><DT >3.6. <A HREF="#content-searching" >Searching</A ></DT ></DL ></DD ><DT >4. <A HREF="#interface" >Managing end-user interfaces</A ></DT ><DD ><DL ><DT >4.1. <A HREF="#interface-context" >Document context and the user shell</A ></DT ><DT >4.2. <A HREF="#interface-params" >Parameters</A ></DT ><DT >4.3. <A HREF="#interface-session" >Session parameters</A ></DT ><DT >4.4. <A HREF="#interface-functions" >Supporting functions</A ></DT ><DD ><DL ><DT >4.4.1. <A HREF="#tkl-admin-header-section" >tkl-admin-header</A ></DT ><DT >4.4.2. <A HREF="#AEN539" >tkl-args</A ></DT ><DT >4.4.3. <A HREF="#AEN550" >tkl-default</A ></DT ><DT >4.4.4. <A HREF="#AEN559" >tkl-file</A ></DT ><DT >4.4.5. <A HREF="#AEN576" >tkl-file-exists</A ></DT ><DT >4.4.6. <A HREF="#AEN585" >tkl-find</A ></DT ><DT >4.4.7. <A HREF="#AEN606" >tkl-grant</A ></DT ><DT >4.4.8. <A HREF="#tkl-header-section" >tkl-header</A ></DT ><DT >4.4.9. <A HREF="#AEN640" >tkl-help</A ></DT ><DT >4.4.10. <A HREF="#AEN652" >tkl-mail</A ></DT ><DT >4.4.11. <A HREF="#AEN663" >tkl-path</A ></DT ><DT >4.4.12. <A HREF="#AEN672" >tkl-search</A ></DT ><DT >4.4.13. <A HREF="#AEN683" >tkl-set-footer</A ></DT ><DT >4.4.14. <A HREF="#AEN689" >tkl-soap</A ></DT ><DT >4.4.15. <A HREF="#AEN713" >tkl-sql</A ></DT ><DT >4.4.16. <A HREF="#AEN746" >tkl-time</A ></DT ><DT >4.4.17. <A HREF="#AEN754" >tkl-unique</A ></DT ><DT >4.4.18. <A HREF="#AEN773" >tkl-user</A ></DT ></DL ></DD ><DT >4.5. <A HREF="#interface-config" >Portal configuration</A ></DT ><DT >4.6. <A HREF="#interface-debugging" >Debugging</A ></DT ><DT >4.7. <A HREF="#interface-base" >The base portal Bibliotheca</A ></DT ><DD ><DL ><DT >4.7.1. <A HREF="#AEN850" >User interface</A ></DT ><DT >4.7.2. <A HREF="#AEN853" >Overall page structure - interface.xsl</A ></DT ><DT >4.7.3. <A HREF="#AEN860" >portal.xsd/xsl</A ></DT ><DT >4.7.4. <A HREF="#AEN867" >subject.xsd/xsl</A ></DT ><DT >4.7.5. <A HREF="#AEN872" >link.xsd</A ></DT ><DT >4.7.6. <A HREF="#AEN875" >document.xsd/xsl</A ></DT ><DT >4.7.7. <A HREF="#AEN880" >search.xsl</A ></DT ><DT >4.7.8. <A HREF="#AEN887" >news.xsd - newspage.xsl</A ></DT ></DL ></DD ></DL ></DD ><DT >5. <A HREF="#oai" >Keystone and OAI Metadata Services</A ></DT ><DD ><DL ><DT >5.1. <A HREF="#oairep" >Keystone as an OAI repository</A ></DT ><DT >5.2. <A HREF="#oaibibliotheca" >Bibliotheca example OAI repository</A ></DT ><DD ><DL ><DT >5.2.1. <A HREF="#AEN940" >Identify</A ></DT ><DT >5.2.2. <A HREF="#AEN945" >ListMetdataFormats</A ></DT ><DT >5.2.3. <A HREF="#AEN952" >ListSets</A ></DT ><DT >5.2.4. <A HREF="#AEN959" >ListIdentifiers</A ></DT ><DT >5.2.5. <A HREF="#AEN966" >GetRecord</A ></DT ><DT >5.2.6. <A HREF="#AEN973" >ListRecords</A ></DT ></DL ></DD ><DT >5.3. <A HREF="#oaiharv" >OAI harvesting from within Keystone</A ></DT ></DL ></DD ></DL ></DIV ><DIV CLASS="chapter" ><HR><H1 ><A NAME="keystone-introduction" ></A >Chapter 1. Introduction</H1 ><P > <A HREF="http://www.indexdata.dk/keystone/" TARGET="_top" >The Keystone Library (TKL)</A > is a programming framework for fast deployment of information-driven web portals. </P ><P > By <I CLASS="firstterm" >portals</I >, we mean websites that make information resources available. These may be corporate or personal homepages, ``subject gateways'', or any number of other types of resources. Along with Keystone, we provide a ``base portal'' called Bibliotheca, which supports a hierarchically organised collection of metadata about external resources, but which is also capable or presenting its own content, such as articles, news entries, etc. By building on top of the base portal, or by constructing a completely new template, practically any type of information-based website can be constructed. </P ><P > The Keystone Library can be seen as an alternative to traditional, closed ``content management'' systems, or to custom-built, database-driven websites. Especially in the case of sites that are naturally organized as collections of documents of various types, Keystone can be a useful tool. The system is particularly well-suited to situations where the separation of content from presentation is important. </P ><P > The Keystone Library was developed, and has been extensively tested, using the Apache web server version 1.3. Other servers should work provided that they support PHP and have the necessary extra components. </P ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="intro-basics" >1.1. Basic principles</A ></H2 ><P > A Keystone portal is a collection of <I CLASS="firstterm" >documents</I > of different types. The documents are typically organized in a directory structure which reflects the logical structure of the portal. The documents are represented in standard XML. A portal generally is made up from documents of different types. Each type is represented by an XML <I CLASS="firstterm" >Schema</I > that describes which data elements appear in the documents, which elements are repeatable, which are mandatory, what their types are, etc. The XML schema also contains additional information specific to Keystone, for instance whether a given element is multi-lingual, corresponds to a restricted vocabulary, etc. </P ><P > The Keystone Library includes a content management system which allows editors to manage the file structure and its contents (and hence the structure and contents of the user-facing portal) via a web-based interface. Access control is supported and the system supports the delegation of responsibility for the maintenance of different parts of a portal. </P ><P > The documents are generally stored as ordinary XML text files on the server's filesystem. All Keystone documents have the filename suffix <TT CLASS="literal" >.tkl</TT >. The web server is configured so that when the end-user attempts to access a document with the <TT CLASS="literal" >.tkl</TT > suffix, The Keystone Library ``user shell'' is invoked. The user shell is responsible for displaying the document to the user in a suitable way. </P ><P > For the purpose of display, each document type is associated with a presentation format. In this way, a help file is displayed differently from a subject group or a metadata record, and so on. The presentation formats are realised as XSLT <I CLASS="firstterm" >Stylesheets</I > which are processed to display each type of document. So the task of the Keystone user shell is, for a given document, to determine which XSLT stylesheet should be used to display the document. Apart from the document itself, the user shell makes a range of information available to the stylesheet, such as where in the directory hierarchy the document is located, whether there are other documents nearby (eg. in subdirectories), etc. It also provides the stylesheet with functions for invoking external services such as information retrieval (searching), remote procedure calls and database access. </P ><P > In summary, a Keystone portal generally consists of three basic elements: </P ><P ></P ><UL ><LI ><P > Documents, represented in XML </P ></LI ><LI ><P > Document types, expressed as XML schemas </P ></LI ><LI ><P > Presentation formats, expressed as XSLT stylesheets </P ></LI ></UL ><P > The documents are the pivot of the system - they constitute the <SPAN CLASS="emphasis" ><I CLASS="emphasis" >content</I ></SPAN > of a portal. The document types are primarily used in portal maintainance. The presentation formats are used when a user views a document. </P ><P > The maintenance and user-facing sides of Keystone are fully independent. You can replace the content management interface with another system, or use <I CLASS="foreignphrase" >ad-hoc</I > scripts or batch jobs to maintain all or parts of the contents of a site. In the same way, you can use the Keystone management interface to provide remote administration for a group of files, and then use different techniques to actually present their content to end-users (assuming you even care about presenting the content to users). </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="intro-base" >1.2. The base portal Bibliotheca</A ></H2 ><P > Together with Keystone we provide an example of a fairly basic type of portal. It is included as an example of how Keystone may be used, and it may be freely used and expanded as a starting point for new portals. </P ><P > All of the pages of the portal are built around a shared structure, starting with a graphical bar at the top of the page to lend identity to the website. Underneath is a horizontal menu with certain fixed functions, including a search field. Below this, on the left side of the display, is the main menu, which also enables a language selection (if relevant). In the centre of the display is the content window, where the content of each page or document is presented. In the right-hand side is a column for news entries. </P ><P > Clicking on a news item, the ``show more news'' link (displayed when not all news entries fit in the left-hand column) or the ``News'' menu-item in the main menu brings the user to a news page where the full content of all articles in the news directory are displayed. </P ><P > Clicking on ``Help'' in the top menu brings the user to a system of help pages. </P ><P > The ``Articles'' item in the left-hand menu brings you to a collection of local articles. </P ><P > Typing in one or more keywords in the search field allows a free-text search through the portal. The administrator controls which directories should be searchable. </P ><P > Articles and help documents are both based on the same document type, described by the schema document.xsd. This is a general-purpose ``article'' type with the quality that it can display a ``mini-menu'' of any sub-documents. This makes it possible to use this simple document type to construct complex structures of documents (such as encyclopaedia articles with multiple chapters). </P ></DIV ></DIV ><DIV CLASS="chapter" ><HR><H1 ><A NAME="keystone-installation" ></A >Chapter 2. Installation</H1 ><P > The Keystone Library relies on a number of software components. </P ><P > If you are running Debian, you can use its package management facilities to handle most of the installation: web server, PHP, XML parser, XSLT, DOM support and Z39.50 support. See <A HREF="#debian" >Section 2.1</A >, then skip ahead to <A HREF="#keystone-content" >Chapter 3</A >. </P ><P > Similarly, on Red Hat boxes, you can use its RPM package management facilities for part of the work. Unfortunately, some our tools are not packaged as RPMs yet. See <A HREF="#redhat" >Section 2.2</A >, then skip ahead to <A HREF="#install-php-yaz" >Section 2.9</A >. </P ><P > If you are not running Debian or Red Hat, skip those section and read the instructions on how to install the components by hand, beginning with <A HREF="#web-server" >Section 2.3</A >. </P ><P > It is recommended to think twice before upgrading The Keystone Library <TT CLASS="filename" >TKL</TT > packages: Some functionality has changed and will not be compatible with behavior fond in the pre 1.5.2 releases. We do recommend testing an upgrade in a safe, non-production sandbox before applying to a production server. </P ><DIV CLASS="warning" ><P ></P ><TABLE CLASS="warning" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/warning.gif" HSPACE="5" ALT="Warning"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > IMPORTANT INSTALL NOTICE: </P ><P > This release has Sablotron dependencies changed to Index Data's own bug fixed versions 1:1.0-1 and larger, which are conflicting with the original Gingerall versions 1.0.1/1.0.2, and Debian version 1.0.2-2. A bug fixed Sablotron version is available from our web site as tarball and as Debian packages. </P ><P > The original 1.0.1 and 1.0.2 release has a bug which cripples XSLT handler arguments by applying unnecessary and unwanted mangeling of handler arguments, and another which segfaults Sablotron most unpleasently. </P ><P > The full description of the bug is found on Sablotrons maillist <A HREF="http://archive.gingerall.cz/archives/public/sablot2004/msg00000.html" TARGET="_top" >http://archive.gingerall.cz/archives/public/sablot2004/msg00000.html</A > and the Gingerall bug report with patch is found at <A HREF="http://bugzilla.gingerall.cz/show_bug.cgi?id=2811" TARGET="_top" >http://bugzilla.gingerall.cz/show_bug.cgi?id=2811</A > <A HREF="http://bugzilla.gingerall.cz/attachment.cgi?id=713&action=view" TARGET="_top" >http://bugzilla.gingerall.cz/attachment.cgi?id=713&action=view</A > See also the following debian bug reports: <A HREF="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=311609" TARGET="_top" >http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=311609</A > <A HREF="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=311611" TARGET="_top" >http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=311611</A > </P ><P > Unfortunately, the supplied patch which prevents unwanted mangeling of Args/URI for user-defined scheme XSLT handler calls has not yet been accepted upstream. This forces any Keystone installation to use the patched Sablotron version 1:1.0.1-2 found at our web site. <A HREF="ftp://ftp.indexdata.dk/pub/sablotron/sablotron-1.0-ID-patched-1.tar.gz" TARGET="_top" >ftp://ftp.indexdata.dk/pub/sablotron/sablotron-1.0-ID-patched-1.tar.gz</A > </P ><P > The pached version is available as Debian package from our Debian package site. </P ><P > Furthermore, this Keystone release 1.5.3 has changed the admin XSLT handler calling code to work with our Sablotron patch. All XSLT handler args are modified from <PRE CLASS="screen" > document('a-handler:xyz') </PRE > to <PRE CLASS="screen" > document('a-handler:/xyz') </PRE > (notice the slash) to avoid breaking by non-mangeling. </P ><P > Notice please that your own XSLT portal code needs to be changed the same way to be compliant with Keystone 1.5.3 and the patched Sablotron 1:1.0.1-2. </P ><P > Similar changes have to be applied to XSLT include and import statements. For example <PRE CLASS="screen" > <xsl:include href="tkl-file:some.xsl"/> </PRE > has to be changed to <PRE CLASS="screen" > <xsl:include href="tkl-file:/some.xsl"/> . </PRE > </P ><P > It is safe to apply these changes to portals running versions < 1.5.3 of Keystone, as the Sablotron packages version 0.90 and 0.95 do not touch the argument list of handlers in case there is one single or a double slash present. </P ></TD ></TR ></TABLE ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="debian" >2.1. Debian</A ></H2 ><P > Index Data's Debian packages must be in your Debian sources list (<TT CLASS="filename" >/etc/apt/sources.list</TT >). You must add the following two lines: <PRE CLASS="screen" > deb http://www.indexdata.dk/debian indexdata/sarge released deb-src http://www.indexdata.dk/debian indexdata/sarge released </PRE > Then install using the usual commands: <PRE CLASS="screen" > apt-get update apt-get install tkl </PRE > The following Debian packages will be installed when installing the Keystone binaries: apache apache-common idzebra libdigest-md5-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libmime-base64-perl libmm11 libnet-perl libsablot0 libtclrobot-tcl libtkl-perl liburi-perl libwww-perl libxml-libxml-perl libxml-libxslt-perl libxml-namespacesupport-perl libxml-parser-perl libxml-sax-perl libxslt1 libyaz php4 php4-domxml php4-xslt php4-yaz tkl tklite. </P ><DIV CLASS="warning" ><P ></P ><TABLE CLASS="warning" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/warning.gif" HSPACE="5" ALT="Warning"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > In case you have controlled the Sablotron versions in your <TT CLASS="filename" >/etc/apt/preferences</TT >, you need to erase those entries. The Index Data patched Sablotron packages have the epoch number 1, and will therefore be preferred over the original Debian Sablotron packages. </P ></TD ></TR ></TABLE ></DIV ><P > Please make sure that the following lines are found in the PHP4 config file <TT CLASS="filename" > /etc/php4/apache/php.ini</TT > <PRE CLASS="screen" > extension=domxml.so extension=yaz.so extension=xslt.so </PRE > and restart the Apache web server. </P ><P > The Keystone binaries are found in the packages <TT CLASS="filename" >tklite_1.4.7-1_i386.deb</TT > (core PHP and XSLT functionality, necessary package), <TT CLASS="filename" >libtkl-perl_1.4.7-1_i386.deb</TT > (Perl libraries, man pages, necessary package), <TT CLASS="filename" >tkl_1.4.7-1_i386.deb</TT > (init scripts, man pages, cron scripts, Perl binaries, necessary package), <TT CLASS="filename" >tkl-oai-harvester_1.4.7-1_i386.deb</TT > (complete OAI harvester including init scripts, man pages, cron scripts, Perl binaries, additional package), <TT CLASS="filename" >tkl-web-harvester_1.4.7-1_i386.deb</TT > (complete WEB harvester including init scripts, man pages, cron scripts, Tcl binaries, additional package), <TT CLASS="filename" >libtclrobot-tcl_0.2.0-1_i386.deb</TT > (Tcl extention needed by tkl-web-harvester_1.4.7-1_i386.deb, additional package), and <TT CLASS="filename" >tkl-doc_1.4.7-1_i386.deb</TT > (this documentation in HTML and PDF). </P ><P > The image uploading features are not part of the core Keystone packages but are contained in the example portal <TT CLASS="filename" >bibliotheca</TT >. They do not work properly unless you install the <TT CLASS="filename" >ImageMagick</TT > package as well. </P ><P > After installation of one or more portals in the Apache DocumentRoot directory <TT CLASS="filename" >/var/www</TT > these should be registered in either the system global Keystone configuration file <TT CLASS="filename" >/etc/tkl.conf</TT >, or in the main Apache configuration file <TT CLASS="filename" >/etc/apache/httpd.conf</TT >. The files must contain valid Apache Directory stanzas - or virtual host stanzas containing the "Directory" directive. See the commented examples in the configuration file <TT CLASS="filename" >/etc/tkl.conf</TT >. </P ><P > While all Apache stanzas can be omitted from <TT CLASS="filename" >/etc/tkl.conf</TT >, if just included from some other file, the "Directory" tags must be present and correctly pointing to Keystone portal root directories in order to start the indexing and harvesting servers. </P ><P > Now the installed portals are registered, and the idzebra search daemon of each portal can be indexed, started, and stopped by the root UID using the <TT CLASS="filename" >/etc/init.d/tkl</TT > script with one of the folowing options: <PRE CLASS="screen" > # /etc/init.d/tkl index [/path/to/portal [/sub/dir]] # /etc/init.d/tkl start [/path/to/portal] # /etc/init.d/tkl restart [/path/to/portal] # /etc/init.d/tkl stop [/path/to/portal] </PRE > </P ><P > The OAI harvesting and web harvesting daemons are started and stopped by the root UID using the <TT CLASS="filename" >/etc/init.d/tkl-oai-harvester</TT > and/or <TT CLASS="filename" >/etc/init.d/tkl-web-harvester</TT > script with one of the folowing options: <PRE CLASS="screen" > # /etc/init.d/tkl-oai-harvester start # /etc/init.d/tkl-oai-harvester restart # /etc/init.d/tkl-oai-harvester stop </PRE > Please consult the appropriate man pages for detailed information. <PRE CLASS="screen" > $ man 8 tkl $ man 5 tkl.conf $ man 5 tkl.config $ man 8 tkl-oai-harvester $ man 8 tkl-web-harvester </PRE > </P ><P > Furthermore, the daemons are started after each reboot, and the portals are indexed during each install of the tkl package. That is, installing the portals before installing tkl even removes the burden to remember the initial IDZebra indexing. </P ><P > Notice also that the default Apache configuration has changed. To enable Keysatone you need to uncomment these lines in the apache config files: <PRE CLASS="screen" > # LoadModule mime_module /usr/lib/apache/1.3/mod_mime.so # LoadModule dir_module /usr/lib/apache/1.3/mod_dir.so # LoadModule action_module /usr/lib/apache/1.3/mod_actions.so # LoadModule php4_module /usr/lib/apache/1.3/libphp4.so </PRE > </P ><P > In case you use the new configuartion layout of Apache, you might prefer to make the requested changes manually using the <TT CLASS="filename" >modules-config</TT > utility: </P ><P > <PRE CLASS="screen" > /usr/sbin/modules-config apache enable mod_mime /usr/sbin/modules-config apache enable mod_dir /usr/sbin/modules-config apache enable mod_actions /usr/sbin/modules-config apache enable mod_php4 </PRE > </P ><P > Be careful not to misspell any of the module names when invoking <TT CLASS="literal" >modules-config</TT >. It will not report any such errors, but quietly do nothing, leaving you with a hard-to-debug problem down the line. </P ><P > Finally, manually add the <PRE CLASS="screen" > Include /etc/tkl.conf </PRE > line to <TT CLASS="filename" >/etc/apache/httpd.conf</TT > </P ><P > It is possible that the installation of the Sablotron PHP extension will not configure PHP to recognise that extension. If not, you will see messages like <TT CLASS="literal" >Call to undefined function: xslt_create()</TT >. In this case, manually add the line <PRE CLASS="screen" > extension=xslt.so </PRE > to the end of <TT CLASS="filename" >/etc/php4/apache/php.ini</TT > </P ><P > Debian installations are nice and smooth, so skip ahead to <A HREF="#keystone-content" >Chapter 3</A > for the fun part of Keystone. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="redhat" >2.2. Red Hat</A ></H2 ><P > Index Data provides some Red Hat 9.0 packages, and most of external packages, which our software depends on, can conveniently be installed by the RPM system. This approach is only tested on Red Hat 9.0 systems, although it should work on running Red Hat 8, or even 7. </P ><P > You need to install the following official RPM packages: <TT CLASS="filename" >httpd</TT >, <TT CLASS="filename" >php</TT >, <TT CLASS="filename" >php-devel</TT >, <TT CLASS="filename" >tcl</TT >, <TT CLASS="filename" >tcp_wrappers</TT >, as well as the following packages from Index Data: <TT CLASS="filename" >libyaz</TT >, <TT CLASS="filename" >libyaz-devel</TT >, <TT CLASS="filename" >idzebra</TT >. </P ><P > For the remaining components there are no official packages. PHP/YAZ as well as Sablotron and PHP/XSLT must be compiled and installed separately. </P ><P > Install Sablotron first. It exists as a RPM package, but this one will not work with Keystone due to misconfiguring of the bundled Java interface. Therefore, get the tarball from <A HREF="http://ftp.indexdata.dk/pub/tkl/support" TARGET="_top" > Keystone support </A > or from the official <A HREF="http://www.gingerall.com" TARGET="_top" >Sablotron</A > site. Configure, make and make install as usual. </P ><DIV CLASS="warning" ><P ></P ><TABLE CLASS="warning" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/warning.gif" HSPACE="5" ALT="Warning"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Unfortunately, the Sablotron packages version 1.0 have a nasty bug which segfaults the Keystone applications using the tkl-file xslt macro. This forces any Keystone installation to use the patched sablotron version 1:1.0.1-2 found at our web site. <TT CLASS="filename" >http://ftp.indexdata.dk/pub/sablotron/sablotron-1.0-ID-patched-1.tar.gz</TT > </P ></TD ></TR ></TABLE ></DIV ><P > The rest of this guide assumes installation of Sablotron in <TT CLASS="filename" >/usr/local</TT >. </P ><P > For PHP/XSLT we need to make a PHP dynamic shared object (DSO) that links with the official RPM packages for PHP. Check the version number of PHP for RedHat (in Red Hat 9 the version is 4.2.2), and fetch a PHP source tarball of the same version number. You may download the 4.2.2 tarball from <A HREF="http://ftp.indexdata.dk/pub/tkl/support" TARGET="_top" > Keystone support</A >, or from the <A HREF="http://www.php.net/downloads.php" TARGET="_top" > PHP download area.</A > </P ><P > Then unpack the PHP source and do the following: <PRE CLASS="screen" > cd ext/xslt vi config.m4 </PRE > Remove the check for iconv in config.m4. iconv is installed already and is part of the running PHP! Remove or comment out the three following lines, starting at PHP_SETUP_ICONV. <PRE CLASS="screen" > PHP_SETUP_ICONV(XSLT_SHARED_LIBADD, [], [ AC_MSG_ERROR([iconv not found, in order to build sablotron you need the iconv library]) ]) </PRE > </P ><P > Now we have a proper <TT CLASS="filename" >config.m4</TT >, and can generate the <TT CLASS="filename" >configure</TT > script, and run it. <PRE CLASS="screen" > phpize ./configure --enable-xslt --with-xslt-sablot=/usr/local make # su # make install </PRE > Make sure that the file <TT CLASS="filename" >/usr/lib/php4/xslt.so</TT > exists. </P ><P > Now, we need to enable XSLT support in PHP. On a Red Hat 9 system using Apache version 2, we inject the following lines in the PHP included INI files like this: <PRE CLASS="screen" > # echo 'extension=xslt.so' >/etc/php.d/xslt.ini </PRE > On a Red Hat 8 system, we add the following lines to the <TT CLASS="filename" >/etc/php.ini</TT > configuration file: <PRE CLASS="screen" > extension=xslt.so </PRE > </P ><P > That's it. Restart/start apache and ensure that xml, domxml, xslt, and yaz are all enabled: <PRE CLASS="screen" > # echo '<?phpinfo()?>' >/var/www/html/phpinfo.php # /etc/init.d/httpd stop # /etc/init.d/httpd start </PRE > Then open the url <TT CLASS="literal" >http://localhost/phpinfo.php</TT > and inspect the <CODE CLASS="function" >phpinfo()</CODE > output. </P ><P > Proceed to <A HREF="#install-php-yaz" >Section 2.9</A >. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="web-server" >2.3. Web Server</A ></H2 ><P > <A HREF="http://www.php.net" TARGET="_top" >http://www.php.net</A > works with many different web servers. Keystone has only been tested with <A HREF="http://httpd.apache.org/" TARGET="_top" >Apache</A > versions 1.3.23-1.3.27 on Linux. We have also tried Keystone on RedHat 8 and 9, which use Apache 2.0.40. </P ><P > We suggest you compile Apache yourself rather than using a preinstalled one because the Expat library which is included with Apache may conflict with another version of Expat elsewhere. Configure Apache without Expat and with shared libraries enabled, like this: </P ><PRE CLASS="screen" > ./configure --disable-rule=EXPAT --enable-module=so ... </PRE ><P > Compile and install as usual: </P ><PRE CLASS="screen" > $ make $ su # make install </PRE ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > On Solaris for DSO to work, you may have to use configure option <TT CLASS="literal" >--enable-rule=SHARED_CORE</TT >. </P ></TD ></TR ></TABLE ></DIV ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="expat" >2.4. XML parser</A ></H2 ><P > <A HREF="http://expat.sourceforge.net/" TARGET="_top" >Expat</A > is a very common XML parser. On many Linux systems it's already installed, in which case you can ignore the rest of this section. Check if <TT CLASS="filename" >libexpat.so</TT > or <TT CLASS="filename" >libexpat.so.1</TT > is available. </P ><P > If not, get Expat from its <A HREF="http://sourceforge.net/projects/expat/" TARGET="_top" >download page</A >. Configure, compile and install. </P ><PRE CLASS="screen" > $ ./configure $ make $ su # make install </PRE ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="dom" >2.5. DOM XML</A ></H2 ><P > <A HREF="http://www.xmlsoft.org/" TARGET="_top" >GNOME XML</A > offers a DOM API. Check if you already have it on your system by checking if <TT CLASS="filename" >libxml2.a</TT > or <TT CLASS="filename" >libxml2.so</TT > is present. </P ><P > If you don't have libxml already, then download it from the <A HREF="http://www.xmlsoft.org/" TARGET="_top" >home page</A >. Configure, compile and install as always. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="xslt" >2.6. Sablotron XSLT</A ></H2 ><P > <A HREF="http://www.gingerall.com/" TARGET="_top" >Sablotron</A > is a XSLT processor. You can check if it's available by looking for <TT CLASS="filename" >libsablot.a</TT > or <TT CLASS="filename" >libsablot.so</TT >. </P ><P > If these are not present, download Sablotron from the home page. Configure, compile and install as usual. </P ><DIV CLASS="warning" ><P ></P ><TABLE CLASS="warning" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/warning.gif" HSPACE="5" ALT="Warning"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Unfortunately, the Sablotron packages version 1.0 have a nasty bug which segfaults the Keystone applications using the tkl-file xslt macro. This forces any Keystone installation to use the patched sablotron version 1:1.0.1-2 found at our web site. <TT CLASS="filename" >http://ftp.indexdata.dk/pub/sablotron/sablotron-1.0-ID-patched-1.tar.gz</TT > </P ></TD ></TR ></TABLE ></DIV ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="yaz" >2.7. Z39.50 Support</A ></H2 ><P > If Keystone uses Z39.50 to provide searching facilities, <A HREF="http://www.indexdata.dk/yaz/" TARGET="_top" >YAZ</A > and <A HREF="http://www.indexdata.dk/zebra/" TARGET="_top" >Zebra</A > must be installed. Get the source for those these from <A HREF="http://indexdata.dk/software/" TARGET="_top" >the Index Data software area</A >. Configure, compile and install as usual. </P ><P > The PHP server must be able to access the <TT CLASS="literal" >zebraidx</TT > and <TT CLASS="literal" >zebrasrv</TT > binaries. You must edit the top of <TT CLASS="filename" >search.php</TT > and ensure that values of <TT CLASS="literal" >$zebraidx</TT > and <TT CLASS="literal" >$zebrasrv</TT > are correct. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="php" >2.8. PHP</A ></H2 ><P > <A HREF="http://www.php.net/" TARGET="_top" >PHP</A > binds all the XML tools together and a PHP script does the processing of Keystone pages. Download the PHP source from <A HREF="http://www.php.net/downloads.php" TARGET="_top" >here</A >. </P ><P > Configure PHP </P ><PRE CLASS="screen" > ./configure --with-apxs=/usr/local/apache/bin/apxs --enable-sockets \ --with-dom=/usr/local \ --with-xslt-sablot=/usr/local \ --enable-xslt </PRE ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Do <SPAN CLASS="emphasis" ><I CLASS="emphasis" >not</I ></SPAN > enable yaz for PHP in the configure line above, since the PHP/YAZ shipped with PHP 4 versions is old and outdated. Install PHP/YAZ as a <A HREF="http://pecl.php.net/" TARGET="_top" >PECL</A > module instead. See <A HREF="#install-php-yaz" >Section 2.9</A >. </P ></TD ></TR ></TABLE ></DIV ><P > Option <TT CLASS="literal" >--with-apxs</TT > tells PHP where the Apache Extension tool is located - the location given here is the default location for Apache (unless changed with <TT CLASS="literal" >--prefix</TT >). Option <TT CLASS="literal" >--enable-sockets</TT > enables socket support for PHP - no external library is required. Option <TT CLASS="literal" >--with-dom</TT > enables DOM support and the argument specifies prefix of libxml2. Option <TT CLASS="literal" >--with-xslt-sablot</TT > sets the XSLT handler to be Sablotron. Last option <TT CLASS="literal" >--enable-xslt</TT > enables XSLT. </P ><DIV CLASS="tip" ><P ></P ><TABLE CLASS="tip" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/tip.gif" HSPACE="5" ALT="Tip"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P >If you wish to reconfigure PHP you can edit/rerun the script <TT CLASS="filename" >config.nice</TT > which includes the most recently used options. </P ></TD ></TR ></TABLE ></DIV ><P > Compile and install PHP </P ><PRE CLASS="screen" > $ make $ su # make install </PRE ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="install-php-yaz" >2.9. PHP/YAZ</A ></H2 ><P > PHP/YAZ is now distributed as a <A HREF="http://pecl.php.net/" TARGET="_top" >PECL</A > module. Get the source from <A HREF="http://pecl.php.net/package/yaz" TARGET="_top" >here</A >, then install as follows: <PRE CLASS="screen" > $ tar zxf yaz-<version>.tar.gz $ cd yaz-<version> $ phpize $ ./configure --with-php-config=/usr/bin/php-config $ make $ sudo make install </PRE > Make sure that the file <TT CLASS="filename" >/usr/lib/php4/yaz.so</TT > exists after installation. If configure above does not find YAZ, specify option <TT CLASS="literal" >--with-yaz</TT > followed by the YAZ installation prefix (such as <TT CLASS="filename" >/usr/local/</TT >). </P ><P > Ensure that PHP/YAZ is loaded by PHP by ensuring that <TT CLASS="filename" >php.ini</TT > has a line reading: <PRE CLASS="screen" > extension=yaz.so </PRE > Restart Apache after modifying <TT CLASS="filename" >php.ini</TT > and inspect the output of PHP's phpinfo. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="install-keystone-files" >2.10. Installing Keystone files</A ></H2 ><P > You can put Keystone support wherever the Web server reads its normal HTML content. A possible location for a default Apache installation is the document root from which HTML content is served, this might be <TT CLASS="filename" >/var/www/tklite</TT >, or <TT CLASS="filename" >/var/www/html/tklite</TT > (on Red Hat systems). Check your DocumentRoot directive in your Apache config file for details. </P ><P > You can download Keystone files, packages, etc from our <A HREF="http://ftp.indexdata.dk/pub/tkl" TARGET="_top" > Keystone area</A >. </P ><P > Get the Keystone tarball <TT CLASS="filename" >tklite-1.2.0.tar.gz</TT >. Unpack it, and configure, make and make install as usual. </P ><P > <PRE CLASS="screen" > $ ./configure $ make $ su # make install </PRE > </P ><P > This process will create the sub directory <TT CLASS="filename" >/usr/local/share/tklite</TT >, where the core files are installed, and the <TT CLASS="filename" >/usr/local/share/doc/tklite</TT >, where the documentation is installed. Finally, add a symlink to make the files accessible from within the server DocumentRoot directory using the correct one of the following: </P ><P > <PRE CLASS="screen" > # ln -s /usr/local/share/tklite /var/www # ln -s /usr/local/share/tklite /var/www/html </PRE > </P ><DIV CLASS="warning" ><P ></P ><TABLE CLASS="warning" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/warning.gif" HSPACE="5" ALT="Warning"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Make sure that the <TT CLASS="literal" >tklite</TT > directory is physically within the DocumentRoot directory (a symbolic link will do, an Apache <TT CLASS="literal" >Alias</TT > will not!) otherwise you will get mystifying errors like these: </P ><P > <PRE CLASS="screen" > Warning: Unknown scheme 'tkl-header' Warning: Unknown scheme 'tkl-file' Error: Error Number 3, Level 0, Fields; msgtype => error code => 69 module => Sablotron URI => tkl-file://interface.xsl line => 1 msg => unknown encoding '' error </PRE > </P ><P > This is because, although Apache can find the Keystone scripts through the alias, the XSLT processor - which needs to call back into Keystone - knows nothing of Apache's <TT CLASS="literal" >Alias</TT >es, and so can not find the necessary scripts. </P ></TD ></TR ></TABLE ></DIV ><P > Check that the Keystone install area <TT CLASS="filename" >/usr/local/share/tklite</TT > is readable by the userid of the web server - often <TT CLASS="literal" >nobody</TT >. On Debian, the user is <TT CLASS="literal" >www-data</TT >, and on Red Hat systems it is <TT CLASS="literal" >apache</TT >. Check your Apache configuration for details, and change ownership or read-access bits recursively if necessary. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="enable-tkl1" >2.11. Enabling Keystone in Apache 1.3.X</A ></H2 ><P > The Apache configuration must be modified in order to enable Keystone files to be processed by the PHP module. Edit the file <TT CLASS="filename" >httpd.conf</TT >. </P ><P > First, the page type <TT CLASS="filename" >index.tkl</TT > must be added to the list of files for the <TT CLASS="literal" >DirectoryIndex</TT > directive. Example: </P ><PRE CLASS="screen" > <IfModule mod_dir.c> DirectoryIndex index.html index.php ... index.tkl </IfModule> </PRE ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Apache module <TT CLASS="literal" >mod_dir</TT > must be enabled </P ></TD ></TR ></TABLE ></DIV ><P > Second, the tkl-handler must be set. Add the following two lines inbetween the <TT CLASS="literal" >mod_actions</TT > brackets. (If your redhat configuration does not have the mod_actions brakcets, you can add those lines somewhere near the other AddHandler lines) </P ><PRE CLASS="screen" > <IfModule mod_actions> AddHandler tkl-handler .tkl Action tkl-handler /tklite/shell.php ... </IfModule> </PRE ><P > Make sure the last line reflects the actual location of the shell script relative to the document root of the server. </P ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Apache module <TT CLASS="literal" >mod_actions</TT > must be enabled </P ></TD ></TR ></TABLE ></DIV ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="enable-tkl2" >2.12. Enabling Keystone in Apache 2</A ></H2 ><P > You can change the main web-server configuration file <TT CLASS="filename" >httpd.conf</TT > or add a file that is included by Apache. On some systems, there is already a include directory for extensions such as PHP etc. On Red Hat 8 and 9 that directory is <TT CLASS="filename" >/etc/httpd/conf.d</TT >. </P ><P > The following directives has to be added: <PRE CLASS="screen" > AcceptPathInfo On AddHandler tklhandler .tkl Action tklhandler /tklite/shell.php DirectoryIndex index.tkl </PRE > </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="installing-portal" >2.13. Installing the Sample Portal</A ></H2 ><P > A sample portal called Bibliotheca can currently be found at the URL <A HREF="http://ftp.indexdata.dk/pub/tkl/bibliotheca-1.0.tar.gz" TARGET="_top" > bibliotheca</A >. </P ><P > Get the basic portal and unpack it in the HTML root of the web server, or any other convenient directory acessible by the web server. It will create sub directory <TT CLASS="filename" >bibliotheca-1.0</TT >, which you will probably wish to access by a symlink <TT CLASS="filename" >bibliotheca</TT >. Make sure that this directory and its subdirectories are writable by the user of the web server - often <TT CLASS="literal" >nobody</TT >. On Debian, the web server user is <TT CLASS="literal" >www-data</TT >. Check your Apache configuration for details. </P ><P > The semi-structured XML databases bundled with the portal must be indexed before the search engine can use them. Indexing is done in the root directory of the portal (in this case, the <TT CLASS="filename" >bibliotheca</TT > directory) by issuing the following three commands as the user the web server runs as: </P ><P > <PRE CLASS="screen" > zebraidx -l db/server.log -c db/zebra.cfg init zebraidx -l db/server.log -c db/zebra.cfg update articles help links news suggest zebraidx -l db/server.log -c db/zebra.cfg commit </PRE > </P ><P > The <TT CLASS="literal" >init</TT > command is only used once, at the very first time when a new portal is created. The <TT CLASS="literal" >update</TT > and <TT CLASS="literal" >commit</TT > commands are used whenever new documents have been added to the portal. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="starting-zebra" >2.14. Starting the Zebra server</A ></H2 ><P > You must start the Zebra server for each running portal. </P ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Ideally, this should be handled by PHP/Apache, but we have not yet found an good way to do this. </P ></TD ></TR ></TABLE ></DIV ><P > In your portal's <TT CLASS="filename" >db/zebra.cfg</TT > file, check that the <TT CLASS="literal" >profilePath</TT > parameter has the correct value. For a typical Zebra installation, such as one done from Debian or Red Hat packages, the value is <TT CLASS="filename" >/usr/share/idzebra/tab</TT >. </P ><P > Check that the PHP scripts correctly refer to zebraidx/zebrasrv, by inspecting the definitions of <CODE CLASS="varname" >$zebraidx</CODE > and <CODE CLASS="varname" >$zebrasrv</CODE > at the top of <TT CLASS="filename" >.../tklite/search.php</TT >. </P ><P > In the root directory of the portal, start Zebra as follows: <PRE CLASS="screen" > zebrasrv -l db/server.log -c db/zebra.cfg unix:db/socket & </PRE > </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="testing-it" >2.15. Testing it</A ></H2 ><P > You can check the sample portal by visiting <TT CLASS="literal" >http://</TT ><TT CLASS="replaceable" ><I >myhost</I ></TT ><TT CLASS="literal" >/bibliotheca/</TT > . The administration interface is started by using the URL <TT CLASS="literal" >http://</TT ><TT CLASS="replaceable" ><I >myhost</I ></TT ><TT CLASS="literal" >/tklite/admin.php?cwd=</TT ><TT CLASS="replaceable" ><I >site</I ></TT >. </P ><P > Check that the search box (toward the upper right-hand corner of the page) works. If you get an error message like this: <PRE CLASS="screen" > ERROR!!!! 'Connect failed at target unix:/usr/local/src/z39.50/bibliotheca/db/socket' </PRE > That indicates that the Zebra server is not running, or that its socket is not in the correct placed (the <TT CLASS="literal" >db</TT > subdirectory of the portal's root.) See <A HREF="#starting-zebra" >the previous section</A >. </P ><P > If you get an error message that says <PRE CLASS="screen" > ERROR!!!! 'Database unavailable' </PRE > you probably forgot to initialize the zebra database, or something went wrong during the initialization phase. See <A HREF="#installing-portal" > Installing the Portal </A >. </P ><P > To administrate the <TT CLASS="literal" >keystone-portal</TT > on <TT CLASS="literal" >localhost</TT > you would use <TT CLASS="literal" >http://localhost/tklite/admin.php?cwd=bibliotheca</TT >. </P ></DIV ><DIV CLASS="sect1" ><HR><H2 CLASS="sect1" ><A NAME="virtual-hosting" >2.16. Virtual hosting</A ></H2 ><P > For Keystone to work, the <TT CLASS="filename" >shell.php</TT > must always be accessible using the same path. To enforce this, either use Apache's <TT CLASS="literal" >Alias</TT > directive or use a symbolic link. </P ><P > Consider we have Web server which runs <TT CLASS="literal" >www.domain</TT > and you want Keystone to run on virtual host <TT CLASS="literal" >tkl.domain</TT >. The relevant Apache 1.3.X config will look like this: <PRE CLASS="screen" > # If you want to use name-based virtual hosts you need to define at # least one IP address (and port number) for them. # NameVirtualHost 1.2.3.4 # Our primary domain <virtualHost 1.2.3.4> ServerName www.domain </VirtualHost> # Our Keystone domain <virtualHost 1.2.3.4> ServerAdmin tklmanager@domain DocumentRoot /home/tkl/html ServerName tkl.domain ServerAlias tkl ErrorLog /home/tkl/logs/error.log CustomLog /home/tkl/logs/access.log common Alias /tklite/ /var/www/tklite/ </VirtualHost> </PRE > </P ></DIV ></DIV ><DIV CLASS="chapter" ><HR><H1 ><A NAME="keystone-content" ></A >Chapter 3. Content administration</H1 ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > Some the the examples in this section refer to document types associated with the base portal Bibliotheca. They might appear different in another portal with different document types, etc. </P ></TD ></TR ></TABLE ></DIV ><P > The normal way to edit the content of a portal is using Keystone's administration interface (admin). Since the content of the portal is represented as files in the server's file system, it is also possible to maintain the files using a common user shell on the system, or using custom scripts, etc. The admin module allows controlled maintenance of the key portal content using an ordinary web-browser. Editing is controlled to minimize the possibility of mishaps (access is controlled, documents are checked for XML validity prior to storage, updated or added documents are indexed for searching on the fly, etc.). </P ><P > The admin interface is started by providing the URL path for the script admin.php (the exact URL depends on the installation, with the parameter cwd to give the location of the portal based on the root document directory of the web server. </P ><P > For instance, if the test portal has been installed under the directory HTDOCS/keystone on the server at www.indexdata.dk (so that the portal can be reached using the URL http://www.indexdata.dk/keystone/), and the administrative scripts are available under the directory HTDOCS/keystone-admin, the admin interface can be invoked like this: </P ><P > http://www.indexdata.dk/keystone-admin/admin.php?cwd=keystone </P ><P > If all's gone well, the admin interface main window should now appear, and in the following sections, we will describe the individual elements in this interface. </P ><DIV CLASS="note" ><P ></P ><TABLE CLASS="note" WIDTH="100%" BORDER="0" ><TR ><TD WIDTH="25" ALIGN="CENTER" VALIGN="TOP" ><IMG SRC="../images/note.gif" HSPACE="5" ALT="Note"></TD ><TD ALIGN="LEFT" VALIGN="TOP" ><P > In the following sections, we will describe the organisation of files in the the server file system in boxes such as this one. Editors or administrators who intend to use the admin interface only to maintain their portal can skip these boxes if they wish. </P ></TD ></TR ></TABLE ></DIV ><P > 1: Keystone administration interface - main window </P ><P > In the top of the admin interface is a horizontal menu containing special functions. Underneath is the Path from the root of the portal to the current directory. Then comes a listing of sub-directories, if any, and finally a list of documents in the current directory. </P ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="content-dir" >3.1. Directory structure</A ></H2 ><P > The directory structure of a portal will generally have at least a rough correspondence to the structure of the portal (although this is not a requirement or a neccesity). In the base portal Bibliotheca, we find the following main directories: </P ><P ></P ><UL ><LI ><P > Articles, holding local articles </P ></LI ><LI ><P > Help, holding one or more documents comprising a help system </P ></LI ><LI ><P > links, holding a hierarchy of subject groups and external ressources </P ></LI ><LI ><P > news, holding a collection of news items of relevance to the portal </P ></LI ></UL ><P > (It's important to realise that this list is not cast in stone - the portal designer is free to add more, delete unwanted directories, or start from scratch and design his own structure). </P ><P > You can always delete empty directories, rename directories or create new directories, but the display formats associated with a portal may expect to find some directories or files in certain locations (for the base portal, this includes the link directory which is expected to be called 'link'. </P ><P > If you click on a directory name, the admin interface moves to that directory. The directory path in the top of the display can always be used to retrace your steps. </P ><P > The admin interface shows files and directories located at the given level of the server filesystem. However, some system-specific directories are hidden to simplify the interface. </P ><P > The main directory - or root directory - of a portal is defined as the directory what contains a file by the name tkl.config. All functions in the system use this directory as a reference when resolving directory paths. </P ><P > When displaying a file, the administration interface shows both the filename and, if found, the contents of the <SPAN CLASS="emphasis" ><I CLASS="emphasis" >title</I ></SPAN > XML element of the document. When displaying directories, the admin interface attempts to display both the filename and the contents of the <SPAN CLASS="emphasis" ><I CLASS="emphasis" >title</I ></SPAN > element of an index.tkl file in the directory, if one is found. </P ><P > Under the directory listing is a list of the documents stored in the given directory. Most documents can be edited by anyone with editing privileges for the directory, with a couple of important exceptions. The files users.tkl and directory.tkl may only be edited (or viewed) by users with administrative privileges. The file users.tkl lists the administrative and editorial users. The usage of the file directory.tkl is described hereunder. </P ><P > If a document contains a file called index.tkl, then that is the document the user will see if he attempts to view the directory (ie. without referring to an explicit filename within the directory). This corresponds exactly to the file index.html in a conventional website. </P ><P > All directories <SPAN CLASS="emphasis" ><I CLASS="emphasis" >may</I ></SPAN > contain a file called directory.tkl. It defines different qualities that may be associated with a directory (and its subdirectories, except those that themselves contain directory.tkl files). The directory.tkl file (according to the document type directory.xsd) may contain the following elements: </P ><P ></P ><DIV CLASS="variablelist" ><DL ><DT >Searchable</DT ><DD ><P > Determines whether the subdirectory (and subdirectories) should be searchable. </P ></DD ><DT >Auto index type</DT ><DD ><P > Determines whether an index.tkl file should automatically be created when a new subdirectory is created (and if so, what type/schema it should belong to). </P ></DD ><DT >Allowed schema (repeateble)</DT ><DD ><P > Determines whic document types are allowed in a given directory. For instance, in the base portal Bibliotheca, under the "news" directory, only news items may be created, not other types of documents. You can also control how new files of a given type should be named. The 'allowed schema' construction reduces the workload of editors, and ensures that administrative work can be delegated without risk of harm to the portal structure. </P ></DD ></DL ></DIV ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="content-docs" >3.2. Documents</A ></H2 ><P > All documents can (if you have the privileges) be edited using a common editing window. Different document types will look differently in the window, with different input fields, etc. </P ><P > The editor has buttons to validate and store the document, or to return to the admin interface without changing the document. </P ><P > Some element types may be associated with type-specific controls. For instance, an element designated to hold an URL will have a button to check if the URL is 'live' and check if there are already documents in the portal which refer to this URL. Date fields may have a button to automatically insert today's date, while text entry fields designed for larger amounts of text have a "focus" button which pops up a larger edit window. </P ><P > For elements which are language-dependent, the editor is asked to fill in the element in all relevant languages. </P ><P > In the documents, multi-lingual fields are simply repeated, with different xml:lang attributes associated. Prior to processing, the user-shell filters the document (and stylesheet) to remove any elements which belong to another language than the user's current language. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="content-schemas" >3.3. Schemas</A ></H2 ><P > Schema editor is not yet described here. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="content-users" >3.4. User Administration</A ></H2 ><P > The user list is located in the portal main directory. It can only be edited by users with administrator privileges. Here, you can create new users, or change attributes associated with existing users. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="content-base" >3.5. The base portal Bibliotheca</A ></H2 ><P > The base portal consists of four main areas, which are visible as subdirectories when the admin-interface is opened under the root directory. </P ><P ></P ><UL ><LI ><P > News - with the news articles shown in the news window </P ></LI ><LI ><P > Articles - with the local articles in the portal </P ></LI ><LI ><P > Help - with the documents which constitute the help system </P ></LI ><LI ><P > links - which contains the top of the subject hierarchy </P ></LI ></UL ><P > The news items have the simplest structure. If you go to this directory (with the admin interface), you will find that you can only create files of one type - news.xsd. To create a new article, you simply click on the "create document" button and fill in the given fields. The file index.tkl in this directory cannot be edited, since it has no real content (and no associated schema) - it simply forces the user shell to use a specific stylesheet to view the news listing, and this stylesheet in turn accesses the individual news items to construct a listing (but more on all this below, if you're interested in writing or modifying stylesheets). </P ><P > If you go back to the root directory and click on 'articles', you find both another index.tkl file and a collection of subdirectories. Index.tk is the document you see when you click on "Articles" from the main menu of the -end-user interface of the portal. The sub-directories correspond to subdocuments which in the end-user interface will be displayed as a bullet list at the bottom of the page (once you have selected "Articles" from the menu. Each of these subdirectories contain their own index.tkl file which provides the content of this document, and the subdirectories may contain further subdirectories, and so on. By building a hierarchy of articles in this wway, you can construct an entire encyclopaedia or article collection for your portal. It is also possible to extend the portal by creating different article directories, which you can link to from the main menu if desired. Note the file directory.tkl (its structure described above) in the article directory. It is this file which tells the admin interface that this directory may only contain documents of the type document.xsd, and not, for instance, news documents. It would, however, be possible to allow more different document types to co-exist with the document.xsd type. </P ><P > The help directory is structured in the same way as the article directory. </P ><P > The link directory, on the other hand, has a different stucture (again, enforced by the directory.tkl file). In this directory, you will find a collection of subdirectories, corresponding to the top level of the subject hierarchy of the portal. Under these directories you may find further subdirectories, and, in certain cases, metadata about external resources. Every subject directory contains an index.tkl file of the type subject.xsd, which provides the name of the subject group. If resources have been cataloged under any given group, their metadata will be in the corresponding directory, in files of the type link.xsd. You can add new resources at any given level simply by creating new documents of the type 'link', and you can create new sub-catagories at any level simply by creating a new subdirectory. The admin interface will automatically ask you to fill in a subject document for the new subject group/sub-directory. You can freely pick the names of the subdirectories - they are not displayed to the user. </P ><P > As an aside, it would be possible to mix the metadata records with, for example, local articles following the document.xsd schema. However, to handle these correctly would require a minor modification to the XSLT stylesheet which displays the subject groups (more on these stylesheets below). </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="content-searching" >3.6. Searching</A ></H2 ><P > The admin interface ensures that every time a document has been added or modified, it will be made searchable by looking for a directory.tkl file in the current directory or in any parent directory. If the file is to be made searchable, the indexing program is called to immediately add the the file to the index files of the search function. You can also, from the admin interface, perform a total re-indexing of the entire portal (for instance, if you have changed the directories which are to be searchable). </P ></DIV ></DIV ><DIV CLASS="chapter" ><HR><H1 ><A NAME="interface" ></A >Chapter 4. Managing end-user interfaces</H1 ><P > This section describes the Keystone approach to managing end-user interfacees for web portals. In general, the administration of user interfaces - unlike the administration of portal <SPAN CLASS="emphasis" ><I CLASS="emphasis" >content</I ></SPAN >, requires certain technical skills; in particular, a basic understanding of file management and editing under Unix (although this work <SPAN CLASS="emphasis" ><I CLASS="emphasis" >can</I ></SPAN > be done over a remote filesystem from another platform, if required), as well as an understanding of HTML, XML, and XSLT. </P ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-context" >4.1. Document context and the user shell</A ></H2 ><P > In the Apache configuration file, requests for filenames with the suffix ".tkl" are set up to be handled by a script called shell.php, which is part of the Keystone distribution. This is the user shell for Keystone; the program which ensures that a given document is connected with the correct presentation format (in the form of an XSLT stylesheet). </P ><P > The first thing the user shell does is to locate the root directory of the portal to which the givel .tkl file belongs. This is done by examining every directory, starting at the location of the file, and moving upwards, until a file named tkl.config is located. It is an error if this file doesn't exist under the document hierarchy of the current (logical) webserver. The portal root directory is the baseline for various references to file paths that can be made from inside stylesheets. Since the portal root is associated with the location of the file config.tkl, it is easy to run multiple portals under Keystone on a single webserver - each with their own portal root. </P ><P > After this step, the user shell determines which XSLT stylesheet should be used to display the document to the user. To this end, the shell looks up the name of the root, or document tag in the XML document, for instance, "<metaData>". After this, the user shell searches in the current directory and and from there upwards towards the portal root for a .xsl (XSLT stylesheet) file with a matching name (note that this makes it possible to have multiple stylesheets for the same document type in a single portal, because the user shell always chooses the one closest to the document in the directory hierarchy. </P ><P > The next step is to preprocess both the XML (Keystone) document and the stylesheet. In this stage, all XML elements which have an xml:lang attribute not matching the current language selection, are removed. The user selects his language by giving a 'lang' parameter with his HTTP request - generally based on a button or link from the interface - and his selection is stored in a session-persistent variable (based on a cookie). This technique makes it a simple matter to maintain multilingual data and user interfaces. The stylesheet editor simply has to remember to maintain multi-lingual variants of relevant parts of the stylesheet. If a single language-dependent part occurs in the middle of a large block of HTML, the <span> element can be used with an xml:lang attribute to encapsulate any language-dependent parts. </P ><P > The preprocessing stage also looks for declarations of session-persistent variables within the stylesheet (see below). </P ><P > Now, the stylesheet is processed with the document as input, and the result is sent to the user <A NAME="AEN505" HREF="#FTN.AEN505" ><SPAN CLASS="footnote" >[1]</SPAN ></A > </P ><P > Because the stylesheet may need to know something about the document's place in a greater context (unlike typical stylesheets which execute a context-neutral conversion of a document), the user shell makes a collection of parameters and services (functions) available to the stylesheet. These are user parameters that are provided by HTML forms or URL parameters that are made visible as parameters to the stylesheet, and functions encapsulated in private retrieval URL schemes, that can be accessed using XSLT's standard document() function, and which return different kinds of information. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-params" >4.2. Parameters</A ></H2 ><P > Simply by declaring an XSLT parameter using <xsl:param>, the programmer gains access to any user-supplied parameters in the HTTP request (note that the 'lang' parameter is always available). </P ><P > In addition, there are certain system-specific parameters that may be used if needed, in particular: </P ><P ></P ><UL ><LI ><P > root, which gives the portal root directory relative to the web-servers root-directory. This parameter can be used to construct. </P ></LI ><LI ><P > portpath provides the absolute path to the portal root in the server file system. This parameter is rarely used. </P ></LI ></UL ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-session" >4.3. Session parameters</A ></H2 ><P > Sometimes it is useful to associate parameters with the user session, to avoid having to carry an extensive list of parameters around through each link within the system. In the user shell, this can be done using a special tag, for example: </P ><P > <xsl:param name="sort"/> </P ><P > <portcom:session-var name="sort" default="title" </P ><P > xmlns:portcom="http://www.indexdata.dk/keystone"/> </P ><P > The <xsl:param> element introduces a parameter in the usual way. <portcom:session-var> tells the user shell that the given parameter should be stored and associated with the user session. The Default parameter is optional. The stored value is overwritten if the user (via GET/POST parameters) provides an explicit value for the parameter. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-functions" >4.4. Supporting functions</A ></H2 ><P > The supporting functions are absolutely central to the function of the user shell, and it important to understand the possibilities that they make available, if you are to build a serious application using Keystone. </P ><P > Since there is currently no good, standardised way to define and involke external functins from XSLT, we have chosen to implement our supporting functions as private URI schemes that can be used via the document() function, but also, for example, in the <xsl:import> and <xsl:include> elements. </P ><P > In the following sections, we describe the individual support functions. All of the functions return their results in the form of an XML document which can treated in the usual way in XSLT. </P ><P > Please not that when these functions are involked from XSLT, the ampersand (&), which is used to spearate parameters, must be escaped as &. </P ><P > Note also that Keystone allows the portal to provide its own extension functions in either PHP or Perl, without requiring changes to the user shell. This is very useful in situations where a task is simply not well-suited for implementation in XSLT alone. </P ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="tkl-admin-header-section" >4.4.1. tkl-admin-header</A ></H3 ><P > Function: Generates relevant HTTP headers for the admin interfaces. </P ><P > Synopsis: This handler is invoked automatically by the Keystone system. </P ><P > Result: This handler is special in the sense that it is invoked automatically by the shell.php in case the admin interface is about to be loaded. If not overwritten, this handler generates the relevant HTTP headers for authentication. </P ><P > Warning: In general, this handler should not be overwritten except when you know exactly what you're doing! </P ><P > See also <A HREF="#tkl-header-section" ><I >tkl-header</I ></A >. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN539" >4.4.2. tkl-args</A ></H3 ><P > Function: Query HTTP GET'd/POST'd variables. </P ><P > Synopsis: <TT CLASS="literal" >tkl-args:</TT > </P ><P > Result: XML encoding representation of a list of key => value pairs, for instance, </P ><PRE CLASS="screen" > <args> <var1>value1</var1> <var2>value2</var2> </args> </PRE ><P > If an array variable, i.e. </P ><PRE CLASS="screen" > Array ( [key1] => value1 [key2] => value2 ) </PRE ><P > is present, it will be encoded this way: </P ><PRE CLASS="screen" > <args> <list i="key1">value1</list> <list i="key2">value2</list> </args> </PRE ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN550" >4.4.3. tkl-default</A ></H3 ><P > Function: Looks for the presence of a file in portal area and Keystone core area. </P ><P > Synopsis: <TT CLASS="literal" >tkl-default:?path=my_dir/my_file</TT > </P ><P > Result: Path is the name of the file relative to the portal root. The handler looks for this file in the portal area. If it exists, the absolute path is returned. Otherwise, the file is looked for in the Keystone core area. If it exists here, the absolute path is returned, otherwise an error flag is raised. Example: </P ><PRE CLASS="screen" > <tkl-default> <abs_path>/var/www/bibliotheca/my_dir/my_file</abs_path> </tkl-default> </PRE ><P > on success. On error, the returned XML looks like this: </P ><PRE CLASS="screen" > <tkl-default> <error>Error message</error> </tkl-default> </PRE ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN559" >4.4.4. tkl-file</A ></H3 ><P > Function: Opens a Keystone file and filters language dependent elements. </P ><P > Synopsis: <TT CLASS="literal" >tkl-file://subdir/my_file.tkl</TT > </P ><P > Result: This handler returns an XML document tree ready to process in XSLT. It is important not to confuse the "tkl-file" handler with the generic XSLT handler "file". The differences can be summarized this way: </P ><P ></P ><UL ><LI ><P >File name paths are resolved relative to the Keystone portal root rather than relative to the root of the filesystem. Thus, tkl-file://my_file.tkl will look in the portal root for my_file.tkl.</P ></LI ><LI ><P >Language dependent elements are subjected to filtering. A specific language is associated with an XML element with the xml:lang="xx" attribute, e.g.</P ><P > <PRE CLASS="screen" > <tag xml:lang="en">Here goes some CDATA in english</tag> <tag xml:lang="oz">Here goes the CDATA in some other language</tag> </PRE > </P ><P > If the chosen interface language is set to "oz", then elements with, for instance xml:lang="en" will be suppressed. </P ></LI ></UL ><P > The tkl-file handler is well-suited for importing general XML into your stylesheet, Keystone files as well as XSLT stylesheets, or any other type of XML. The tkl-file handler works excellent in combination with the XSLT import and include instructions, i.e. </P ><PRE CLASS="screen" > <xsl:import href="tkl-file://my_stylesheet.xsl"/> <xsl:include href="tkl-file://subdir/some_other_stylesheet.xsl"/> </PRE ><P > which are the preferred XSL stylesheet inclusion idioms. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN576" >4.4.5. tkl-file-exists</A ></H3 ><P > Function: Checks if a specified file exists. </P ><P > Synopsis: <TT CLASS="literal" >tkl-file-exists:?file=path/to/file.tkl</TT > </P ><P > Result: Returns the following XML structure: </P ><PRE CLASS="screen" > <tkl-file-exists> <exists>1</exists> </tkl-file-exists> </PRE ><P > if the file exists, and of course otherwise, </P ><PRE CLASS="screen" > <tkl-file-exists> <exists>0</exists> </tkl-file-exists> </PRE ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN585" >4.4.6. tkl-find</A ></H3 ><P > Function: Finds files which match a given pattern or mask. </P ><P > Synopsis: <TT CLASS="literal" >tkl-find://?path=pathmask&mask=filemask&select=fieldmask&level=number</TT > </P ><P > Result: A list oif <file> elements, one for each matching file, containing the elements of each file given by fieldmask. </P ><P > Example: tkl-find:/?path=*/index.tkl&select=title </P ><PRE CLASS="screen" > <?xml version="1.0" encoding="ISO-8859-1"?> <tkl-find> <file path="./news/index.tkl"> <title>Nu med links!</title> </file> <file path="./oldcheese/index.tkl"> <title>Gammel ost på nye flasker</title> </file> ..... </tkl-find> </PRE ><P > The tkl-find function can be used in two different ways. IN the simple version, it searches for files which match path-parameters (comparable to a simple listing of files matching a filename (glob) pattern in a Unix shell. It returns a number of file elements, containing the elements selected by the fieldmask parameter. </P ><P > The fieldmask parameter has the form element|element|... - in other words, an arbitrary number of element names separated by vertical bars. </P ><P > The function can also be used like the find-command of Unix, to recursively traverse one or more subtrees.This example from the base test portal finds all index.tkl files in subdirectories of links/* two levels down. </P ><P > tkl-find:/?path=links/*&mask=index.tkl&select=title&level=2 </P ><P > Returns: </P ><PRE CLASS="screen" > <?xml version="1.0" encoding="ISO-8859-1"?> <tkl-find> <dir path="./links/27/" level="1" att="2"> <file path="./links/27/index.tkl"> <title xml:lang="en">General Subjects</title> </file> <dir path="./links/27/01/" level="2" att="2"> <file path="./links/27/01/index.tkl"> <title xml:lang="en">Cross-disciplinary subjects</title> </file> </dir> <dir path="./links/27/03/" level="2" att="2"> <file path="./links/27/03/index.tkl"> <title xml:lang="en">Library collections</title> </file> </dir> </dir> ................ </tkl-find> </PRE ><P > (please note that the structure above represents a hierarchy, where directory elements can contain both files and onther directories). </P ><P > If the path expression only matches files, there is no needs to provide a mask. If the path expression matches directories, the function will search these subdirectories, looking for files matching the mask parameter, until level levels (steps) </P ><P > The <dir> and <file> elements both have a path-attribute, which provides a relative path to the file or directory (with respect to the location of the current document).Usually, this means that these paths can be used without modification in, say, a HTML <A> element, as a relative URL. </P ><P > However, please note that if the original path expression is absolute (ie. starts with a '/'), it is processed relative to the root of the <SPAN CLASS="emphasis" ><I CLASS="emphasis" >portal</I ></SPAN >, and the path-attribute in the <dir> and <file> elements can be used directly as a server-absolute URL. For example: </P ><PRE CLASS="screen" > tkl-find://?path=/news/news*.tkl&select=date|title <tkl-find> <file path="/tkl/news/news1.tkl"> <date>2002-07-08</date> <title xml:lang="en">The test has begun</title> </file> <file path="/tkl/news/news429.tkl"> <date>2002-07-10</date> <title xml:lang="en">The news entries have been cleaned out</title> </file> </tkl-find> </PRE ><P > Please note that tkl-find, like the other functions, pre-process their results so that any elements marked with xml:lang attributes not matching the current language are filtered out. Under normal circumstances, the programmer should not have to worry about resource languages in his stylesheets. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN606" >4.4.7. tkl-grant</A ></H3 ><P > Function: Checks for user read/write/create authorization. This handler only makes sense in an administration interface, i.e. where <TT CLASS="literal" >?admin=1</TT >. </P ><P > Synopsis: <TT CLASS="literal" >tkl-grant://?type=create&file=tkl_cwd/new_file.tkl</TT > </P ><P > Result: In case, the user is authorized to perform the given action, the following XML structure is returned: </P ><PRE CLASS="screen" > <tkl-grant> <granted>1</granted> </tkl-grant> </PRE ><P > Otherwise, false is returned, i.e. </P ><PRE CLASS="screen" > <tkl-grant> <granted>0</granted> </tkl-grant> </PRE ><P > The parameters <TT CLASS="literal" >file</TT > and <TT CLASS="literal" >type</TT > are both mandatory. The <TT CLASS="literal" >file</TT > argument specifies at which object the authorization request is targeted, while <TT CLASS="literal" >type</TT > specifies the type of action. The <TT CLASS="literal" >type</TT > argument must be one of the following: </P ><P ></P ><UL ><LI ><P > read - Read permission is requested and the <TT CLASS="literal" >file</TT > argument must contain the the path to a file. </P ></LI ><LI ><P > write - Request write permission, <TT CLASS="literal" >file</TT > can either be a file or a directory. </P ></LI ><LI ><P > create - Ask for permission to create some kind of object, <TT CLASS="literal" >file</TT > must be a directory. </P ></LI ></UL ><P > All paths are relative to the Keystone portal root. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="tkl-header-section" >4.4.8. tkl-header</A ></H3 ><P > Function: Send HTTP headers to the user interface. </P ><P > Synopsis: This handler is called automatically by the Keystone system. Don't attempt to call it from your XSL stylesheets. </P ><P > Result: This Keystone handler is intentionally born empty. When developing a Keystone portal, the programmer is encouraged to overwrite the tkl-header by something meaningful. Typically, one wants to exploit the fact that this handler is called before any output has been generated by the XSLT processor. Thus, for instance HTTP headers can be created here. </P ><P > See also <A HREF="#tkl-admin-header-section" ><I >tkl-admin-header</I ></A >. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN640" >4.4.9. tkl-help</A ></H3 ><P > Function: Find the closest occurence of help.tkl. </P ><P > Synopsis: <TT CLASS="literal" >tkl-help:</TT > </P ><P > Result: This support function is in a state of development and in the future it may be out-phased in favour of a more generalized implementation. But for the time being, it looks in the current working directory for the presence of a help.tkl file. If there is no such file there, it goes one directory level back and looks there. It keeps doing this process, until it finds a help.tkl file or reaches the Keystone portal root. </P ><P > If a help.tkl file was found, the following XML document is returned: </P ><PRE CLASS="screen" > <tkl-help> <file>../../help.tkl</file> </tkl-help> </PRE ><P > where the content of the <TT CLASS="literal" ><file></TT > tag is the relative path from the current working directory to the location of the help.tkl file. </P ><P > Otherwise, the <TT CLASS="literal" ><file></TT > tag is replaced by some error message. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN652" >4.4.10. tkl-mail</A ></H3 ><P > Function: Support function for sending e-mails. </P ><P > Synopsis: <TT CLASS="literal" >tkl-mail:?to=recipient@domain.org&subject=Subject text&message=Message body.</TT > </P ><P > Result: The following mail is sent with the native system's mail transmission agent: </P ><PRE CLASS="screen" > To: recipient@domain.org Subject: Subject text Message body. </PRE ><P > If the mail was sent successfully, the following XML is returned: </P ><PRE CLASS="screen" > <tkl-mail> <status>OK</status> </tkl-mail> </PRE ><P > Otherwise, an error flag is raised: </P ><PRE CLASS="screen" > <tkl-mail> <error>error message</error> </tkl-mail> </PRE ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN663" >4.4.11. tkl-path</A ></H3 ><P > Function: Returns a path of "breadcrumbs" to the root of the portal. </P ><P > Synopsis: <TT CLASS="literal" >tkl-path:/?select=title</TT > </P ><P > Example: </P ><PRE CLASS="screen" > <?xml version="1.0" encoding="ISO-8859-1"?> <tkl-path> <step path='./../../../'> <title xml:lang="en">Bibliotheca</title> </step> <step path='./../'> <title xml:lang="en">Social Sciences</title> </step> <step path='./'> <title xml:lang="en">Law</title> </step> </tkl-path> </PRE ><P > Tkl-path returns an XML representation of the path from a given document to the root of the portal. In the base portal Bibliotheca, in the file interface.xsl, in the template insert-path, is an example of the usage of this function. In the base portal, the function is used to create a clickable path on each side, as a navigational aid. </P ><P > The function operate by examining all directories from the current directory up to the root. For all directories which contain an index.tkl file, it returns a <step> element with a path-attribute providing a relative path. The path is generally suitable for use as a relative URL in an HTML <A> tag. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN672" >4.4.12. tkl-search</A ></H3 ><P > Function: Searches a Z39.50 database </P ><P > Synopsis: <TT CLASS="literal" >tkl-search://unix:/home/indexdata/html/tkl/db/socket?query=@attrset idxpath computer&start=1&syntax=xml&number=6</TT > </P ><P > Example: </P ><PRE CLASS="screen" > <?xml version="1.0" encoding="ISO-8859-1"?> <search> <start>1</start> <number>6</number> <server url="unix:/home/indexdata/html/tkl/db/socket" status="1"> <hits>6</hits> <end>6</end> <record offset="1"> <subject xmlns:idzebra="http://www.indexdata.dk/zebra/"> <title xml:lang="en">Computer science</title> <idzebra:size>153</idzebra:size> <idzebra:localnumber>332</idzebra:localnumber> <idzebra:filename>links/30/05/index.tkl</idzebra:filename> </subject> </record> ............... <record offset="6"> ....... </record> </server> </search> </PRE ><P > Tkl-search executes a search against the given Z39.50 server. The address of the server generally follows the Z39.50 URL format, even though the example aboce is not standard, but an INdex Data specific extension which allows the use of Unix filesystem sockets (Unix fomain sockets) instead of internet host addresses/port numbers. In the local search function of the base portal Bibliotheca, Unix domain sockets are used to avoid having to allocate a new TCP/IP port to every portal running on the same machine. </P ><P > The parameters start, number, and syntax work as you would expect, and queries are given in the PQF format (described at http://www.indexdata.dk/yaz/doc/tools.php#AEN2265) or ISO CCL (XX add documentation for the configuration of CCL field mapping setup). </P ><P > The results come back as shown above. In particular, the users start and number-parameters are repeated in the XML structure. In the <server> element (which may become repeatable if multi-target searching is introduced) is the number of hits, along with the highest record number returned. After this follows a number of tecord elements, each of which contains one retrieval record from the server. </P ><P > In portals like the base portal Bibliotheca, the tkl-search function is used to search the portal's index, which are hosted by a Zebra server. Note that Zebra for each record returns the path of the record in the element <idzebra:filename>. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN683" >4.4.13. tkl-set-footer</A ></H3 ><P > Function: Name a custom handler to be executed after XSLT processing and output has completed. </P ><P > Synopsis: <TT CLASS="literal" >tkl-set-footer:handler-name:?parameters</TT > </P ><P > Result: Returns an empty XML document. Has the side effect of executing the named custom handler AFTER the current stylesheet has completed. The most obvious use for this is to execute a PHP or Perl script after the main page HTML has been output. This script in turn may perform a time-consuming process, i.e. a search, and provide a dynamic update of the user's screen by emitting a series of Javascript commands which alter the document HTML contents. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN689" >4.4.14. tkl-soap</A ></H3 ><P > Function: Provides access to procedure calls on remote systems via the SOAP protocol. </P ><P > Synopsis: <TT CLASS="literal" >tkl-soap:/MyTestService.wsdl?tkl:fun=weirdFunction&Hello</TT > World </P ><P > Example: </P ><PRE CLASS="screen" > <?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2001/12/soap-envelope" > <env:Header> <t:transaction xmlns:t="http://thirdparty.example.org/transaction" env:encodingStyle="http://example.com/encoding" env:mustUnderstand="true" > 5 </t:transaction> </env:Header> <env:Body> <m:reserveAndChargeResponse env:encodingStyle="http://www.w3.org/2001/12/soap-encoding" xmlns:rpc="http://www.w3.org/2001/12/soap-rpc" xmlns:m="http://travelcompany.example.org/"> <rpc:result>m:status</rpc:result> <m:status>confirmed</m:status> <m:reference>FT35ZBQ</m:reference> <m:viewAt> http://travelcompany.example.org/reservations?code=FT35ZBQ </m:viewAt> </m:reserveAndChargeResponse> </env:Body> </env:Envelope> </PRE ><P > Tkl-soap provides access to any SOAP-based, so-called web-service (or remote API) anywhere on the Internet. The parameters to the the function consist of a reference to a service definition file (WSDL), a function name, and a list of parameters. The parameters can be structured in a variety of ways, to enable the use of different types of remote function. </P ><P > The WSDL file can be identified using a relative or portal-absolute path, or as an HTTP URL. </P ><P > Different SOAP functions pose different requirements to the structure of their parameters. In the example here, concat() is used to construct an argument list soleley in order to increase readability. You can imagine that "$query" in the example is a parameter which has been supplied by the user (see above about accessing user-supplied parameters in XSLT stylesheets). </P ><P > INSERT EXAMPLE FROM bibliotheca/soap/google/google.xsl </P ><P > Parameters for the SOAP functions can also be named explicitly, for example: </P ><P > tkl-soap:MyTestService.wsdl?tkl:fun=myFunction&alpha=1&beta=2 </P ><P > Some functions require structured data, for instance, the arguments </P ><P > primitive=test&alpha.a=bob&alpha.b=bub </P ><P > yields the structure: </P ><P > primitive=>test, alpha=>{a=>bob, b=>bub} </P ><P > Substructures can be anonymous, as in </P ><P > .a=bob&.b=bub </P ><P > which yields </P ><P > {a=bob, b=>bub} </P ><P > As a concrete example, Amazon.com can be searched like this: </P ><PRE CLASS="screen" > <xsl:variable name="result" select="document(concat( 'tkl-soap:AmazonWebServices.wsdl?', 'tkl:fun=KeywordSearchRequest', '&.keyword=', $squery, '&.page=2', '&.mode=books', '&.tag=webservices-20', '&.type=lite', '&.devtag=', token, '&.format=xml', '&.version=1.0' ))"/> </PRE ><P > If debugging has been enabled (adding the parameter debug=1 to the URL line), both the SOAP request and response packages will be displayed so it is easy to determine if you have constructed the right parameters for a given web service. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN713" >4.4.15. tkl-sql</A ></H3 ><P > Function: Connect to a MySQL database server and execute SQL instructions. </P ><P > Synopsis I: <TT CLASS="literal" >tkl-sql:?type=connect&host=mysqlhost.mydomain.org&db=sqlbase</TT > </P ><P > Synopsis II: <TT CLASS="literal" >tkl-sql:?type=exec&sql=SELECT * FROM xxxx WHERE yyyy=zzzz</TT > </P ><P > Result: This Keystone handler sends an SQL query to the database specified and encode the returned records as XML. </P ><P > Before any SQL instruction can be carried out, a database connection must be established. This is performed by calling </P ><PRE CLASS="screen" > tkl-sql:?type=connect&host=mysqlhost.mydomain.org&db=sqlbase </PRE ><P > If you need to login as a specific user with password, simply add a <TT CLASS="literal" >&db_user=mysql_user&db_pwd=secret</TT > to the URI. A database connection has a global scope in the sense that you can only have one open connection at a time. If you attempt to open 2 connections, the last connection will overwrite the first. A connection request with <TT CLASS="literal" >type=connect</TT > does not return any meaningful XML. In the future, some kind of reporting should be returned, i.e. success/failure/diagnostics. </P ><P > Assuming one have an open MySQL connection, any number of manipulations can be made during one HTTP session. You execute SQL instructions the following way: </P ><PRE CLASS="screen" > tkl-sql:?type=exec&sql=SELECT this FROM that WHERE xxxx </PRE ><P > When <TT CLASS="literal" >type=exec</TT >, the returned XML should be interpreted as the matching records. The XML structure is the following: </P ><PRE CLASS="screen" > <sql hits="xx" affected="yy"> <record> <field1>value1</field1> <field2>value2</field2> ... ... </record> <record> <field1>value1</field1> <field2>value2</field2> ... ... </record> ... ... </sql> </PRE ><P > The attribute <TT CLASS="literal" >hits</TT > represents the number of matching records, which is only meaningful for <TT CLASS="literal" >SELECT</TT > SQL instructions. For <TT CLASS="literal" >INSERT</TT > and <TT CLASS="literal" >UPDATE</TT > instructions, the <TT CLASS="literal" >affected</TT > attribute will be set to the number of rows affected by the operation. </P ><P > The XML encoding is crucial when non-XML data from an external database is converted into well-formed XML. Therefore, one needs to decide how to handle two aspects of this situation: </P ><P ></P ><UL ><LI ><P > Character encoding, ISO-8859-1 or UTF-8. - If the data is stored as ISO-8859-1 (also known as latin-1), then we need to transform the data into the UTF-8 character set. In this case, you will need to specify the <TT CLASS="literal" >encoding=ISO-8859-1</TT > URI argument when calling the tkl-sql handler. If the data is already stored as UTF-8, you don't need to specify the <TT CLASS="literal" >encoding</TT > argument. </P ></LI ><LI ><P > Encoding of special characters, i.e. <, >, ", ', & - If such special characters are already escaped in the raw data, please specify the <TT CLASS="literal" >xml=1</TT > arguments, telling the tkl-sql handler not to attempt to encode anything with entities. When this setting is not specified, all 5 characters will be replaced by their corresponding named XML entities. </P ></LI ></UL ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN746" >4.4.16. tkl-time</A ></H3 ><P > Function: Return the current time. </P ><P > Synopsis: <TT CLASS="literal" >tkl-time:?format=xxx</TT > </P ><P > Result: Returns the time as a raw string. The optional setting <TT CLASS="literal" >format</TT > specifies how the time should be formatted. Use the conversion specifiers given at the <A HREF="http://www.php.net/manual/en/function.strftime.php" TARGET="_top" >PHP manual pages</A >. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN754" >4.4.17. tkl-unique</A ></H3 ><P > Function: Filenaming support function for the admin interface. </P ><P > Synopsis: <TT CLASS="literal" >tkl-unique:?format=xxxx&type=yyyy</TT > </P ><P > Result: This support function is designed mostly for the admin interface. It returns information about how to name new files respecting the local schemes specified in directory.tkl. The <TT CLASS="literal" >type</TT > argument can be one of the settings allowed in the naming scheme field in directory.tkl: globallyUnique, fixedName and userProvided. The <TT CLASS="literal" >format</TT > setting is the format used when naming files according to the globallyUnique scheme. Use the reserved character % to mark where to put the unique file identifier, e.g. <TT CLASS="literal" >format=link-%.tkl</TT >. </P ><P > The returned XML document has this form: </P ><PRE CLASS="screen" > <tkl-unique> <unique>link-12345678.tkl</unique> <exists>0</exists> <file_spec>0</file_spec> </tkl-unique> </PRE ><P > When <TT CLASS="literal" >type=globallyUnique</TT >, the <TT CLASS="literal" ><unique></TT > tag contains the unique filename based on the specified format and on some dynamically constructed unique ID. For <TT CLASS="literal" >type=fixed</TT >, the <TT CLASS="literal" ><exists></TT > tag will be set to 1, if the fixed file already exists, otherwise it will be set to 0. Finally, if <TT CLASS="literal" >type=userProvided</TT >, the <TT CLASS="literal" ><file_spec></TT > tag will be set to 1. </P ><P > Warning: Don't overwrite this function unless you know exactly what you are doing! </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN773" >4.4.18. tkl-user</A ></H3 ><P > Function: Get information about user currently logged into the admin interface. </P ><P > Synopsis: <TT CLASS="literal" >tkl-user:</TT > </P ><P > Result: Returns an XML document with information about the user currently logged into the admin interface. The structure of the XML is the following: </P ><PRE CLASS="screen" > <tkl-user> <login>user_login</login> <password>ff402ace53f370435</password> <name>Human Readable Name Of This User;/name> <email>user_login@domain.org</email> <usertype>administrator</usertype> <area>/</area> <area>/news</area> <pass_type>md5</pass_type> </tkl-user> </PRE ><P > The <TT CLASS="literal" ><login></TT > tag contains the user's unique ID used when logging in. The <TT CLASS="literal" ><password></TT > is a representation of the password used when logging in. Since Keystone version 1.4.1 new passwords will automatically be stored as MD-5 checksums. To ensure backwards compatibility, the tag <TT CLASS="literal" ><pass_type></TT > is introduced to provide information about the password type used. Currently, this can only be one of the types <TT CLASS="literal" >md5</TT > or <TT CLASS="literal" >clear</TT > or empty/non-existent, the two latter having the same meaning. The <TT CLASS="literal" ><name></TT > and <TT CLASS="literal" ><email></TT > are self-explaining. The <TT CLASS="literal" ><usertype></TT > can be one of the following: </P ><P ></P ><UL ><LI ><P > <TT CLASS="literal" >administrator</TT > - The user logged in belongs to the super-user class and is allowed to do anything within the web-admin-interface. </P ></LI ><LI ><P > <TT CLASS="literal" >user</TT > - Normal user privileges, i.e. have read privileges anywhere and write privileges in home areas, see below. </P ></LI ><LI ><P > <TT CLASS="literal" >spectator</TT > - This is a read-only user. </P ></LI ></UL ><P > Finally, we have the repeatable <TT CLASS="literal" ><area></TT > tag. This is a list of home directories, where the user has write permissions in addition to the granted read permissions given by default. </P ><P > Since this is an admin interface handler, please don't attempt to overwrite it, unless you know what you are doing! </P ></DIV ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-config" >4.5. Portal configuration</A ></H2 ><P > Besides defining the Keystone portal root, the file <TT CLASS="filename" >tkl.config</TT > plays the role of the master portal configuration file. </P ><P > Using <TT CLASS="filename" >tkl.config</TT > as a configuration file is not required. If there is no XML header in the beginning of <TT CLASS="filename" >tkl.config</TT >, it will not be treated as a configuration file, and the default settings will be used instead. </P ><P > If, on the other hand, there is an XML header in the beginning of <TT CLASS="filename" >tkl.config</TT >, the configuration file parser expects an XML structure like this: </P ><PRE CLASS="screen" > <?xml version="1.0"?> <config>  <setting name="setting_name1" value="setting_value1" />  <setting name="setting_name2"> <setting name="sub_setting_a" value="value A" /> <setting name="sub_setting_b" value="value B" /> </setting>  <setting name="setting_name2" value="setting_value2" /> </config> </PRE ><P > Let's run through the configuration settings currently supported by the system: </P ><P ></P ><UL ><LI ><P > <TT CLASS="literal" >base_url</TT > - The public base URL of the Keystone portal. </P ></LI ><LI ><P > <TT CLASS="literal" >from</TT > - The "from" email address used by various robots attempting to send email to users. </P ></LI ><LI ><P > <TT CLASS="literal" >userlang</TT > - Default user interface language. This language will be used, if (a) no explicit language is specified as a URL parameter <TT CLASS="literal" >lang=xx</TT >, and (b) no language is stored in the session variables. </P ></LI ><LI ><P > <TT CLASS="literal" >cache_dir</TT > - Use this directory (under the portal root directory) for the caching. If this directory exists, caching will take place. NB: This feature is only experimental! </P ></LI ><LI ><P > <TT CLASS="literal" >cache_expires</TT > - Number of seconds after which a cached version of a portal web page expires. </P ></LI ><LI ><P > <TT CLASS="literal" >allow_symlinks</TT > - This setting can be set to "yes" or "no" depending on whether you want to allow users to make symlinks between tkl files. Default is "yes". </P ></LI ><LI ><P > <TT CLASS="literal" >languages</TT > - List of languages in which you want to keep language dependent fields. You must specify your language list using this pattern: <PRE CLASS="screen" > <setting name="languages"> <lang name="da" value="Danish"/> <lang name="en" value="English"/> <lang name="xx" value="My language"/> </setting> </PRE > </P ></LI ><LI ><P > <TT CLASS="literal" >xform_xsl</TT > - Here you can specify a stylesheet - as usual relative to the Keystone portal root - which over-writes the built-in Keystone xml editor display stylesheet (<TT CLASS="literal" >xform2.xsl</TT >). Normally, you would like to xsl:import the built-in version of the stylesheet and only overwrite on a template basis. This should be done in your stylesheet using this instruction: <PRE CLASS="screen" > <xsl:import href="xform-xsl:"/> </PRE > </P ></LI ></UL ><P > Some of the settings can be overwritten by dedicated config files, for instance, <TT CLASS="filename" >urlcheck.tkl</TT > which is the config file for the Keystone urlchecker. Confirm man-pages for the Keystone run-time environment. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-debugging" >4.6. Debugging</A ></H2 ><P > By adding the HTTP parameter debug=1 to any given page in a portal, a diagnostic output is produced. The pre-processed document and stylesheet are displayed, as are any documents loaded using Keystone-specific file schemes. The debug option is a very useful tool in tracking down problems with a portal, or simply for finding the best way to handle complex output for one of the support functions. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="interface-base" >4.7. The base portal Bibliotheca</A ></H2 ><P > This section describes the structure of the base portal Bibliotheca, which is included with Keystone. Go through this section if you're contemplating substantial changes to the stylesheets of that portal, or if you're looking for trips about good portal design in Keystone. </P ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN850" >4.7.1. User interface</A ></H3 ><P > The user interface of the base portal is implemented as standard XHTML with a minimum use of graphical elements, and conventional use of CSS stylesheets. This means that elements of the portals appearance can be fine-tuned simply by modifying the accompanying CSS stylesheet. More substantial changes can be carried out by altering the supplied .xsl files - but this requires at least a passing knowledge of HTML and possibly XHTML (although much of the included XSLT_code is fairly self-explanatory). In addition, it helps to have some knowledge about the extension functions made available to XSLT by Keystone. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN853" >4.7.2. Overall page structure - interface.xsl</A ></H3 ><P > This file, which is located in the root directory of the portal (remember that the root directory is the directory containing the file tkl.config), defined the overall framework of the portal interface. Interface.xsl doesn't correspond to any specific document type, but it is included by most of the other stylesheets. If you look in the file, the bulk of its content is a template called "main-page". This template produces the overall HTML (XHTML) structure of each page. Inside the HTML code are calls to different templates which in turn produce the real content of each page. These content-producing templates are provided by the stylesheet which includes interface.xsl, in an interaction which can be compared to "callback functions" in several other programming languages. </P ><P > Interface.xsl provides default versions of the callback templates, mostly to remind the programmer to replace them with something else. Depending on your temperament, you can compare the approach around interface.xsl with object-oriented programming where each document type inherits a basic layout from interface.xsl, overriding specific details of the interface; or as a traditional, callback-based paradigm. </P ><P > In addition the main-html, interface.xsl also defines the templates insert-path, which produces a graphical bread-crumb path to the root of the portal, and menu, which constructs the left-hand side menu, based on data stored in the file index.tkl in the portal root directory <A NAME="AEN858" HREF="#FTN.AEN858" ><SPAN CLASS="footnote" >[2]</SPAN ></A > </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN860" >4.7.3. portal.xsd/xsl</A ></H3 ><P > Typically, there is only going to be one document of the type 'portal' in any given portal, and that's the file index.tkl in the portal root directory, which defines some overall structural information for the portal, and the corresponding portal.xsl, which presents the front page of the portal. But portal.xsl is also a rather typical example of a sylesheet for a document type, and so it's worth looking at it in slightly greater detail. </P ><P > The processing commences by a template which matches the document element - in this case the xpath "/portal". The only thing this template does - and this is typical - is to call the template main-page, which is defined by interface.xsl. It is the template main-page which subsequently calls the other templates in the file - specifically main-news-content and main-body. </P ><P > Main-news-content doesn't do much other than calling an external utility (from news.xsl) to display the latest news. All document pages in this portal use the right-hand column for news, but they don't have to - some designs might use the right-hand column for context-specific information. </P ><P > The template main-body does the real work - it defines what content will appear in the main window, in the centre of the display. In this case, it uses the document() function and the special Keystone extension "tkl-find" to find information about all subject groups under the directory "links", two levels down. The code underneath - the two nested for-each loops - are responsible for showing the headlines to the user, with links to the relevant subject groups. </P ><P > Different types of subject hierarchies might benefit from different presentation styles. If you want to use a radically different subject hierarchy from the simple one shown here, you may want to use a different approach from the one shown here to display the front page of your portal. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN867" >4.7.4. subject.xsd/xsl</A ></H3 ><P > The display format subject.xsl is asociated with files of the type subject. These files are placed - generally with the name index.tkl - in each directory under the "link" directory, and they represent metadata about the individual subject groups. The file subject.xsl follows the same internal structure as portal.xsl - a template matches the document element -- <subject>. From here, main-page is called, which in turn renders the page with the aid of the callback functions. </P ><P > The template main-body performs two tasks. First, it examines whether there are any sub-categories to the current subject group. This is done simply by looking for any sub-directories containing an index.tkl file. Next, the template looks for resources cataloged in the current group. This is done simply by searching for files in the current directory that match the pattern "link*.tkl". </P ><P > Finally, the lists of sub-categories and cataloged files are displayed to the user. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN872" >4.7.5. link.xsd</A ></H3 ><P > The link type is noticeable in that it does not have its own stylesheet associated in this portal. This is because the portal end-user never actually views a link record on its own - they are displayed either by subject.xsl - when the user browses - or by search.xsl (see below) when the user executes a search. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN875" >4.7.6. document.xsd/xsl</A ></H3 ><P > The document type is designed as a general-purpose workhorse - suitable to maintain any type of s simple text document or hierarchy of documents. </P ><P > If you look in document.xsl, you'll notice the the stylesheet both displays the content of the current document, and searches for any subdirectories containing an index.tkl file. If any are found, a list of links is shown in the bottom of the displayed document. </P ><P > In the base portal, document.xsd/xsl are quite simple - they consist only of a title, a creator, an abstract, and a body text (which is assumed to contain either plain text or an XHTML fragment). There are rich possibilities for extending or adapting this document type for specific applications - such as displaying illustrations, chapters, related links (eg. in the right-hand column), etc. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN880" >4.7.7. search.xsl</A ></H3 ><P > Search.xsl executes a search when the user hits the "search" button in the top menu of the portal (and hence submits the associated form). The stylesheet follows the usual structure, and the template main-body does the real work. </P ><P > In the top of the template, three parameters are declared: portpath, query, and start. Portpath is a "built-in" parameter which is made available by the user shell. It contains the aboslute path to the root of the portal - it is used get hold of the communications channel for the search server. The parameters query and start come from the user's HTTP request (ie. from the search form). </P ><P > If a query has been supplied, the real work begins. The search is executed using the function tkl-search, which is made available by the user shell. The search is directed against a Z39.50 server with a Unix domain address called db/socket under the portal root directory. The parameters directing the parsing of the user's query are taken from the index.tkl file. </P ><P > If the search was successful, the number of hits is displayed, then the template "previous-next" (defined further down in the stylesheet is called to produce links to navigate the result set (if necessary). </P ><P > Finally, the records are displayed, in a for-each loop. Here, we check whether the documents are link records with metadata for external documents, or whether they represent internal content. The link (internal or external) is placed in the variable $url. Hits corresponding to entire subject groups are also shown, but differentiated with a different graphical symbol. Finally, the title of the resource is shown, as a hyperlink to the resource, followed by a description, if available. The XSLT fragment which displays each record should be easily extensible if there is a requirement to display more information about each record. </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN887" >4.7.8. news.xsd - newspage.xsl</A ></H3 ><P > The news document type is not shown by its own stylesheet. Instead, news documents are displayed either by the template read-news in news.xsl, which produces the summary right-hand news column, or by the stylesheet newspage.xsl, which is located in the news directory. Newspage.xsl produces the page which is displayed when you ask to see a detailed news listing or a single news item. </P ></DIV ></DIV ></DIV ><DIV CLASS="chapter" ><HR><H1 ><A NAME="oai" ></A >Chapter 5. Keystone and OAI Metadata Services</H1 ><P > Keystone has two distinct interfaces to OAI metadata: it can be used as a OAI repository, thus serving OAI metadata upon valid requests, and it has facilities to harvest OAI metadata from other OAI servers, which then can be shown as usual Keystone record posts using any browser. </P ><P > For general information on the OAI standard we refer to <A HREF="http://www.openarchives.org/" TARGET="_top" > http://www.openarchives.org/</A >. A nice tutuorial can be found at <A HREF="http://www.oaforum.org/tutorial/" TARGET="_top" > http://www.oaforum.org/tutorial/</A >. </P ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="oairep" >5.1. Keystone as an OAI repository</A ></H2 ><P > Keystone has been designed to share its information contents using the Open Archives Initiative (OAI) protocol for metadata harvesting (PMH). The current version supports enough functionality to act as a basic OAI data provider, but the level of protocol support is intended to develop over time as requirements warrant. At present, the following OAI-PMH 'verbs' are supported: <P ></P ><UL ><LI ><P >Identify</P ></LI ><LI ><P >ListMetdataFormats</P ></LI ><LI ><P >ListSets</P ></LI ><LI ><P >ListIdentifiers</P ></LI ><LI ><P >GetRecord</P ></LI ><LI ><P >ListRecords</P ></LI ></UL > </P ><P > OAI-PMH supports different data exchange formats, and requires support for a simple Dublin Core-based format called "oai_dc". </P ><P > Keystone allows a portal to share any type of documents - not just metadata documents. In general, a document is offered for exchange if it is located in a directory marked for exchange with the oai-exchange flag (in the directory.tkl file), and if a suitable conversion filter can be located. The server looks for the filter in the directory schemas under the portal root, and the filename convention is <TT CLASS="filename" >schemaname2metadataprefix.xsl</TT > where the schema name is the name of the root element of the given document, and the metadata prefix is simply the metadata prefix requested by the OAI client (or 'service provider', in OAI parlance). </P ><P > As an example, the base portal Bibliotheca, see <A HREF="#interface-base" >Section 4.7</A >, contains a file named <TT CLASS="filename" >bibliotheca/schemas/link2oai_dc.xsl</TT > which defines the transformation from a link-type document to the oai_dc format. </P ><P > The base URL for the OAI server associated with a Keystone portal is simply the URL for the main (front) page of the portal. If, for instance, a portal has been installed under <TT CLASS="filename" >http://www.indexdata.dk/tkl/</TT > the following request will provide description of the portal according to the OAI-PMH: <PRE CLASS="screen" > http://www.indexdata.dk/tkl/?verb=Identify </PRE > </P ><P > Since version 1.4.4, the Keystone OAI service supports selective harvesting enabling the client to address various sub-sets of the full record collection. Currently, the following OAI harvesting criteria are supported: </P ><P ></P ><UL ><LI ><P > Data/time based harvesting. Use the <TT CLASS="literal" >from</TT > and <TT CLASS="literal" >until</TT > parameters to specify bounded or unbouned time intervals. </P ></LI ><LI ><P > Set based harvesting (since version 1.4.5). Use the <TT CLASS="literal" >set</TT > parameter to specify a set. A record is associated with its set using its document type (XML schema name). Thus, for instance a link record with document type <TT CLASS="literal" >link</TT > and schema <TT CLASS="literal" >link.xsd</TT > belongs to the <TT CLASS="literal" >set=link</TT >. Use the OAI <TT CLASS="literal" >verb=ListSets</TT > call to find out which sets are supported by the OAI service. </P ></LI ></UL ><P > Read more about the OAI protocol at <A HREF="http://www.openarchives.org/" TARGET="_top" >http://www.openarchives.org/</A >. </P ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="oaibibliotheca" >5.2. Bibliotheca example OAI repository</A ></H2 ><P > Assuming that the Bibliotheca portal has been installed under <TT CLASS="filename" >http://localhost/bibliotheca/</TT > the most important OAI-PMH commands are supported as follows (from version 1.4.5): </P ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN940" >5.2.1. Identify</A ></H3 ><P > Function: Provide basic server identification and server capabilities. This is always the initial request. </P ><P > Synopsis: <TT CLASS="literal" >http://localhost/bibliotheca/?verb=Identify</TT > </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN945" >5.2.2. ListMetdataFormats</A ></H3 ><P > Function: Get information on which metadata formats are supported. This is used to choose the desired metadata format for the later fetched records. </P ><P > Synopsis: <TT CLASS="literal" >http://localhost/bibliotheca/?verb=ListMetadataFormats</TT > </P ><P > <PRE CLASS="screen" > http://localhost/bibliotheca/?verb=ListMetadataFormats http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_dc http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_def_portal </PRE > </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN952" >5.2.3. ListSets</A ></H3 ><P > Function: Get information on the names of the possible set to use in an request. </P ><P > Synopsis: <TT CLASS="literal" >http://localhost/bibliotheca/?verb=ListSets</TT > </P ><P > Examples: <PRE CLASS="screen" > http://localhost/bibliotheca/?verb=ListSets http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_dc&set=subject http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_dc&set=link </PRE > </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN959" >5.2.4. ListIdentifiers</A ></H3 ><P > Function: Provide a list of record identifiers which can be used in the next query to determine which records to fetch. </P ><P > Synopsis: <TT CLASS="literal" >http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_dc</TT > </P ><P > Examples: <PRE CLASS="screen" > http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_dc http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_def_portal http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_dc&set=subject http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_dc& from=1978-01-01T12:00:00Z&until=2013-03-20T20:30:00Z http://localhost/bibliotheca/?verb=GetRecord&metadataPrefix=oai_dc& identifier=oai:Unknown.tkl.indexdata.com:links/27/01/index.tkl </PRE > </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN966" >5.2.5. GetRecord</A ></H3 ><P > Function: Obtain one specific OAI record. </P ><P > Synopsis: <TT CLASS="literal" >http://localhost/bibliotheca/?verb=GetRecord&metadataPrefix=oai_dc& identifier=oai:Unknown.tkl.indexdata.com:links/27/01/index.tkl</TT > </P ><P > Examples: <PRE CLASS="screen" > http://localhost/bibliotheca/?verb=ListSets http://localhost/bibliotheca/?verb=ListIdentifiers&metadataPrefix=oai_dc&set=link http://localhost/bibliotheca/?verb=GetRecord&metadataPrefix=oai_dc& identifier=oai:Unknown.tkl.indexdata.com:links/27/01/index.tkl </PRE > </P ></DIV ><DIV CLASS="section" ><HR><H3 CLASS="section" ><A NAME="AEN973" >5.2.6. ListRecords</A ></H3 ><P > Function: Obtain a list of OAI records. A time period or a record set can be specified, as well as another metadata prefix. </P ><P > Synopsis: <TT CLASS="literal" >http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_dc</TT > </P ><P > Examples: <PRE CLASS="screen" > http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_dc http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_def_portal http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_dc&set=link http://localhost/bibliotheca/?verb=ListRecords&metadataPrefix=oai_dc& from=1978-01-01T12:00:00Z&until=2013-03-20T20:30:00Z </PRE > </P ></DIV ></DIV ><DIV CLASS="section" ><HR><H2 CLASS="section" ><A NAME="oaiharv" >5.3. OAI harvesting from within Keystone</A ></H2 ><P > In addition, Keystone allows the system administrator to define OAI harvesting tasks. To do so, one must install the <TT CLASS="filename" >libtkl-perl</TT > and the <TT CLASS="filename" >tkl-oai-harvester</TT > Debian packages. </P ><P > The OAI harvester daemon called <TT CLASS="filename" >tkl-oai-harvester</TT > is started and stopped with the scripts </P ><PRE CLASS="screen" > /etc/init.d/tkl-oai-harvester start /etc/init.d/tkl-oai-harvester stop /etc/init.d/tkl-oai-harvester restart </PRE ><P > When installed with <TT CLASS="filename" >apt-get</TT > as <TT CLASS="filename" >.deb</TT > pakages on a Debian system, these start/stop scripts are installed properly, so that the harvesting service is started automatically at boot time. The start/stop scripts are rather simple and it should not be difficult to adjust them to work with other operating systems than Debian GNU/Linux. </P ><P > Harvesting tasks are created in the admin interface. The <TT CLASS="filename" >bibliotheca</TT > example portal contains the task directory called <TT CLASS="filename" >bibliotheca/tasks</TT >, including two subdirectories <TT CLASS="filename" >oaibizigate</TT >, <TT CLASS="filename" >oaitklite</TT >, and two Keystone files <TT CLASS="filename" >directory.tkl</TT >, and <TT CLASS="filename" >index.tkl</TT >, which are tuned to display the resulting <TT CLASS="filename" >oai*.tkl</TT > files containing the harvested OAI metadata records. </P ><P > Navigate within the admin interface to the <TT CLASS="filename" >bibliotheca/tasks</TT > directory, and add a new oai task file. Fill in the starting url (remember the trailing slash when addressing a Keystone OAI server!). Type the target directory relative to the portal root - for example "/tasks/oaitklite/" - and choose select status = pending. After saving the resulting task file should look like this: <PRE CLASS="screen" > <?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?> <task creator="admin" created="2003-07-10, 13:59:50" modifier="admin" modified="2003-07-10, 13:59:50"> <tasktype>oai</tasktype> <url>http://tkl-cvs.indexdata.dk/bibliotheca/</url> <target>/tasks/oaitklite/</target> <description>OAI harvesting job at our Keystone server</description> <status>pending</status> <xslt>oai2link.xsl</xslt> <handler></handler> </task> </PRE > </P ><P > The content of the <TT CLASS="literal" ><handler></TT > tag is interpreted as an optional script called by the harvester, when the job is finished. There is no restrictions upon the script except that it is run by the web server user, typically <TT CLASS="filename" >www-data</TT >, with the corresponding restrictions. The collection of task handler scripts should be placed in the reserved directory <TT CLASS="filename" >tasks/handlers</TT >. </P ><P > Each such task handler script is called with a single argument, the path to the directory, where the harvested records are placed. </P ><P > Please notice, that when you associate a task handler script other than the trivial one, i.e. <TT CLASS="filename" >do_nothing.handler</TT >, to your OAI harvesting task, this handler is given the responsibility for indexing the harvested records. Otherwise, the OAI harvester daemon <TT CLASS="filename" >tkl-oai-harvester</TT > performs the indexing. </P ><P > The <TT CLASS="literal" ><xslt></TT > tag contains the name of an XSLT transformating stylesheet which will be applied to each OAI harvested record before it is stored. The collection of such XSLT transforming stylesheets should be placed in the reserved directory <TT CLASS="filename" >/authorities/oai</TT >. </P ><P > If the optional <TT CLASS="literal" ><prefix></TT > tag is specified, this will be used as the filename prefix when the OAI harvested records are stored as files on your hard-drive. If nothing is specified, <TT CLASS="filename" >link-</TT > is used as the default value. </P ><P > The OAI protocol does not require the repository to support the <TT CLASS="filename" >set</TT > record filter<A NAME="AEN1018" HREF="#FTN.AEN1018" ><SPAN CLASS="footnote" >[3]</SPAN ></A >. If you want to harvest a set-enabled OAI repository, you can optionally use the <TT CLASS="literal" ><set></TT > setting for this purpose. An empty set value is interpreted as no set value! </P ><P > When an oai task file is saved, a spool file is automatically placed in the <TT CLASS="filename" >/var/spool/tkl</TT > directory, and the OAI harvester will fetch and perform the job within a couple of minutes. During execution of the job, the status tag will change from "pending" over "running" to "finished", and after finishing of the job, the spool file will be removed. </P ><P > The harvested OAI metadata records can be inspected by directing the usual user web interface to <TT CLASS="filename" >bibliotheca/tasks/oaitklite</TT >, where all records are displayed in the fetched order. Clicking at the first link of a record displays some more details of it. </P ><P > Although the OAI records are indexed on system boot, or when running <PRE CLASS="screen" > /etc/init.d/tkl index </PRE > they have been initially marked "hidden" and will not be displayed in search result sets. </P ></DIV ></DIV ></DIV ><H3 CLASS="FOOTNOTES" >Notes</H3 ><TABLE BORDER="0" CLASS="FOOTNOTES" WIDTH="100%" ><TR ><TD ALIGN="LEFT" VALIGN="TOP" WIDTH="5%" ><A NAME="FTN.AEN505" HREF="#AEN505" ><SPAN CLASS="footnote" >[1]</SPAN ></A ></TD ><TD ALIGN="LEFT" VALIGN="TOP" WIDTH="95%" ><P > In most cases, the XSLT stylesheet is expected to produce HTML - or rather XHTML. However, it is easy enough to imagine a portal, or a part of a portal, which outputs structured XML - for instance to support a web services-type interface, </P ></TD ></TR ><TR ><TD ALIGN="LEFT" VALIGN="TOP" WIDTH="5%" ><A NAME="FTN.AEN858" HREF="#AEN858" ><SPAN CLASS="footnote" >[2]</SPAN ></A ></TD ><TD ALIGN="LEFT" VALIGN="TOP" WIDTH="95%" ><P > If a more complex menu structure is required, it is fairly trivial to extend this template (and the corresponding data schema in the file portal.xsd). </P ></TD ></TR ><TR ><TD ALIGN="LEFT" VALIGN="TOP" WIDTH="5%" ><A NAME="FTN.AEN1018" HREF="#AEN1018" ><SPAN CLASS="footnote" >[3]</SPAN ></A ></TD ><TD ALIGN="LEFT" VALIGN="TOP" WIDTH="95%" ><P >The Keystone OAI repository supports the set based record harvesting since version 1.4.5</P ></TD ></TR ></TABLE ></BODY ></HTML >