262 lines
6.8 KiB
Plaintext
262 lines
6.8 KiB
Plaintext
=========================
|
|
CML (Cache Meta Language)
|
|
=========================
|
|
|
|
---------------
|
|
Module: mod_cml
|
|
---------------
|
|
|
|
:Author: Jan Kneschke
|
|
:Date: $Date: 2004/11/03 22:26:05 $
|
|
:Revision: $Revision: 1.2 $
|
|
|
|
:abstract:
|
|
CML is a Meta language to describe the dependencies of a page at one side and building a page from its fragments on the other side using LUA.
|
|
|
|
.. meta::
|
|
:keywords: lighttpd, cml, lua
|
|
|
|
.. contents:: Table of Contents
|
|
|
|
Description
|
|
===========
|
|
|
|
CML (Cache Meta Language) wants to solves several problems:
|
|
|
|
* dynamic content needs caching to perform
|
|
* checking if the content is dirty inside of the application is usually more expensive than sending out the cached data
|
|
* a dynamic page is usually fragmented and the fragments have different livetimes
|
|
* the different fragements can be cached independently
|
|
|
|
Cache Decision
|
|
--------------
|
|
|
|
A simple example should show how to a content caching the very simple way in PHP.
|
|
|
|
jan.kneschke.de has a very simple design:
|
|
|
|
* the layout is taken from a template in templates/jk.tmpl
|
|
* the menu is generated from a menu.csv file
|
|
* the content is coming from files on the local directory named content-1, content-2 and so on
|
|
|
|
The page content is static as long non of the those tree items changes. A change in the layout
|
|
is affecting all pages, a change of menu.csv too, a change of content-x file only affects the
|
|
cached page itself.
|
|
|
|
If we model this in PHP we get: ::
|
|
|
|
<?php
|
|
|
|
## ... fetch all content-* files into $content
|
|
$cachefile = "/cache/dir/to/cached-content";
|
|
|
|
function is_cachable($content, $cachefile) {
|
|
if (!file_exists($cachefile)) {
|
|
return 0;
|
|
} else {
|
|
$cachemtime = filemtime($cachefile);
|
|
}
|
|
|
|
foreach($content as $k => $v) {
|
|
if (isset($v["file"]) &&
|
|
filemtime($v["file"]) > $cachemtime) {
|
|
return 0;
|
|
}
|
|
}
|
|
|
|
if (filemtime("/menu/menu.csv") > $cachemtime) {
|
|
return 0;
|
|
}
|
|
if (filemtime("/templates/jk.tmpl") > $cachemtime) {
|
|
return 0;
|
|
}
|
|
}
|
|
|
|
if (is_cachable(...), $cachefile) {
|
|
readfile($cachefile);
|
|
exit();
|
|
} else {
|
|
# generate content and write it to $cachefile
|
|
}
|
|
?>
|
|
|
|
Quite simple. No magic involved. If the one of the files is new than the cached
|
|
content, the content is dirty and has to be regenerated.
|
|
|
|
Now let take a look at the numbers:
|
|
|
|
* 150 req/s for a Cache-Hit
|
|
* 100 req/s for a Cache-Miss
|
|
|
|
As you can see the increase is not as good as it could be. The main reason as the overhead
|
|
of the PHP interpreter to start up (a byte-code cache has been used here).
|
|
|
|
Moving these decisions out of the PHP script into a server module will remove the need
|
|
to start PHP for a cache-hit.
|
|
|
|
To transform this example into a CML you need 'index.cml' in the list of indexfiles
|
|
and the following index.cml file: ::
|
|
|
|
output_contenttype = "text/html"
|
|
|
|
b = request["DOCUMENT_ROOT"]
|
|
cwd = request["CWD"]
|
|
|
|
output_include = { b .. "_cache.html" }
|
|
|
|
trigger_handler = "index.php"
|
|
|
|
if file_mtime(b .. "../lib/php/menu.csv") > file_mtime(cwd .. "_cache.html") or
|
|
file_mtime(b .. "templates/jk.tmpl") > file_mtime(cwd .. "_cache.html") or
|
|
file_mtime(b .. "content.html") > file_mtime(cwd .. "_cache.html") then
|
|
return CACHE_MISS
|
|
else
|
|
return CACHE_HIT
|
|
end
|
|
|
|
Numbers again:
|
|
|
|
* 4900 req/s for Cache-Hit
|
|
* 100 req/s for Cache-Miss
|
|
|
|
Content Assembling
|
|
------------------
|
|
|
|
Sometimes the different fragment are already generated externally. You have to cat them together: ::
|
|
|
|
<?php
|
|
readfile("head.html");
|
|
readfile("menu.html");
|
|
readfile("spacer.html");
|
|
readfile("db-content.html");
|
|
readfile("spacer2.html");
|
|
readfile("news.html");
|
|
readfile("footer.html");
|
|
?>
|
|
|
|
We we can do the same several times faster directly in the webserver.
|
|
|
|
Don't forget: Webserver are built to send out static content, that is what they can do best.
|
|
|
|
The index.cml for this looks like: ::
|
|
|
|
output_content_type = "text/html"
|
|
|
|
cwd = request["CWD"]
|
|
|
|
output_include = { cwd .. "head.html",
|
|
cwd .. "menu.html",
|
|
cwd .. "spacer.html",
|
|
cwd .. "db-content.html",
|
|
cwd .. "spacer2.html",
|
|
cwd .. "news.html",
|
|
cwd .. "footer.html" }
|
|
|
|
return CACHE_HIT
|
|
|
|
Now we get about 10000 req/s instead of 600 req/s.
|
|
|
|
Power Magnet
|
|
------------
|
|
|
|
Next to all the features about Cache Decisions CML can do more. Starting
|
|
with lighttpd 1.4.9 a power-magnet was added which attracts each request
|
|
and allows you to manipulate the request for your needs.
|
|
|
|
We want to display a maintainance page by putting a file in a specified
|
|
place:
|
|
|
|
We enable the power magnet: ::
|
|
|
|
cml.power-magnet = "/home/www/power-magnet.cml"
|
|
|
|
and create /home/www/power-magnet.cml with: ::
|
|
|
|
dr = request["DOCUMENT_ROOT"]
|
|
|
|
if file_isreg(dr .. 'maintainance.html') then
|
|
output_include = { 'maintainance.html' }
|
|
return CACHE_HIT
|
|
end
|
|
|
|
return CACHE_MISS
|
|
|
|
For each requested file the /home/www/power-magnet.cml is executed which
|
|
checks if maintainance.html exists in the docroot and displays it
|
|
instead of handling the usual request.
|
|
|
|
Another example, create thumbnail for requested image and serve it instead
|
|
of sending the big image: ::
|
|
|
|
## image-url is /album/baltic_winter_2005.jpg
|
|
## no params -> 640x480 is served
|
|
## /album/baltic_winter_2005.jpg/orig for full size
|
|
## /album/baltic_winter_2005.jpg/thumb for thumbnail
|
|
|
|
dr = request["DOCUMENT_ROOT"]
|
|
sn = request["SCRIPT_NAME"]
|
|
|
|
## to be continued :) ...
|
|
|
|
trigger_handler = '/gen_image.php'
|
|
|
|
return CACHE_MISS
|
|
|
|
|
|
Installation
|
|
============
|
|
|
|
You need `lua <http://www.lua.org/>`_ and should install `libmemcache-1.3.x <http://people.freebsd.org/~seanc/libmemcache/>`_ and have to configure lighttpd with: ::
|
|
|
|
./configure ... --with-lua --with-memcache
|
|
|
|
To use the plugin you have to load it: ::
|
|
|
|
server.modules = ( ..., "mod_cml", ... )
|
|
|
|
Options
|
|
=======
|
|
|
|
:cml.extension:
|
|
the file extension that is bound to the cml-module
|
|
:cml.memcache-hosts:
|
|
hosts for the memcache.* functions
|
|
:cml.memcache-namespace:
|
|
(not used yet)
|
|
:cml.power-magnet:
|
|
a cml file that is executed for each request
|
|
|
|
Language
|
|
========
|
|
|
|
The language used for CML is provided by `LUA <http://www.lua.org/>`_.
|
|
|
|
Additionally to the functions provided by lua mod_cml provides: ::
|
|
|
|
tables:
|
|
|
|
request
|
|
- REQUEST_URI
|
|
- SCRIPT_NAME
|
|
- SCRIPT_FILENAME
|
|
- DOCUMENT_ROOT
|
|
- PATH_INFO
|
|
- CWD
|
|
- BASEURI
|
|
|
|
get
|
|
- parameters from the query-string
|
|
|
|
functions:
|
|
string md5(string)
|
|
number file_mtime(string)
|
|
string memcache_get_string(string)
|
|
number memcache_get_long(string)
|
|
boolean memcache_exists(string)
|
|
|
|
|
|
What ever your script does, it has to return either CACHE_HIT or CACHE_MISS.
|
|
It case a error occures check the error-log, the user will get a error 500. If you don't like
|
|
the standard error-page use ``server.errorfile-prefix``.
|
|
|