lighttpd
/
lighttpd1.4
2
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

[doc] dos2unix doc/magnet.txt 

git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2402 152afb58-edef-0310-8abb-c4023f1b3aa9
svn/tags/lighttpd-1.4.22
Stefan Bühler 14 years ago
parent
515d800f7c
commit
e47ee9b8ba
1 changed files with 429 additions and 429 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 858
      doc/magnet.txt

858
doc/magnet.txt
Unescape View File

@ -1,429 +1,429 @@
{{{
#!rst
==============
a power-magnet
==============
------------------
Module: mod_magnet
------------------
.. contents:: Table of Contents
Requirements
============
:Version: lighttpd 1.4.12 or higher
:Packages: lua >= 5.1
Overview
========
mod_magnet is a module to control the request handling in lighty.
.. note::
Keep in mind that the magnet is executed in the core of lighty. EVERY long-running operation is blocking
ALL connections in the server. You are warned. For time-consuming or blocking scripts use mod_fastcgi and friends.
For performance reasons mod_magnet caches the compiled script. For each script-run the script itself is checked for
freshness and recompile if neccesary.
Installation
============
mod_magnet needs a lighty which is compiled with the lua-support ( --with-lua). Lua 5.1 or higher are required by
the module. Use "--with-lua=lua5.1" to install on Debian and friends. ::
server.modules = ( ..., "mod_magnet", ... )
Options
=======
mod_magnet can attract a request in several stages in the request-handling.
* either at the same level as mod_rewrite, before any parsing of the URL is done
* or at a later stage, when the doc-root is known and the physical-path is already setup
It depends on the purpose of the script which stage you want to intercept. Usually you want to use
the 2nd stage where the physical-path which relates to your request is known. At this level you
can run checks against lighty.env["physical.path"].
::
magnet.attract-raw-url-to = ( ... )
magnet.attract-physical-path-to = ( ... )
You can define multiple scripts when separated by a semicolon. The scripts are executed in the specified
order. If one of them a returning a status-code, the following scripts will not be executed.
Tables
======
Most of the interaction between between mod_magnet and lighty is done through tables. Tables in lua are hashes (Perl), dictionaries (Java), arrays (PHP), ...
Request-Environment
-------------------
Lighttpd has its internal variables which are exported as read/write to the magnet.
If "http://example.org/search.php?q=lighty" is requested this results in a request like ::
GET /search.php?q=lighty HTTP/1.1
Host: example.org
When you are using ``attract-raw-url-to`` you can access the following variables:
* parts of the request-line
* lighty.env["request.uri"] = "/search.php?q=lighty"
* HTTP request-headers
* lighty.request["Host"] = "example.org"
Later in the request-handling, the URL is splitted, cleaned up and turned into a physical path name:
* parts of the URI
* lighty.env["uri.path"] = "/search.php"
* lighty.env["uri.path-raw"] = "/search.php"
* lighty.env["uri.scheme"] = "http"
* lighty.env["uri.authority"] = "example.org"
* lighty.env["uri.query"] = "q=lighty"
* filenames, pathnames
* lighty.env["physical.path"] = "/my-docroot/search.php"
* lighty.env["physical.rel-path"] = "/search.php"
* lighty.env["physical.doc-root"] = "/my-docroot"
All of them are readable, not all of the are writable (or don't have an effect if you write to them).
As a start, you might want to use those variables for writing: ::
-- 1. simple rewriting is done via the request.uri
lighty.env["request.uri"] = ...
return lighty.RESTART_REQUEST
-- 2. changing the physical-path
lighty.env["physical.path"] = ...
-- 3. changing the query-string
lighty.env["uri.query"] = ...
Response Headers
----------------
If you want to set a response header for your request, you can add a field to the lighty.header[] table: ::
lighty.header["Content-Type"] = "text/html"
Sending Content
===============
You can generate your own content and send it out to the clients. ::
lighty.content = { "<pre>", { filename = "/etc/passwd" }, "</pre>" }
lighty.header["Content-Type"] = "text/html"
return 200
The lighty.content[] table is executed when the script is finished. The elements of the array are processed left to right and the elements can either be a string or a table. Strings are included AS IS into the output of the request.
* Strings
* are included as is
* Tables
* filename = "<absolute-path>" is required
* offset = <number> [default: 0]
* length = <number> [default: size of the file - offset]
Internally lighty will use the sendfile() call to send out the static files at full speed.
Status Codes
============
You might have seen it already in other examples: In case you are handling the request completly in the magnet you
can return your own status-codes. Examples are: Redirected, Input Validation, ... ::
if (lighty.env["uri.scheme"] == "http") then
lighty.header["Location"] = "https://" .. lighty.env["uri.authority"] .. lighty.env["request.uri"]
return 302
end
You every number above and equal to 100 is taken as final status code and finishes the request. No other modules are
executed after this return.
A special return-code is lighty.RESTART_REQUEST (currently equal to 99) which is usually used in combination with
changing the request.uri in a rewrite. It restarts the splitting of the request-uri again.
If you return nothing (or nil) the request-handling just continues.
Debugging
=========
To easy debugging we overloaded the print()-function in lua and redirect the output of print() to the error-log. ::
print("Host: " .. lighty.request["Host"])
print("Request-URI: " .. lighty.env["request.uri"])
Examples
========
Sending text-files as HTML
--------------------------
This is a bit simplistic, but it illustrates the idea: Take a text-file and cover it in a <pre> tag.
Config-file ::
magnet.attract-physical-path-to = server.docroot + "/readme.lua"
readme.lua ::
lighty.content = { "<pre>", { filename = "/README" }, "</pre>" }
lighty.header["Content-Type"] = "text/html"
return 200
Maintainance pages
------------------
Your side might be on maintainance from time to time. Instead of shutting down the server confusing all
users, you can just send a maintainance page.
Config-file ::
magnet.attract-physical-path-to = server.docroot + "/maintainance.lua"
maintainance.lua ::
require "lfs"
if (nil == lfs.attributes(lighty.env["physical.doc-root"] .. "/maintainance.html")) then
lighty.content = ( lighty.env["physical.doc-root"] .. "/maintainance.html" )
lighty.header["Content-Type"] = "text/html"
return 200
end
mod_flv_streaming
-----------------
Config-file ::
magnet.attract-physical-path-to = server.docroot + "/flv-streaming.lua"
flv-streaming.lua::
if (lighty.env["uri.query"]) then
-- split the query-string
get = {}
for k, v in string.gmatch(lighty.env["uri.query"], "(%w+)=(%w+)") do
get[k] = v
end
if (get["start"]) then
-- missing: check if start is numeric and positive
-- send te FLV header + a seek into the file
lighty.content = { "FLV\x1\x1\0\0\0\x9\0\0\0\x9",
{ filename = lighty.env["physical.path"], offset = get["start"] } }
lighty.header["Content-Type"] = "video/x-flv"
return 200
end
end
selecting a random file from a directory
----------------------------------------
Say, you want to send a random file (ad-content) from a directory.
To simplify the code and to improve the performance we define:
* all images have the same format (e.g. image/png)
* all images use increasing numbers starting from 1
* a special index-file names the highest number
Config ::
server.modules += ( "mod_magnet" )
magnet.attract-physical-path-to = "random.lua"
random.lua ::
dir = lighty.env["physical.path"]
f = assert(io.open(dir .. "/index", "r"))
maxndx = f:read("*all")
f:close()
ndx = math.random(maxndx)
lighty.content = { { filename = dir .. "/" .. ndx }}
lighty.header["Content-Type"] = "image/png"
return 200
denying illegal character sequences in the URL
----------------------------------------------
Instead of implementing mod_security, you might just want to apply filters on the content
and deny special sequences that look like SQL injection.
A common injection is using UNION to extend a query with another SELECT query.
::
if (string.find(lighty.env["request.uri"], "UNION%s")) then
return 400
end
Traffic Quotas
--------------
If you only allow your virtual hosts a certain amount for traffic each month and want to
disable them if the traffic is reached, perhaps this helps: ::
host_blacklist = { ["www.example.org"] = 0 }
if (host_blacklist[lighty.request["Host"]]) then
return 404
end
Just add the hosts you want to blacklist into the blacklist table in the shown way.
Complex rewrites
----------------
If you want to implement caching on your document-root and only want to regenerate
content if the requested file doesn't exist, you can attract the physical.path: ::
magnet.attract-physical-path-to = ( server.document-root + "/rewrite.lua" )
rewrite.lua ::
require "lfs"
attr = lfs.attributes(lighty.env["physical.path"])
if (not attr) then
-- we couldn't stat() the file for some reason
-- let the backend generate it
lighty.env["uri.path"] = "/dispatch.fcgi"
lighty.env["physical.rel-path"] = lighty.env["uri.path"]
lighty.env["physical.path"] = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"]
fi
luafilesystem
+++++++++++++
We are requiring the lua-module 'lfs' (http://www.keplerproject.org/luafilesystem/).
I had to compile lfs myself for lua-5.1 which required a minor patch as compat-5.1 is not needed::
$ wget http://luaforge.net/frs/download.php/1487/luafilesystem-1.2.tar.gz
$ wget http://www.lighttpd.net/download/luafilesystem-1.2-lua51.diff
$ gzip -cd luafilesystem-1.2.tar.gz | tar xf -
$ cd luafilesystem-1.2
$ patch -ls -p1 < ../luafilesystem-1.2-lua51.diff
$ make install
It will install lfs.so into /usr/lib/lua/5.1/ which is where lua expects the extensions on my system.
SuSE and Gentoo are known to have their own lfs packages and don't require a compile.
Usertracking
------------
... or how to store data globally in the script-context:
Each script has its own script-context. When the script is started it only contains the lua-functions
and the special lighty.* name-space. If you want to save data between script runs, you can use the global-script
context:
::
if (nil == _G["usertrack"]) then
_G["usertrack"] = {}
end
if (nil == _G["usertrack"][lighty.request["Cookie"]]) then
_G["usertrack"][lighty.request["Cookie"]]
else
_G["usertrack"][lighty.request["Cookie"]] = _G["usertrack"][lighty.request["Cookie"]] + 1
end
print _G["usertrack"][lighty.request["Cookie"]]
The global-context is per script. If you update the script without restarting the server, the context will still be maintained.
Counters
--------
mod_status support a global statistics page and mod_magnet allows to add and update values in the status page:
Config ::
status.statistics-url = "/server-counters"
magnet.attract-raw-url-to = server.docroot + "/counter.lua"
counter.lua ::
lighty.status["core.connections"] = lighty.status["core.connections"] + 1
Result::
core.connections: 7
fastcgi.backend.php-foo.0.connected: 0
fastcgi.backend.php-foo.0.died: 0
fastcgi.backend.php-foo.0.disabled: 0
fastcgi.backend.php-foo.0.load: 0
fastcgi.backend.php-foo.0.overloaded: 0
fastcgi.backend.php-foo.1.connected: 0
fastcgi.backend.php-foo.1.died: 0
fastcgi.backend.php-foo.1.disabled: 0
fastcgi.backend.php-foo.1.load: 0
fastcgi.backend.php-foo.1.overloaded: 0
fastcgi.backend.php-foo.load: 0
Porting mod_cml scripts
-----------------------
mod_cml got replaced by mod_magnet.
A CACHE_HIT in mod_cml::
output_include = { "file1", "file2" }
return CACHE_HIT
becomes::
content = { { filename = "/path/to/file1" }, { filename = "/path/to/file2"} }
return 200
while a CACHE_MISS like (CML) ::
trigger_handler = "/index.php"
return CACHE_MISS
becomes (magnet) ::
lighty.env["request.uri"] = "/index.php"
return lighty.RESTART_REQUEST
}}}
{{{
#!rst
==============
a power-magnet
==============
------------------
Module: mod_magnet
------------------
.. contents:: Table of Contents
Requirements
============
:Version: lighttpd 1.4.12 or higher
:Packages: lua >= 5.1
Overview
========
mod_magnet is a module to control the request handling in lighty.
.. note::
Keep in mind that the magnet is executed in the core of lighty. EVERY long-running operation is blocking
ALL connections in the server. You are warned. For time-consuming or blocking scripts use mod_fastcgi and friends.
For performance reasons mod_magnet caches the compiled script. For each script-run the script itself is checked for
freshness and recompile if neccesary.
Installation
============
mod_magnet needs a lighty which is compiled with the lua-support ( --with-lua). Lua 5.1 or higher are required by
the module. Use "--with-lua=lua5.1" to install on Debian and friends. ::
server.modules = ( ..., "mod_magnet", ... )
Options
=======
mod_magnet can attract a request in several stages in the request-handling.
* either at the same level as mod_rewrite, before any parsing of the URL is done
* or at a later stage, when the doc-root is known and the physical-path is already setup
It depends on the purpose of the script which stage you want to intercept. Usually you want to use
the 2nd stage where the physical-path which relates to your request is known. At this level you
can run checks against lighty.env["physical.path"].
::
magnet.attract-raw-url-to = ( ... )
magnet.attract-physical-path-to = ( ... )
You can define multiple scripts when separated by a semicolon. The scripts are executed in the specified
order. If one of them a returning a status-code, the following scripts will not be executed.
Tables
======
Most of the interaction between between mod_magnet and lighty is done through tables. Tables in lua are hashes (Perl), dictionaries (Java), arrays (PHP), ...
Request-Environment
-------------------
Lighttpd has its internal variables which are exported as read/write to the magnet.
If "http://example.org/search.php?q=lighty" is requested this results in a request like ::
GET /search.php?q=lighty HTTP/1.1
Host: example.org
When you are using ``attract-raw-url-to`` you can access the following variables:
* parts of the request-line
* lighty.env["request.uri"] = "/search.php?q=lighty"
* HTTP request-headers
* lighty.request["Host"] = "example.org"
Later in the request-handling, the URL is splitted, cleaned up and turned into a physical path name:
* parts of the URI
* lighty.env["uri.path"] = "/search.php"
* lighty.env["uri.path-raw"] = "/search.php"
* lighty.env["uri.scheme"] = "http"
* lighty.env["uri.authority"] = "example.org"
* lighty.env["uri.query"] = "q=lighty"
* filenames, pathnames
* lighty.env["physical.path"] = "/my-docroot/search.php"
* lighty.env["physical.rel-path"] = "/search.php"
* lighty.env["physical.doc-root"] = "/my-docroot"
All of them are readable, not all of the are writable (or don't have an effect if you write to them).
As a start, you might want to use those variables for writing: ::
-- 1. simple rewriting is done via the request.uri
lighty.env["request.uri"] = ...
return lighty.RESTART_REQUEST
-- 2. changing the physical-path
lighty.env["physical.path"] = ...
-- 3. changing the query-string
lighty.env["uri.query"] = ...
Response Headers
----------------
If you want to set a response header for your request, you can add a field to the lighty.header[] table: ::
lighty.header["Content-Type"] = "text/html"
Sending Content
===============
You can generate your own content and send it out to the clients. ::
lighty.content = { "<pre>", { filename = "/etc/passwd" }, "</pre>" }
lighty.header["Content-Type"] = "text/html"
return 200
The lighty.content[] table is executed when the script is finished. The elements of the array are processed left to right and the elements can either be a string or a table. Strings are included AS IS into the output of the request.
* Strings
* are included as is
* Tables
* filename = "<absolute-path>" is required
* offset = <number> [default: 0]
* length = <number> [default: size of the file - offset]
Internally lighty will use the sendfile() call to send out the static files at full speed.
Status Codes
============
You might have seen it already in other examples: In case you are handling the request completly in the magnet you
can return your own status-codes. Examples are: Redirected, Input Validation, ... ::
if (lighty.env["uri.scheme"] == "http") then
lighty.header["Location"] = "https://" .. lighty.env["uri.authority"] .. lighty.env["request.uri"]
return 302
end
You every number above and equal to 100 is taken as final status code and finishes the request. No other modules are
executed after this return.
A special return-code is lighty.RESTART_REQUEST (currently equal to 99) which is usually used in combination with
changing the request.uri in a rewrite. It restarts the splitting of the request-uri again.
If you return nothing (or nil) the request-handling just continues.
Debugging
=========
To easy debugging we overloaded the print()-function in lua and redirect the output of print() to the error-log. ::
print("Host: " .. lighty.request["Host"])
print("Request-URI: " .. lighty.env["request.uri"])
Examples
========
Sending text-files as HTML
--------------------------
This is a bit simplistic, but it illustrates the idea: Take a text-file and cover it in a <pre> tag.
Config-file ::
magnet.attract-physical-path-to = server.docroot + "/readme.lua"
readme.lua ::
lighty.content = { "<pre>", { filename = "/README" }, "</pre>" }
lighty.header["Content-Type"] = "text/html"
return 200
Maintainance pages
------------------
Your side might be on maintainance from time to time. Instead of shutting down the server confusing all
users, you can just send a maintainance page.
Config-file ::
magnet.attract-physical-path-to = server.docroot + "/maintainance.lua"
maintainance.lua ::
require "lfs"
if (nil == lfs.attributes(lighty.env["physical.doc-root"] .. "/maintainance.html")) then
lighty.content = ( lighty.env["physical.doc-root"] .. "/maintainance.html" )
lighty.header["Content-Type"] = "text/html"
return 200
end
mod_flv_streaming
-----------------
Config-file ::
magnet.attract-physical-path-to = server.docroot + "/flv-streaming.lua"
flv-streaming.lua::
if (lighty.env["uri.query"]) then
-- split the query-string
get = {}
for k, v in string.gmatch(lighty.env["uri.query"], "(%w+)=(%w+)") do
get[k] = v
end
if (get["start"]) then
-- missing: check if start is numeric and positive
-- send te FLV header + a seek into the file
lighty.content = { "FLV\x1\x1\0\0\0\x9\0\0\0\x9",
{ filename = lighty.env["physical.path"], offset = get["start"] } }
lighty.header["Content-Type"] = "video/x-flv"
return 200
end
end
selecting a random file from a directory
----------------------------------------
Say, you want to send a random file (ad-content) from a directory.
To simplify the code and to improve the performance we define:
* all images have the same format (e.g. image/png)
* all images use increasing numbers starting from 1
* a special index-file names the highest number
Config ::
server.modules += ( "mod_magnet" )
magnet.attract-physical-path-to = "random.lua"
random.lua ::
dir = lighty.env["physical.path"]
f = assert(io.open(dir .. "/index", "r"))
maxndx = f:read("*all")
f:close()
ndx = math.random(maxndx)
lighty.content = { { filename = dir .. "/" .. ndx }}
lighty.header["Content-Type"] = "image/png"
return 200
denying illegal character sequences in the URL
----------------------------------------------
Instead of implementing mod_security, you might just want to apply filters on the content
and deny special sequences that look like SQL injection.
A common injection is using UNION to extend a query with another SELECT query.
::
if (string.find(lighty.env["request.uri"], "UNION%s")) then
return 400
end
Traffic Quotas
--------------
If you only allow your virtual hosts a certain amount for traffic each month and want to
disable them if the traffic is reached, perhaps this helps: ::
host_blacklist = { ["www.example.org"] = 0 }
if (host_blacklist[lighty.request["Host"]]) then
return 404
end
Just add the hosts you want to blacklist into the blacklist table in the shown way.
Complex rewrites
----------------
If you want to implement caching on your document-root and only want to regenerate
content if the requested file doesn't exist, you can attract the physical.path: ::
magnet.attract-physical-path-to = ( server.document-root + "/rewrite.lua" )
rewrite.lua ::
require "lfs"
attr = lfs.attributes(lighty.env["physical.path"])
if (not attr) then
-- we couldn't stat() the file for some reason
-- let the backend generate it
lighty.env["uri.path"] = "/dispatch.fcgi"
lighty.env["physical.rel-path"] = lighty.env["uri.path"]
lighty.env["physical.path"] = lighty.env["physical.doc-root"] .. lighty.env["physical.rel-path"]
fi
luafilesystem
+++++++++++++
We are requiring the lua-module 'lfs' (http://www.keplerproject.org/luafilesystem/).
I had to compile lfs myself for lua-5.1 which required a minor patch as compat-5.1 is not needed::
$ wget http://luaforge.net/frs/download.php/1487/luafilesystem-1.2.tar.gz
$ wget http://www.lighttpd.net/download/luafilesystem-1.2-lua51.diff
$ gzip -cd luafilesystem-1.2.tar.gz | tar xf -
$ cd luafilesystem-1.2
$ patch -ls -p1 < ../luafilesystem-1.2-lua51.diff
$ make install
It will install lfs.so into /usr/lib/lua/5.1/ which is where lua expects the extensions on my system.
SuSE and Gentoo are known to have their own lfs packages and don't require a compile.
Usertracking
------------
... or how to store data globally in the script-context:
Each script has its own script-context. When the script is started it only contains the lua-functions
and the special lighty.* name-space. If you want to save data between script runs, you can use the global-script
context:
::
if (nil == _G["usertrack"]) then
_G["usertrack"] = {}
end
if (nil == _G["usertrack"][lighty.request["Cookie"]]) then
_G["usertrack"][lighty.request["Cookie"]]
else
_G["usertrack"][lighty.request["Cookie"]] = _G["usertrack"][lighty.request["Cookie"]] + 1
end
print _G["usertrack"][lighty.request["Cookie"]]
The global-context is per script. If you update the script without restarting the server, the context will still be maintained.
Counters
--------
mod_status support a global statistics page and mod_magnet allows to add and update values in the status page:
Config ::
status.statistics-url = "/server-counters"
magnet.attract-raw-url-to = server.docroot + "/counter.lua"
counter.lua ::
lighty.status["core.connections"] = lighty.status["core.connections"] + 1
Result::
core.connections: 7
fastcgi.backend.php-foo.0.connected: 0
fastcgi.backend.php-foo.0.died: 0
fastcgi.backend.php-foo.0.disabled: 0
fastcgi.backend.php-foo.0.load: 0
fastcgi.backend.php-foo.0.overloaded: 0
fastcgi.backend.php-foo.1.connected: 0
fastcgi.backend.php-foo.1.died: 0
fastcgi.backend.php-foo.1.disabled: 0
fastcgi.backend.php-foo.1.load: 0
fastcgi.backend.php-foo.1.overloaded: 0
fastcgi.backend.php-foo.load: 0
Porting mod_cml scripts
-----------------------
mod_cml got replaced by mod_magnet.
A CACHE_HIT in mod_cml::
output_include = { "file1", "file2" }
return CACHE_HIT
becomes::
content = { { filename = "/path/to/file1" }, { filename = "/path/to/file2"} }
return 200
while a CACHE_MISS like (CML) ::
trigger_handler = "/index.php"
return CACHE_MISS
becomes (magnet) ::
lighty.env["request.uri"] = "/index.php"
return lighty.RESTART_REQUEST
}}}

Write Preview
Loading…
Cancel
Save