Commit Graph

1845 Commits

Author SHA1 Message Date
Glenn Strauss d7638b9b10 fix some warnings reported by static analysis tool
iterate over environ via array-index notation with char **ptr on stack
(instead of repeatedly re-accessing global 'environ')

check getsockname() return values including addrlen

[mod_dirlisting] pass buf size into http_list_directory_sizefmt()

github: resolves #48
2016-04-24 20:28:03 -04:00
Glenn Strauss 49c74fff65 [core] compile with upcoming openssl 1.1.0 release (fixes #2727)
(thx falemagn)

  "Won't compile with OpenSSL 1.1.0"
2016-04-24 20:28:03 -04:00
Glenn Strauss 4db255ca39 make (compile and link) cleanly under cygwin 2016-04-24 20:24:28 -04:00
Glenn Strauss 3888a1419e [doc] add ref to RFC 7232 for conditional requests 2016-04-24 20:24:28 -04:00
Glenn Strauss d0dc881d73 [doc] enhance error msg for backend server config
enhance error message for backend server config file parse
(fastcgi.server, scgi.server, proxy.server)

2016-04-24 20:24:28 -04:00
Glenn Strauss 8de5f41577 [doc] add mimetype.use-xattr to conf.d/mime.conf
  "Migrate to definition of xattr mimetype"
2016-04-24 20:24:28 -04:00
Glenn Strauss eb0cbb00ff improve dynamic handler control flow logic
Merge branch 'feature-dynamic-handler-control-flow'
2016-04-24 20:08:03 -04:00
Glenn Strauss 7c0f8a775d [mod_cgi] asynchronous send of request body to CGI
Like other dynamic handler in prior commit,
mod_cgi can read response before sending req body

mod_cgi no longer blocks sending request body to CGI
2016-04-18 06:19:04 -04:00
Glenn Strauss 2f21aaa973 handlers can read response before sending req body (fixes #131, #2566)
dynamic handlers mod_fastcgi, mod_scgi, and mod_proxy can now read
response from backend prior to finishing sending request body.

If the backend closes the connections (or shuts down socket write
end so that lighttpd read() 0 to indicate EOF), then lighttpd will
abort attempting to send request body to backend.

  "mod_fastcgi should handle "quick" responses"
2016-04-18 06:19:04 -04:00
Glenn Strauss 7f942ce09a remove handle_joblist hook
remove handle_joblist hook and remove the hooks defined in
mod_fastcgi and mod_scgi.  The calls made to fdevent management
are redundant.  If the calls were actually needed, then
mod_proxy would have needed a handle_joblist handler, too.
2016-04-18 06:19:03 -04:00
Glenn Strauss 4b0c822ed0 always poll for client POLLHUP/POLLERR events (fixes #399)
to detect client disconnect.  Do so even when waiting on backend,
and not polling for POLLRD or POLLWR on client connection.

This reduces unnecessary load on backends when backends are slow
to respond and client has given up waiting.

  FastCGI performance on high load
2016-04-18 06:19:03 -04:00
Glenn Strauss 8f27ff8cd4 defer reading request body until handle subrequest
read request body in dynamic handlers supporting request body
(mod_cgi, mod_fastcgi, mod_proxy, mod_scgi, mod_webdav)

(In the future, each dynamic handler might choose whether or not to
 buffer request body or to stream request body to backend as request
 body is received.)

modify mod_webdav to mark request in handle_physical hook, and move
the main logic to handle_subrequest hook, where the main logic is
for other dynamic handlers.
2016-04-18 06:19:03 -04:00
Glenn Strauss 635ab6f802 mv funcs from connections.c to connections-glue.c

no code changes besides making connection_handle_read() public
(by removing 'static' and adding to connections.h)
2016-04-18 06:19:03 -04:00
Glenn Strauss c263bc6a11 defer reading request body until handle subrequest (fixes #2541)
read request body right before calling subrequest handler,
allowing request to be handled prior to reading request body,
e.g. to send 401 Unauthorized response when authentication is required

(In the future, this might move into each dynamic handler which supports
 request body (mod_cgi, mod_fastcgi, mod_proxy, mod_scgi, mod_webdav) so
 that each dynamic handler can choose whether or not to buffer request
 body or to stream request body to backend as request body is received.)

keep-alive is disabled if request body has not been completely read
prior to sending response

  "HTTP 401 Unauthorized only sent back after full POST request is read"
2016-04-18 06:19:03 -04:00
Glenn Strauss bbbbfb3de0 untangle overly complex control flow logic
for dynamic handlers mod_cgi, mod_fastcgi, mod_scgi, mod_proxy
(mod_cgi control flow logic simplification began in a prior commit)

- connection state machine calls the subrequest handler
- subrequest handler sets up the connection to the backend
  and registers fdevent handler to handle backend events
- fdevent handler handles backend events and then schedules
  a call to connection state machine
- when retrying an alternate backend, backend state is reset
  and then response state is reset so that the connection state
  machine will call back into the subrequest handler to retry

Dynamic handlers no longer directly modify connection state
(calls to connection_set_state() from dynamic handlers were removed)

Dynamic handlers no longer reset con->physical.path, and they
preserve con->mode when retrying alternate backends.  This is done
to skip repeated processing in response.c:http_response_prepare()

While this patch increases consistency in control flow handling,
there is more work to be done that can further improve upon this.

  "handle-req time too long"
2016-04-18 06:19:03 -04:00
Glenn Strauss 77bd45121c [core] retry tempdirs on partial write, ENOSPC (fixes #2588)
Previous code would fail on partial write, EINTR, and ENOSPC.
Upon any of the above errors, this patch tries next tempdir in list,
if list of tempdirs provided by config option server.upload-dirs

  "Problem when uploading large files"

Closes #54
2016-04-18 04:30:55 -04:00
Glenn Strauss 71ed1912c7 [config] server.listen-backlog option (fixes #1825, #2116)
See doc/config/lighttpd.conf for explanation of listen() backlog queue

Additionally, mod_fastcgi and mod_scgi backend servers can now also be
configured with separate listen-backlog settings per server

  "add server.listen-backlog option instead of hard-coded value (128 * 8) for listen()"
  "Don't disable backend when overloaded"

Closes #50
2016-04-18 04:29:28 -04:00
Glenn Strauss a65c57a548 [core] open fd when appending file to cq (fixes #2655)
http_chunk_append_file() opens fd when appending file to chunkqueue.
Defers calculation of content length until response is finished.

This reduces race conditions pertaining to stat() and then (later)
open(), when the result of the stat() was used for Content-Length
or to generate chunked headers.

Note: this does not change how lighttpd handles files that are modified
in-place by another process after having been opened by lighttpd --
don't do that.  This *does* improve handling of files that are
frequently modified via a temporary file and then atomically renamed
into place.

mod_fastcgi has been modified to use http_chunk_append_file_range() with
X-Sendfile2 and will open the target file multiple times if there are
multiple ranges.

Note: (future todo) not implemented for chunk.[ch] interfaces used by
range requests in mod_staticfile or by mod_ssi.  Those uses could lead
to too many open fds.  For mod_staticfile, limits should be put in place
for max number of ranges accepted by mod_staticfile.  For mod_ssi,
limits would need to be placed on the maximum number of includes, and
the primary SSI file split across lots of SSI directives should either
copy the pieces or perhaps chunk.h could be extended to allow for an
open fd to be shared across multiple chunks.  Doing either of these
would improve the performance of SSI since they would replace many file
opens on the pieces of the SSI file around the SSI directives.

  "Serving a file that is getting updated can cause an empty response or incorrect content-length error"

Closes #49
2016-04-18 04:27:08 -04:00
Glenn Strauss e0bafab2db [mod_indexfile] save physical path to env (fixes #448, #892)
save physical path to environment (PATH_TRANSLATED_DIRINDEX)
when index generator is full path (not relative path)

fix uri.path when index generator is full path (not relative path)
2016-04-14 16:02:11 -04:00
Glenn Strauss 186e7073fb [mod_redirect,mod-rewrite] short-circuit if blank replacement (fixes #2085)
  "null redirects for mod_redirect"
2016-04-14 16:01:45 -04:00
Glenn Strauss fa3a36ffd4 [mod_ssi] config ssi.exec (fixes #2051)
(thx benbrown)

  "mod_ssi Add configuration item to disable SSI exec."
2016-04-14 08:59:07 -04:00
fbrosson 6982b1930e [mod_ssi] config ssi.conditional-requests
A new SSI directive, "ssi.conditional-requests", allows to inform
lighttpd which SSI pages should be considered as cacheable and which
should not. In particular, the "ETag" & "Last-Modified" headers will
only be sent for those SSI pages for which the directive is enabled.

Long description:
"ETag" and "Last-Modified" headers were being sent for all SSI pages,
regardless of whether they were cacheable or not. And yet, there was
no cache validation at all for any SSI page.
This commit fixes these two minor issues by adding a new directive,
"ssi.conditional-requests", which allows to specify which SSI pages
are cacheable and which are not, and by adding cache validation to
those SSI pages which are cacheable. And since sending ETags for
non-cacheable documents is not appropriate, they are no longuer
computed nor sent for those SSI pages which are not cacheable.
Regarding the "Last-Modified" header for non-cacheable documents,
the standards allow to either send the current date and time for
that header or to simply skip it. The approach chosen is to not send
it for non-cacheable SSI pages. "ETag" and "Last-Modified" headers
are therefore only sent for an SSI page if ssi.conditional-requests
is enabled for that page.

The ssi.conditional-requests directive can be enabled or disabled
globally and/or in any context. It is disabled by default.

An index.shtml which only includes deterministic SSI commands such as:
<!--#echo var="LAST_MODIFIED"-->
is a trivial example of a dynamic SSI page that is cacheable.
2016-04-14 12:35:10 +00:00
Glenn Strauss bb95317774 [core] setrlimit max-fds <= rlim_max for non-root (fixes #2723)
  "setrlimit can increase RLIMIT_NOFILE up to rlim_max for non-root"
2016-04-13 03:54:06 -04:00
Glenn Strauss 51e0f2087b [core] define __STDC_WANT_LIB_EXT1__ (fixes #2722)
  "define __STDC_WANT_LIB_EXT1__ for memset_s()"
2016-04-13 03:54:04 -04:00
Glenn Strauss c4edd35655 skip spawning backends for preflight tests (#2642)
Also, update man page for lighttpd -tt

  "add option for "fail on warning""
2016-04-13 03:53:23 -04:00
Glenn Strauss c63427c999 [mod_dirlisting] class for dir <tr> (fixes #2304)
  "special class for directories tr's in directory listing"
2016-04-13 02:18:16 -04:00
Glenn Strauss dc9f95c75d [mod_status] table w/ count of con states (fixes #2427)
(replaces "legend")
2016-04-13 02:12:54 -04:00
Glenn Strauss c68f83aed4 [mod_status] page refresh option (fixes #2170)
e.g. http://server-address/server-status?refresh=10

  "server-status page should have an automatic reload"
2016-04-13 02:08:24 -04:00
Glenn Strauss 760baed402 [mod_expire] reset caching response headers for error docs (fixes #1919)
remove Cache-Control and Expires headers before handling error docs
(caching headers may have been set by mod_expire before http status
 was determined to be an error)

  "mod_expires sends headers on 404 responses"
2016-04-13 01:04:39 -04:00
Glenn Strauss f4cb07f723 [mod_webdav] readdir POSIX compat (fixes #1826)
do not expect '.' to be part of dir listing

  "mod_webdav readdir POSIX compatibility issue"
2016-04-13 00:23:31 -04:00
Glenn Strauss e5e5548b88 [mod_extforward] reset cond_cache for scheme (fixes #1499)
bug #1499 was mostly fixed in 05858f6c
This patch additionally resets the cond_cache since scheme might change

  "HTTPS env var should be "on" when using mod_extforward and the X-Forwarded-Proto header is set."
2016-04-13 00:17:46 -04:00
Glenn Strauss 659ce5e78e [mod_magnet] rename var for clarity (fixes #1483)
"length" argument is more accurately described as 0-index end of range

  "magnet offset > length bug"
2016-04-13 00:07:26 -04:00
Glenn Strauss 4b412797b8 [mod_auth] send charset="UTF-8" in WWW-Authenticate (fixes #1468) and (September 2015)
update Digest and Basic auth to allow server to recommend charset
which should be used by client.

  "LDAP UTF-8 encoding"
2016-04-12 23:56:55 -04:00
Glenn Strauss f1681ca29b [mod_cgi] always set QUERY_STRING (fixes #1339)
(thx alexo)

  "lighttpd doesn't set empty QUERY_STRING in cgi environment"
2016-04-12 23:48:14 -04:00
Glenn Strauss 97556d992b [mod_fastcgi,mod_scgi] check for spawning on same unix socket (fixes #319)
error if unix socket path is duplicated

(does not check across modules, but will detect duplicated unix socket
 paths within fastcgi directives, and separately, duplicated unix socket
 paths within scgi directives)
2016-04-08 16:39:29 +02:00
Glenn Strauss de08a135ea [core] clean up srv before exiting for lighttpd -[vVh]
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 17:38:32 +00:00
Glenn Strauss 36ab0587c0 [stream] fstat() after open() to obtain file size
Common case is on non-empty files, and doing fstat() after open()
eliminates ToC-ToU between stat() and then open().  While file size
of the target file might still change, the fstat() is on the opened
file, and can not be on different file (which was possible with stat())

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 17:38:29 +00:00
Glenn Strauss 47f3dbebe4 use li_[iu]tostrn() instead of li_[iu]tostr()
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 17:38:26 +00:00
Glenn Strauss e5006d88eb pass buf size to li_tohex()
also change passing of fixed-sized arrays: need to pass pointer to array
as otherwise size does not get enforced

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 16:54:46 +00:00
Glenn Strauss dac02e257c [mod_status] use snprintf() instead of sprintf()
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 16:54:43 +00:00
Glenn Strauss ab829cee5e [mod_webdav] allow Depth: Infinity lock on file (fixes #2296)
(still not supporting Depth: Infinity on directories)

patch by mstorsjo submitted as part of feature request #1953

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-04-01 16:26:25 +00:00
Stefan Bühler dd33a350bb [configparser] fix small leak on config failure
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 13:57:03 +00:00
Glenn Strauss d17d48e01e [stat] mimetype.xattr-name global config option (fixes #2631)
For backwards compatibility with existing lighttpd configs, default is
  mimetype.xattr-name = "Content-Type"

Those who wish to use the definition of xattr mimetype
can set the following in the global lighttpd config:
  mimetype.xattr-name = "user.mime_type"

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 13:49:43 +00:00
Glenn Strauss 292309f88b [core] lighttpd -tt performs preflight startup checks (fixes #411)
lighttpd -t loads config file and performs syntax check
lighttpd -tt (new) performs preflight startup checks,
  including loading and initializing modules, but skipping any
  potentially destructive actions which might affect an already
  running server (separate instance).  These currently include:
  - skipping pidfile modification
  - skipping bind() to network sockets
  - skipping open of error and access logs

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 13:39:54 +00:00
Jan Kneschke 9ae7813685 [core] fixed the loading for default modules if they are specified explicitly
backported 1836 from trunk

From: Jan Kneschke <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 13:07:48 +00:00
Glenn Strauss 06d3c75440 [core] respond 411 Length Required if request has Transfer-Encoding: chunked (fixes #631)
lighttpd does not currently support request body transfer-codings

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 12:58:33 +00:00
Glenn Strauss f11089ed2b [core] wait for grandchild to be ready when daemonizing (fixes #2712, thx pasdVn)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 12:52:23 +00:00
Glenn Strauss 0aa2ea74e3 [mod_proxy] accept LF delimited headers, not just CRLF (fixes #2594)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 11:32:13 +00:00
Glenn Strauss b4a4afdaf7 [config] warn if server.upload-dirs has non-existent dirs (fixes #2508)
Warn at startup if any dirs in server.upload-dirs do not exist.
Take server.chroot into account, if set.

From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 11:24:15 +00:00
fbrosson a579e7ffc0 [mod_ssi] Add SSI vars SCRIPT_{URI,URL} and REQUEST_SCHEME (fixes #2721)
This is a proposal to add to lighttpd the famous SSI variables
SCRIPT_URI and SCRIPT_URL (known to Apache users), as well as a bonus
ENV variable called REQUEST_SCHEME.

SCRIPT_URI and SCRIPT_URL will be available as SSI variables from
within documents handled by mod_ssi.
They can be used like any other SSI var with the "#echo var" command:
<!--#echo var="SCRIPT_URI"-->
<!--#echo var="SCRIPT_URL"-->
Webmasters willing to display links to the W3C Validator will be able
to use:
<a href="<!--#echo var="SCRIPT_URI"-->">…</a>
instead of the generic link
which does not work on some (most?) browsers which do not send
referers when the link itself resides in a document sent through

REQUEST_SCHEME will be available both as an environment variable. It
is defined as "http" or "https", depending on the scheme of the
connection. It is safe to use this name as it does not conflict with
any existing variable on Apache or Nginx. This is slightly different
from the HTTPS var which is often added by webadmins on their server's
configuration. EDIT: Some Apache modules also define REQUEST_SCHEME
with the same possible values as this proposal.

From: fbrosson <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-26 11:14:21 +00:00