large code move, but minimal changes made to code (besides whitespace),
so that code builds
next: need to isolate openssl data structures and config parsing
handle_request_env (called on demand by handlers to populate env)
handle_connection_accept
handle_connection_shut_wr (was handle_connection_close)
handle_connection_close (now occurs at socket close())
support Transfer-Encoding: chunked request body in conjunction with
server.stream-request-body = 0
dynamic handlers will still return 411 Length Required if
server.stream-request-body = 1 or 2 (!= 0)
since CGI-like env requires CONTENT_LENGTH be set
(and mod_proxy currently sends HTTP/1.0 requests to backends,
and Content-Length recommended for robust interaction with backend)
x-ref:
"request: support Chunked Transfer Coding for HTTP PUT"
https://redmine.lighttpd.net/issues/2156
modules may now keep state for the lifetime of a connection,
rather than being required to be reset after every request (when
there can be multiple keep-alive requests on the same connection)
limits total size per request of request headers submitted by client
default limit set to 8k (prior lighttpd <= 1.4.41 hard-coded 64k limit)
(similar to Apache directive LimitRequestFieldSize)
x-ref:
"limits the size of HTTP request header"
https://redmine.lighttpd.net/issues/2130
lots of fixes and improvements
limitations: see comments at top of mod_deflate.c
missing functionality: encode streaming response
(module currently requires response be collected before being sent)
potential functionality: addition of compressed file cache would
allow mod_deflate to fully supplant mod_compress in lighttpd 1.4.x
x-ref:
"Adding mod_deflate to 1.4.xx"
https://redmine.lighttpd.net/issues/1824
"mod_deflate backport compile error if ENABLE_MMAP not defined"
https://redmine.lighttpd.net/issues/2753
github: closes #67
do not enter handler a second time in connection_state_machine()
after read body completes if dynamic handler is still waiting for event
and CON_STATE_READ_POST transitioned to CON_STATE_HANDLE_REQUEST
Check if client half-closed TCP connection if POLLHUP is received.
This more robustly handles if client called shutdown(fd, SHUT_WR).
This patch reverts commit:ab05eb7c which should now be handled properly.
(Time will tell.)
x-ref:
"1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
https://redmine.lighttpd.net/issues/2743
If server.upload-dirs is not configured, then attempt to use TMPDIR
from the environment, if set, or else use /tmp. Warn at startup if
tempdirs are not present.
reverts part of commit:dbdab5db which swapped REQUEST_URI, REDIRECT_URI
x-ref:
"mediawiki redirect loop if REQUEST_URI not orig req in 1.4.40"
https://redmine.lighttpd.net/issues/2738
Explanation:
REQUEST_URI and REDIRECT_URI are not part of CGI standard environment.
The reason for their existence is that PATH_INFO in CGI environment may
be different from the path in the current request. The main reason for
this potential difference is that the URI path is normalized to a path
in the filesystem and tested against the filesystem to determine which
part is SCRIPT_NAME and which part is PATH_INFO. In case-insensitive
filesystems, the URI might be lowercased before testing against the
filesystem, leading to loss of case-sensitive submission in any
resulting PATH_INFO. Also, duplicated slashes "///" and directory
references "/." and "/.." are removed, including prior path component in
the case of "/..". This might be undesirable when the information after
the SCRIPT_NAME is virtual information and there target script needs the
virtual path preserved as-is. In that case, the target script can
re-parse REQUEST_URI (or REDIRECT_URI, as appropriate) to obtain the
unmodified information from the URI.
con->request.uri is equivalent to con->request.orig_uri unless the
request has been internally rewritten (e.g. by mod_rewrite, mod_magnet,
others), in which case con->request.orig_uri is the request made by the
client, and con->request.uri is the current URI being processed.
Historical REQUEST_URI (environment variable) lighttpd inconsistencies
- mod_cml set REQUEST_URI to con->request.orig_uri
- mod_cgi set REQUEST_URI to con->request.orig_uri
- mod_fastcgi set REQUEST_URI to con->request.orig_uri
- mod_scgi set REQUEST_URI to con->request.orig_uri
- mod_ssi set REQUEST_URI to current con->request.uri
- mod_magnet set MAGNET_ENV_REQUEST_URI to current con->request.uri
and MAGNET_ENV_REQUEST_ORIG_URI to con->request.orig_uri
Historical REDIRECT_URI (environment variable) previously set only in
mod_fastcgi and mod_scgi, and set to con->request.uri
Since lighttpd 1.4.40 provides REDIRECT_URI with con->request.orig_uri,
changes were made to REQUEST_URI for consistency, with the hope that
there would be little impact to existing configurations since the
request uri and original request uri are the same unless there has been
an internal redirect. It turns out that various PHP frameworks use
REQUEST_URI and require that it be the original URI requested by client.
Therefore, this change is being reverted, and lighttpd will set
REQUEST_URI to con->request.orig_uri in mod_cgi, mod_fastcgi, mod_scgi
as was done in lighttpd 1.4.39 and earlier. Similarly, REDIRECT_URI
also has the prior behavior in mod_fastcgi and mod_scgi, and added to
mod_cgi.
A future release of lighttpd might change mod_ssi to be consistent with
the other modules in setting REQUEST_URI to con->request.orig_uri and to
add REDIRECT_URI, when an internal redirect has occurred.
fd leak in mod_dirlisting.c
use after free in error condition in mod_proxy.c
NULL pointer dereference in error message in chunk.c
additional minor code changes made to quiet other coverity warnings
This allows admin to configure if response is collected in entirety
prior to sending data to client
For compatibility with existing configs, default is existing behavior:
buffer entire response prior to sending data to client
The following are config options, though not all implemented yet
// default: buffer entire request body before connecting to backend
server.stream-request-body = 0
// stream request body to backend; buffer to temp files
server.stream-request-body = 1
// stream request body to backend; minimal buffering might block upload
server.stream-request-body = 2
// default: buffer entire response body before sending to client
server.stream-request-body = 0
// stream response body to client; buffer to temp files
server.stream-request-body = 1
// stream response body to client; minimal buffering might block backend
server.stream-request-body = 2
x-ref:
"fastcgi, cgi, flush, php5 problem."
https://redmine.lighttpd.net/issues/949
"Reimplement upload (POST) handling to match apache/zeus/thttpd/boa functionality"
https://redmine.lighttpd.net/issues/376
(e.g. when called from xinetd)
Note: lighttpd is designed as a high performance, long-running server,
not a one-shot executable. This one-shot mode of operation has not been
tuned for performance. lighttpd server start-up and initialization aims
for correctness, not speed. If using this one-shot mode as part of fork
and exec from xinetd, then performance is already not of high concern.
x-ref:
"support for xinetd"
https://redmine.lighttpd.net/issues/1584
server.error-handler preserves HTTP status error code when error page
is static, and allows dynamic handlers to change HTTP status code
when error page is provided by dynamic handler. server.error-handler
intercepts all HTTP status codes >= 400 except when the content is
generated by a dynamic handler (cgi, ssi, fastcgi, scgi, proxy, lua).
The request method is unconditionally changed to GET for the request
to service the error handler, and the original request method is
later restored (for logging purposes). request body from the
original request, if present, is discarded.
server.error-handler is somewhat similar to server.error-handler-404,
but server.error-handler-404 is now deprecated, intercepts only 404
and 403 HTTP status codes, and returns 200 OK for static error pages,
a source of confusion for some admins. On the other hand, the new
server.error-handler, when set, will intercept all HTTP status error
codes >= 400. server.error-handler takes precedence over
server.error-handler-404 when both are set.
NOTE: a major difference between server.error-handler and the
now-deprecated server.error-handler-404 is that the values of the
non-standard CGI environment variables REQUEST_URI and REDIRECT_URI
have been swapped. Since REDIRECT_STATUS is the original HTTP
status code, REDIRECT_URI is now the original request, and REQUEST_URI
is the current request (e.g. the URI/URL to the error handler).
The prior behavior -- which reversed REQUEST_URI and REDIRECT_URI values
from those described above -- is preserved for server.error-handler-404.
Additionally, REDIRECT_STATUS is now available to mod_magnet, which
continues to have access to request.uri and request.orig_uri.
See further discussion at https://redmine.lighttpd.net/issues/2702
and https://redmine.lighttpd.net/issues/1828
github: closes #36
set REDIRECT_STATUS to con->error_handler_saved_status in dynamic
handlers for PHP compiled with --force-redirect. Set to "200"
if (0 == con->error_handler_saved_status)
(mod_cgi, mod_fastcgi, mod_scgi, mod_ssi)
FYI: setting REDIRECT_STATUS in con->environment allows access and
manipulation by mod_magnet.
x-ref:
"REDIRECT_STATUS == 200 on 404 redirect"
https://redmine.lighttpd.net/issues/1828
github: closes #35
to detect client disconnect. Do so even when waiting on backend,
and not polling for POLLRD or POLLWR on client connection.
This reduces unnecessary load on backends when backends are slow
to respond and client has given up waiting.
x-ref:
"https://redmine.lighttpd.net/issues/399"
FastCGI performance on high load
connection_handle_read()
connection_handle_read_ssl()
connection_handle_read_post_state()
no code changes besides making connection_handle_read() public
(by removing 'static' and adding to connections.h)
read request body right before calling subrequest handler,
allowing request to be handled prior to reading request body,
e.g. to send 401 Unauthorized response when authentication is required
(In the future, this might move into each dynamic handler which supports
request body (mod_cgi, mod_fastcgi, mod_proxy, mod_scgi, mod_webdav) so
that each dynamic handler can choose whether or not to buffer request
body or to stream request body to backend as request body is received.)
keep-alive is disabled if request body has not been completely read
prior to sending response
x-ref:
"HTTP 401 Unauthorized only sent back after full POST request is read"
https://redmine.lighttpd.net/issues/2541
http_chunk_append_file() opens fd when appending file to chunkqueue.
Defers calculation of content length until response is finished.
This reduces race conditions pertaining to stat() and then (later)
open(), when the result of the stat() was used for Content-Length
or to generate chunked headers.
Note: this does not change how lighttpd handles files that are modified
in-place by another process after having been opened by lighttpd --
don't do that. This *does* improve handling of files that are
frequently modified via a temporary file and then atomically renamed
into place.
mod_fastcgi has been modified to use http_chunk_append_file_range() with
X-Sendfile2 and will open the target file multiple times if there are
multiple ranges.
Note: (future todo) not implemented for chunk.[ch] interfaces used by
range requests in mod_staticfile or by mod_ssi. Those uses could lead
to too many open fds. For mod_staticfile, limits should be put in place
for max number of ranges accepted by mod_staticfile. For mod_ssi,
limits would need to be placed on the maximum number of includes, and
the primary SSI file split across lots of SSI directives should either
copy the pieces or perhaps chunk.h could be extended to allow for an
open fd to be shared across multiple chunks. Doing either of these
would improve the performance of SSI since they would replace many file
opens on the pieces of the SSI file around the SSI directives.
x-ref:
"Serving a file that is getting updated can cause an empty response or incorrect content-length error"
https://redmine.lighttpd.net/issues/2655
github:
Closes #49
remove Cache-Control and Expires headers before handling error docs
(caching headers may have been set by mod_expire before http status
was determined to be an error)
x-ref:
"mod_expires sends headers on 404 responses"
https://redmine.lighttpd.net/issues/1919
instead of looping over all config blocks for each conditional var that
gets enabled, enable them all and run over them once.
Right now it seems we actually set all variables at once in normal
config handling (SNI only sets a subset); future modifications
might introduce new variables which are activated at a later stage
(physical path related for example).
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@3083 152afb58-edef-0310-8abb-c4023f1b3aa9
- a lot of code tried to handle manually adding terminating zeroes and
keeping track of the correct "used" count.
Replaced all "external" usages with simple wrapper functions:
* buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
prefer buffer_string_is_empty
* buffer_string_set_length
* buffer_string_length
* CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
supresses the warning about unused but defined (static) functions;
don't care whether it actually gets inlined or not.
From: Stefan Bühler <stbuehler@web.de>
git-svn-id: svn://svn.lighttpd.net/lighttpd/branches/lighttpd-1.4.x@2979 152afb58-edef-0310-8abb-c4023f1b3aa9
