parse $HTTP["remote-ip"] CIDR mask into structured data at startup
note: adds buffer_move() to configparser.y to reduce memory copying
for all config values, and is required for remote-ip to preserve the
structured data added after the config value string. (Alternatively,
could have normalized the remote-ip value after copying into dc->string)
This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.
Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.
In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func. In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.
- check for empty strings at config time and set value to NULL if blank
string will be ignored at runtime; at runtime, simple pointer check
for NULL can be used to check for a value that has been set and is not
blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
and use buffer_is_unset() instead of buffer_is_empty(),
where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
truncate string, and use buffer_extend() to extend
Examples where buffer known not to be NULL:
- cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
(though we might set it to NULL if buffer_is_blank(cpv->v.b))
- address of buffer is arg (&foo)
(compiler optimizer detects this in most, but not all, cases)
- buffer is checked for NULL earlier in func
- buffer is accessed in same scope without a NULL check (e.g. b->ptr)
internal behavior change:
callers must not pass a NULL buffer to some funcs.
- buffer_init_buffer() requires non-null args
- buffer_copy_buffer() requires non-null args
- buffer_append_string_buffer() requires non-null args
- buffer_string_space() requires non-null arg
link http_auth.c into mod_auth
link http_vhostdb.c into mod_vhostdb
ensure that mod_auth loads before mod_authn_*
ensure that mod_vhostdb loads before mod_vhostdb_*
add option to read config file from stdin using: -f -
(with "-" indicating to read config file from stdin)
This was already been possible using temporary files, unlinked and
passed to lighttpd as -f /proc/self/fd/XX (with XX as fd number)
on systems (or containers) in which /proc/self/fd/ is available
Note: using -f - is incompatible with one-shot mode (-1)
since one-shot mode expects stdin to be connected to the client
enabled by default
disable using server.feature-flags += ("server.pcre_jit" => "disable")
Available since pcre-8.20 (2011), and improved in pcre-8.32 (2012),
PCRE_STUDY_JIT_COMPILE can greatly speed up repeated execution of PCRE
patterns. (https://zherczeg.github.io/sljit/pcre.html)
lighttpd continues to use pcre_exec() instead of pcre_jit_exec(),
even though doing so does not realize all of the performance increase
potentially available with PCRE_STUDY_JIT_COMPILE and pcre_jit_exec().
pcre_jit_exec() is available with PCRE 8.32 and later, if PCRE is
compiled with --enable-jit, but lighttpd does not currently use
pcre_jit_exec() since the PCRE library might not have been compiled
with --enable-jit (though this could be solved with a weak symbol).
Similarly, lighttpd does not currently configure the pcre_jit_stack.
(Using pcre_jit_exec() may be revisited in the future.)
x-ref:
"add support for pcre JIT"
https://redmine.lighttpd.net/issues/2361
(thx altblue)
reorder dc->context_ndx to match srv->config_context->data[] index.
srv->config_context->data[] may have been re-ordered in configparser.y.
Since the dc->context_ndx (id) is reused by config_insert*() and by
plugins to index into srv->config_context->data[], reorder into the
order encountered during config file parsing for least surprise to
end-users writing config files.
x-ref:
"mismatched nested config conditions in lighttpd 1.4.56"
https://redmine.lighttpd.net/issues/3047
more consistent use of shared code config_plugin_value_tobool()
(thx tow-conf)
x-ref:
"The on/off keywords in boolean configuration options is inconsistent, which might be misleading and error-prone."
https://redmine.lighttpd.net/issues/3036
Location response header is permitted to use relative-path in
RFC 7231 Section 7.1.2. Location
Prefer relative path in redirection for the benefit of reverse proxies
and CDNs. Doing so also avoids potentially disclosing internal schemes
and server names which client might not be able to directly reach.
To restore prior behavior of sending a fully-qualified absolute URI:
server.feature-flags += ("absolute-dir-redirect" => "enable")
x-ref:
https://bz.apache.org/bugzilla/show_bug.cgi?id=63357
translate config server.modules "mod_compress" to "mod_deflate"
accept compress.* directives, but issue DEPRECATED warning trace
mod_deflate differences from mod_compress:
- mod_compress compress.filetype was exact match; deflate.mimetypes is
prefix match (behavior change might compress longer mimetype matches,
which are likely of similar type and compressability)
- mod_compress always sent entire (compressed) file for Range request
mod_deflate will stream compress range result (not stored in cache)
- mod_compress would short-circuit request with 403 Forbidden error
if request file did not exist (stat() failed) (This behavior was
unfriendly to other handlers)
- mod_compress compress.cache-dir layout differs from deflate.cache-dir
layout; file cache should be cleared (or renamed) when migrating from
mod_compress to mod_deflate
- mod_deflate does not issue Vary: Accept-Encoding if request does not
contain Accept-Encoding. The identity response can be cache by
proxies and served to clients. Historically, some proxies disabled
caching if any Vary: response was seen. If the Vary header is
desirable, mod_deflate code which checks for Accept-Encoding and
compression type can be moved down a few lines to be below the
setting of the Vary response header.
(expansion of buffer_string_lenth() inline function and CONST_BUF_LEN()
macro, which always check for NULL, appears to cause the analyzer to
believe that a pointer might be NULL in cases where it otherwise can
not be NULL)
x-ref:
http://clang-analyzer.llvm.org/faq.html
Prefer some WolfSSL native APIs when building with WolfSSL.
However, some functionality in WolfSSL is available only through the
WolfSSL compatibility layer for OpenSSL, so the effort to create a
native mod_wolfssl halted here.
(experimental)
mod_gnutls supports most ssl.* config options supported by mod_openssl
x-ref:
"GnuTLS support for the mod_ssl"
https://redmine.lighttpd.net/issues/109
(experimental)
mod_mbedtls supports most ssl.* config options supported by mod_openssl
thx Ward Willats for the initial discussion and attempt in the comments
https://redmine.lighttpd.net/boards/3/topics/7029
NB: r->tmp_buf == srv->tmp_buf (pointer is copied for quicker access)
NB: request read and write chunkqueues currently point to connection
chunkqueues; per-request and per-connection chunkqueues are
not distinct from one another
con->read_queue == r->read_queue
con->write_queue == r->write_queue
NB: in the future, a separate connection config may be needed for
connection-level module hooks. Similarly, might need to have
per-request chunkqueues separate from per-connection chunkqueues.
Should probably also have a request_reset() which is distinct from
connection_reset().
fdevent.c no longer directly uses struct server *srv
srv->srvconf.max_fds (if set) is used to set rlimits
set max_conns in server.c after fdevent_init(), which sets srv->max_fds
using srv->srvconf.max_fds (if set) as input hint
e.g. different server.errorlog for different virtual hosts
Also, support different server.breakagelog to have separate script
error logs, applicable to mod_cgi and mod_ssi exec.
