This commit is a large set of code changes and results in removal of
hundreds, perhaps thousands, of CPU instructions, a portion of which
are on hot code paths.
Most (buffer *) used by lighttpd are not NULL, especially since buffers
were inlined into numerous larger structs such as request_st and chunk.
In the small number of instances where that is not the case, a NULL
check is often performed earlier in a function where that buffer is
later used with a buffer_* func. In the handful of cases that remained,
a NULL check was added, e.g. with r->http_host and r->conf.server_tag.
- check for empty strings at config time and set value to NULL if blank
string will be ignored at runtime; at runtime, simple pointer check
for NULL can be used to check for a value that has been set and is not
blank ("")
- use buffer_is_blank() instead of buffer_string_is_empty(),
and use buffer_is_unset() instead of buffer_is_empty(),
where buffer is known not to be NULL so that NULL check can be skipped
- use buffer_clen() instead of buffer_string_length() when buffer is
known not to be NULL (to avoid NULL check at runtime)
- use buffer_truncate() instead of buffer_string_set_length() to
truncate string, and use buffer_extend() to extend
Examples where buffer known not to be NULL:
- cpv->v.b from config_plugin_values_init is not NULL if T_CONFIG_BOOL
(though we might set it to NULL if buffer_is_blank(cpv->v.b))
- address of buffer is arg (&foo)
(compiler optimizer detects this in most, but not all, cases)
- buffer is checked for NULL earlier in func
- buffer is accessed in same scope without a NULL check (e.g. b->ptr)
internal behavior change:
callers must not pass a NULL buffer to some funcs.
- buffer_init_buffer() requires non-null args
- buffer_copy_buffer() requires non-null args
- buffer_append_string_buffer() requires non-null args
- buffer_string_space() requires non-null arg
create API in chunk.[ch] for writing a chunk to an fd
(pull similar code from mod_cgi and mod_webdav)
This new API is intended for use on request body input, which is
written to size-limited temporary files controlled by lighttpd and
written to files or pipes.
(network_backend_write() is for writing chunkqueues to sockets)
(thx flynn)
fix large memory usage for large file downloads from dynamic backends
reuse or release large memory chunks
x-ref:
"Memory Growth with PUT and full buffered streams"
https://redmine.lighttpd.net/issues/3033
(replace existing check which suffered from ToC-ToU race condition)
enhances logic from 2015 commit 593599f1 and avoids repeated fstat()
checks when sending large files
For mmap(), lighttpd catches SIGBUS if file is (externally) truncated
and lighttpd attempts to access bytes in a read-only mapping more than
a memory page boundary following the end of the file.
For sendfile(), lighttpd returns an error if sendfile() reports no error
and that no bytes have been sent after lighttpd attempts to send a
non-zero number of bytes.
defer optimization to read small files into memory until after
response_start hooks have a chance to run, e.g. until after
mod_deflate chooses whether or not to serve file from compressed
cache, if deflate.cache-dir is configured
convert all log_error_write() to log_error() and pass (log_error_st *)
use con->errh in preference to srv->errh (even though currently same)
avoid passing (server *) when previously used only for logging (errh)
code reuse, simplification, and inlining
remove excess calls to chunkqueue_remove_finished_chunks()
(it may still be possible for there to be an empty chunk in chunkqueue
if nothing were written to a temporary file (need to verify this), so
preserve some calls to chunkqueue_remove_finished_chunks() for now)
reduce initial buffer size if large POST to backend stored in temp files
regression in lighttpd 1.4.52
(thx rgenoud)
x-ref:
"[regression] lighttpd gets killed after uploading a big file"
https://redmine.lighttpd.net/issues/2922
quickly clear buffer instead of buffer_string_set_length(b, 0) or
buffer_reset(b). Avoids free() of large buffers about to be reused,
or buffers that are module-scoped, persistent, and reused.
(buffer_reset() should still be used with buffers in connection *con
when the data in the buffers is supplied by external, untrusted source)
