Commit Graph

4158 Commits (243510dbb4d79a3866c288a7d6530f6015c5b537)
 

Author SHA1 Message Date
Glenn Strauss 243510dbb4 [core] fdlog.[ch]; fdevent_*_logger_* -> fdlog_*
code move and rename fdevent_*_logger_*() to fdlog_*()
2 years ago
Glenn Strauss 4763e08c07 [core] update comment about server.max-write-idle 2 years ago
Glenn Strauss e97a5b7e49 [core] clear buffer after backend dechunk
(thx flynn)

clear buffer after backend dechunk if not sending chunked to client

x-ref:
  "Memory fragmentation with HTTP/2 enabled"
  https://redmine.lighttpd.net/issues/3084
2 years ago
Glenn Strauss 9a37ba54d1 [tests] skip time-sensitive test during CI testing 2 years ago
Glenn Strauss f99cb7d7ab [core] quiet coverity warnings 2 years ago
Glenn Strauss f9c817680d [build] define rsize_t on FreeBSD
(unsure why lighttpd CI CMake build has rsize_t defined,
 but not autotools or SCons lighttpd CI builds)
2 years ago
Glenn Strauss a3e9faa479 [multiple] quiet coverity warnings 2 years ago
Glenn Strauss f364c8ef36 [core] remove redundant waitpid() on each backend
remove redundant waitpid() on each backend
(performed each second during periodic maintenance)

Doing so is redundant with handle_waitpid hook since lighttpd 1.4.46
2 years ago
Glenn Strauss e77bc3995c [core] reduce wait time in gw_spawn_connection()
reduce wait time in gw_spawn_connection() from 10ms down to 1ms.
That is still plenty of time to detect immediate execve()/clone()
failures.
2 years ago
Glenn Strauss 90978eb808 [core] mark cold more gateway maintenance code 2 years ago
Glenn Strauss 9c7ef4f064 [core] consolidate more gw_host, gw_proc init code
clearer code: host->num_procs as num gw_proc in host->first linked list
2 years ago
Glenn Strauss 33ffec35ad [mod_cgi] cgi.limits "tcp-fin-propagate" => "SIG"
cgi.limits += ("tcp-fin-propagate" => "<signal>")  (e.g. "SIGTERM")

send specified signal to CGI if TCP FIN is received from client
(default: do not send signal to CGI)
2 years ago
Glenn Strauss ee9352b1bb [core] realloc buffer power-2 size + 1 for '\0'
realloc buffer power-2 size + 1 for '\0' to avoid power-2 allocation
doubling buffer size for sole reason of storing '\0' at end of block
2 years ago
Glenn Strauss 715699ba50 [core] write all cq MEM_CHUNK if spill to tempfile
If chunkqueue size grows large enough in memory to use tempfiles,
write all MEM_CHUNK in chunkqueue to tempfiles to free up memory.

If earlier chunks in chunkqueue are MEM_CHUNK, then a prior attempt
to write was not able to proceed, or items are being added to
chunkqueue as they are decoded from TLS.  In either case, choose to
free up memory sooner.  This may result in some cases where lighttpd
soon reads data back from disk to send to the backend, but since some
(now all) of the data is in tempfiles, sendfile() will avoid reading
back into userspace, though sendfile() will not be used if lighttpd
needs to read data back into memory to encrypt data using TLS.
2 years ago
Glenn Strauss 6800b0821c [core] reduce stack use in main()
mark noinline for setup and other cold funcs
2 years ago
Glenn Strauss ec88f62b5a [core] change backtrace format to put addr first
(better monospaced alignment of frame num, addr, offset, name)
2 years ago
Glenn Strauss 84ff9ac9c9 [core] expose ck_bt() for debugging 2 years ago
Glenn Strauss bd72d1927a [core] remove server.upload-temp-file-size limit
previously undocumented server.upload-temp-file-size in lighttpd 1.4.38
preceded introduction of lighttpd streaming options in lighttpd 1.4.40
(server.stream-request-body and server.stream-response-body)
2 years ago
Glenn Strauss 33808a2e73 [core] TCP_CORK w/ MEM_CHUNK then FILE_CHUNK
TCP_CORK with MEM_CHUNK followed by FILE_CHUNK,
or multiple MEM_CHUNK with TLS
2 years ago
Glenn Strauss ad8a27f356 [core] cfg server.bindhost after $SERVER["socket"]
init global config for server.bindhost and server.port after
initializing $SERVER["socket"] so that if bindhost and port matches
another $SERVER["socket"], the $SERVER["socket"] config is used,
as the $SERVER["socket"] config inherits from the global scope and
can the be overridden.

x-ref:
  "Activate SSL with lighttpd on a Raspberry Pi"
  https://stackoverflow.com/questions/68939760/activate-ssl-with-lighttpd-on-a-raspberry-pi
2 years ago
Glenn Strauss cd738d4daa [build] propagate HAVE_DLFCN_H in builds 2 years ago
Glenn Strauss 1a8d8e0ca5 [core] periodically malloc_trim() to release mem (fixes #3084)
x-ref:
  "Memory fragmentation with HTTP/2 enabled"
  https://redmine.lighttpd.net/issues/3084
2 years ago
Glenn Strauss 43f0106f81 [core] config mallopt(M_ARENA_MAX, 2) (#3084)
Preemptively cap the max number of arenas that might be created by glibc

Each thread attempts to use the previously-used arena.
lighttpd is single-threaded, so in general, only one arena is used.

x-ref:
  "Memory fragmentation with HTTP/2 enabled"
  https://redmine.lighttpd.net/issues/3084
2 years ago
Glenn Strauss 1783550154 [build] look for malloc.h and mallopt() 2 years ago
Glenn Strauss 18ec4f419c [tests] move tests/docroot/www contents up 1 level
reduce directories under tests
(because automake is sloooow and the fewer dirs, the better)
2 years ago
Glenn Strauss 3f0833328c [mod_indexfile] update path with buffer path funcs
use buffer_copy_path_len2() or buffer_append_path_len()
to construct r->physical.path filesystem path
2 years ago
Glenn Strauss 83338cb7dd [tests] IO::Socket::INET->new( Timeout => 1 )
(to be used when supported by the underlying Perl lib)
2 years ago
Glenn Strauss ca2898f678 [tests] t/test_mod_indexfile
also remove some now-redundant tests from request.t
and reduce scripts and directories under tests
(because automake is sloooow and the fewer dirs, the better)
2 years ago
Glenn Strauss 7a21b3856e [core] log_error_multiline()
rename log_error_multiline_buffer() to log_error_multiline()
and take (char *)ptr and (size_t)len instead of (buffer *)b

When debug printing request and response headers,
print each header on separate line for readability
and omit '\r' if "\r\n" ends line
2 years ago
Glenn Strauss 41d4ffad99 [mod_openssl] default disable client renegotiation 2 years ago
Glenn Strauss 70195d069d [mod_cgi] improve CGI offloading
improve CGI offloading when not streaming request body

If not streaming request body, collect request body into single tempfile
and use fd to tempfile as stdin to CGI (in lieu of input pipe()).
For prior behavior, set the following config option, enabled by default:
  server.feature-flags += ("cgi.tempfile-accum" => "disable")

If there is no request body, open /dev/null instead of creating pipe().
2 years ago
Glenn Strauss bed08f1292 [core] base64url pad char is '='; change from '.'
RFC4648 base64url pad char is '='; change from '.' in lighttpd
The base64url encoding was introduced in lighttpd 1.4.38 in 2015,
but at the time, does not appear to have been used in base64url
decoding where padding might be present.

In lighttpd 1.4.50, base64url decoding was possible with %{decb64u:...}
potentially used in mod_redirect and mod_rewrite rules.  However, this
is not believed to be widely used, and even if used, it is strongly
recommended that the URLs contain a base64url-encoded string WITHOUT
padding, since padding of '=' might be %-encoded and require decoding
before base64url-decoding.

Note: this change may affect %{decb64u:...} substitution in
mod_redirect and mod_rewrite, but *ONLY* in case where URL contains
a base64url-encoded string *WITH* padding, and that padding had been
created specifically for use with lighttpd, using '.' as padding char

x-ref:
  RFC4648 https://datatracker.ietf.org/doc/html/rfc4648#section-5
2 years ago
Glenn Strauss 517010f2a6 [mod_echo] test module to echo request as response
test/debugging module to echo request back to client as response

(not part of build Makefile)
2 years ago
Glenn Strauss 6ed2777204 [mod_magnet] ignore 1xx return in response start
ignore 1xx return code from lua in response start phase.

Since response is about to start, send any added/modified headers
along with final response.  (If we did not ignore, then 1xx return
code from lua would incorrectly overwrite the final response status.)
2 years ago
Glenn Strauss 3a8b965438 [mod_magnet] detect MAGNET_RESTART_REQUEST loops
throw error after 9 MAGNET_RESTART_REQUEST loops
2 years ago
Glenn Strauss e7290fad9a [mod_magnet] attrs for remote port and server port
lighty.r.req_attr["request.remote-port"]
lighty.r.req_attr["request.server-port"]
2 years ago
Glenn Strauss f7018e3d39 [core] sock_addr_set_port() 2 years ago
Glenn Strauss d9bac3f423 [mod_magnet] reset config cache for remote addr
reset config cache for changes to
  lighty.r.req_attr["request.remote-addr"]

Changes might affect subsequent modules when those modules patch config,
but does not affect the core config options which have already been
processed.

If lighttpd config needs to be reprocessed after making changes, then
return lighty.RESTART_REQUEST from lua script to tell lighttpd to
restart the request, including reprocessing the core config options.
2 years ago
Glenn Strauss 2d8619db57 [mod_magnet] reset config cache for uri components
reset config cache after modifications to uri components
  lighty.r.req_attr["uri.scheme"]
  lighty.r.req_attr["uri.authority"]
  lighty.r.req_attr["uri.path"]
  lighty.r.req_attr["uri.query"]

Changes might affect subsequent modules when those modules patch config,
but does not affect the core config options which have already been
processed.

If lighttpd config needs to be reprocessed after making changes, then
return lighty.RESTART_REQUEST from lua script to tell lighttpd to
restart the request, including reprocessing the core config options.
2 years ago
Glenn Strauss 0a5cbc62bd [mod_magnet] rename magnet_cgi_ to magnet_envvar_
rename magnet_cgi_* to magnet_envvar_* to be less confusing
The (array *)r->env is populated with connection-level env (e.g TLS)
but is not populated with CGI/1.1 environment variables.
Values similar to most CGI/1.1 environment variables are already available
from lighty.r.req_attr[] (or lighty.env[]) which are prepared on demand.
2 years ago
Glenn Strauss c0ea7f8e27 [mod_magnet] use blank str for nil (do not panic)
substitute blank string for nil param (instead of lua script panic)
2 years ago
Glenn Strauss 3ddf457560 [mod_magnet] lighty.c.readdir
lighty.c.readdir           dir walk

Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
2 years ago
Glenn Strauss ed94ae88e8 [mod_magnet] lighty.c.cookie_tokens
lighty.c.cookie_tokens     parse cookie header into table

Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
2 years ago
Glenn Strauss c82ca970fb [core] refuse excess h2 streams at con start (fixes #3093)
refuse excess h2 streams up to stream id 199 within 1-2 seconds of
establishing connection.  This attempts to more gracefully handle
excessively agressive browsers sending too many new requests *before*
processing the initial SETTINGS frame sent by the server.

lighttpd HTTP/2 support limits SETTINGS_MAX_CONCURRENT_STREAMS to 8
and communicates this in the initial SETTINGS sent by the server.

x-ref:
  "Chrome 92, HTTP/2, fcgi, mutiple puts no response"
  https://redmine.lighttpd.net/issues/3093
  "POST request DATA part for non-existing URI closes HTTP/2 connection prematurely"
  https://redmine.lighttpd.net/issues/3078
2 years ago
Glenn Strauss aa1027ba24 [mod_magnet] lighty.r.* interfaces to request
lighty.r.req_header[]     HTTP request headers
lighty.r.req_attr[]       HTTP request attributes / components
lighty.r.req_env[]        HTTP request environment variables
lighty.r.resp_header[]    HTTP response headers
lighty.r.resp_body.*      HTTP response body accessors
lighty.r.resp_body.len    HTTP response body length
lighty.r.resp_body.add()  HTTP response body add (string or table)
lighty.r.resp_body.set()  HTTP response body set (string or table)

lighty.r.req_header[] allows get/set of request headers
If modifications would affect config processing, script should return
lighty.MAGNET_RESTART to have lighttpd restart the modified request
This differs from the older lighty.env[] table, which (previously) did
not permit modification of request headers.

lighty.r.resp_header[] allows get/set of response headers
(certain connection-level headers such as Connection and
Transfer-Encoding are restricted from modification)
This differs from the older lighty.header[] table, which is
collected and deferred, being applied after the script exits.

lighry.r.resp_body.* adds/sets response body content.
This differs from the older lighty.content[] table, which is
collected and deferred, being applied after the script exits.

e.g.
  r = lighty.r
  resp_header = r.resp_header
  resp_header["Content-Type"] = "text/html"
or
  lighty.r.resp_header["Content-Type"] = "text/html"
older syntax (less clear)
  lighty.header["Content-Type"] = "text/html"
2 years ago
Glenn Strauss 86c87d3c6d [core] use mod name in trace instead of mod_gw 2 years ago
Glenn Strauss a3b76ed5c4 [core] reuse some cold duplicate hdr match code 2 years ago
Glenn Strauss b596680ba4 [mod_magnet] look up env id by strlen, then strcmp 2 years ago
Glenn Strauss 614250b4d8 [mod_magnet] expose enc/dec str funcs to lua
lighty.c.b64urldec         validate and decode base64url
lighty.c.b64urlenc         base64url encode, no padding
lighty.c.b64dec            validate and decode base64
lighty.c.b64enc            base64 encode, no padding
lighty.c.hexdec            validate and decode hex str
lighty.c.hexenc            uc; lc w/ lua s = s:lower()
lighty.c.xmlenc            xml-encode/html-encode: <>&'\"`
lighty.c.urldec            url-decode
lighty.c.urlenc            url-encode
lighty.c.urldec_query      url-decode query-string
lighty.c.urlenc_query      url-encode query-string
lighty.c.urlenc_normalize  url-encode normalization
lighty.c.fspath_simplify   simplify fspath

Note: the "lighty.c.*" namespace is EXPERIMENTAL / UNSTABLE
In the future, these may be removed, altered, or moved to a different
namespace.
2 years ago
Glenn Strauss 76ed1f4462 [mod_magnet] protect and control lighty table mod
protect lighty table against mistaken modification
(lighty table is resused between script invocations)

detect if lighty.content[] is set by script (or if not set)
2 years ago