Commit Graph

131 Commits (d28bac32fef755997b720f6f2cd5962cb1cb57d3)

Author SHA1 Message Date
Glenn Strauss daa5f7c576 [mod_accesslog] attempt to reconstruct req line
cease http_request_parse_reqline() unconditionally copying request line,
as request line is currently used only by mod_accesslog 'r' format
4 years ago
Glenn Strauss 23b0d867c5 [core] replace con->response.keep_alive
set con->keep_alive = 0 to indicate backend request to close connection
4 years ago
Glenn Strauss 25185d1de0 [core] pass req hdrs buffer to http_request_parse 4 years ago
Glenn Strauss aa2d0fb087 [core] srv->max_fds_lowat and srv->max_fds_hiwat 4 years ago
Glenn Strauss ce7b47c015 [core] systemd socket activation support 4 years ago
Glenn Strauss cc1c2f0e37 [core] server.compat-module-load = "disable"
controls whether default and compatibility modules are loaded,
including default modules mod_indexfile, mod_staticfile, mod_dirlisting
and if mod_authn* modules are loaded when mod_auth in server.modules
and if mod_openssl is loaded when ssl. directives are seen

default: server.compat-module-load = "enable"
(preserve existing behavior)

recommended config: explicitly load desired modules in desired order
4 years ago
Glenn Strauss 9725299587 [core] code reuse with http_response_body_clear()
code reuse with http_response_body_clear()
rename con->response.transfer_encoding to con->response.send_chunked
5 years ago
Glenn Strauss 3dd3cde902 [core] abstraction layer for HTTP header manip
http_header.[ch]
convert existing calls to manip request/response headers
convert existing calls to manip environment array (often header-related)
5 years ago
Glenn Strauss d161f53de0 [core] security: use-after-free invalid Range req
(thx Marcus Wengelin)
5 years ago
Glenn Strauss 3eb7902e10 [core] server.http-parseopts URL normalization opt (fixes #1720)
server.http-parseopts = ( ... ) URL normalization options

Note: *not applied* to CONNECT method

Note: In a future release, URL normalization likely enabled by default
  (normalize URL, reject control chars, remove . and .. path segments)
  To prepare for this change, lighttpd.conf configurations should
  explicitly select desired behavior by enabling or disabling:
    server.http-parseopts = ( "url-normalize" => "enable", ... )
    server.http-parseopts = ( "url-normalize" => "disable" )

x-ref:
  "lighttpd ... compares URIs to patterns in the (1) url.redirect and (2) url.rewrite configuration settings before performing URL decoding, which might allow remote attackers to bypass intended access restrictions, and obtain sensitive information or possibly modify data."
  https://www.cvedetails.com/cve/CVE-2008-4359/
  "Rewrite/redirect rules and URL encoding"
  https://redmine.lighttpd.net/issues/1720
5 years ago
Glenn Strauss c56b21084e [core] http_kv.[ch] method, status, version str
move method, status, version strings from keyvalue.[ch] to http_kv.[ch]
5 years ago
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
5 years ago
Glenn Strauss cb371557e5 [core] merge redirect/rewrite pattern substitution
merge redirect/rewrite pattern substitution function (share code)
5 years ago
Glenn Strauss e8226c11cb [core] do not reparse request if async cb
do not reparse request if async callback, e.g. for mod_auth
5 years ago
Glenn Strauss 7b2514cdad [core] quiet pedantic cc warning for excess comma 5 years ago
Glenn Strauss 142971a80c [core] consolidate backend network write handlers
network_write.[ch] isolates various write, writev, sendfile wrappers
5 years ago
Glenn Strauss 9287c87dcd [core] cleanup: consolidate FAM code in stat_cache 5 years ago
Glenn Strauss 6e87da7195 [core] cleanup unused ifndef 5 years ago
Glenn Strauss 1367f60626 [core] isolate sock_addr manipulation 5 years ago
Glenn Strauss 9030cfaecf [core] SIGCHLD handle_waitpid hook for modules
centralize most waitpid() handling in core server, with hooks for
modules to be informed of pid and status when a process exits.

This enables faster discovery (and restart) of exited processes,
and also allows for lighttpd to manage backend processes in the
parent (master) process when server.max-worker > 0.
6 years ago
Glenn Strauss 3c8afd194c [core] base_decls.h to quiet compiler warnings
quiet compiler warning for -Wtypedef-redefinition
(redefinition of typedef is a C11 feature)
6 years ago
Glenn Strauss 5248b46c95 [core] sock_addr_from_str_hints reusable name res
[core] sock_addr_from_str_hints() reusable name resolution func
6 years ago
Glenn Strauss d15ddcb6fa [core] server.socket-perms to set perms on unix (fixes #656)
server.socket-perms = "0770" to set perms on unix domain socket
on which lighttpd listens for requests, e.g. $SERVER["socket"] == "..."

x-ref:
  "Feature request: add server config for setting permissions on Unix domain socket"
  https://redmine.lighttpd.net/issues/656
6 years ago
Glenn Strauss e685ef70f1 [core] act as transparent proxy after con Upgrade 6 years ago
Glenn Strauss 036d3d3d66 [mod_proxy] simple host/url mapping in headers (fixes #152)
Provide a simple mechanism for mapping host and urlpath header strings
in proxied request and response well-known headers.  This *is not*
intended as a one-size-fits-all, infinitely extensible, regex rewriting
engine.  Instead, the proxy.header directive aims to provide built-in
functionality in mod_proxy for a few common use cases by performing
simple host matching or urlpath prefix matching, and using the
mapping of the first match.  More complex use cases could possibly be
handled by a custom lighttpd module (which does not currently exist).

Note: the contents of the HTTP request-line and HTTP headers may or
may not be in normalized canonical forms, which may or may not influence
the simple matching performed.  Admins should take care to provide safe
defaults (fail closed) if mapping is expected to occur and blindly
passing non-mapped requests is undesirable.

proxy.header = (
    #"map-host-request" => (
        #"-" => "...",#replace provided given Host request authority
        #"..." => "-",#preserve existing authority (no further matching)
        #"..." => "", #preserve existing authority (no further matching)
        #             #(equivalent to "xxx" => "xxx")
        #"xxx" => "yyy", #map one string ("xxx") to another ("yyy")
    #),
    #"map-host-response" => (
        #"-" => "...",#replace authority used in backend request
        #"..." => "-",#replace with original authority
        #"..." => "", #preserve existing authority (no further matching)
        #             #(equivalent to "xxx" => "xxx")
        #"xxx" => "yyy", #map one string ("xxx") to another ("yyy")
    #),
    #"map-urlpath" => (
        #"/xxx"  => "/yyy",#map one urlpath prefix to another
        #"/xxx/" => "/",   #map one urlpath prefix to another
        #"/xxx"  => "",    #map one urlpath prefix to another
        #"/key"  => "/value",
        # Note: request headers have matching "key" prefix replaced with
        # "value", and response headers have matching "value" prefix
        # replaced with "key", with a pre-test of the "value" from the
        # first-matched "key" in request headers (if there was a match)
    #),
    #"https-remap" => "enable",
        # For https requests from client, map https:// to http://
        # when map-host-request matches URI in request, and map http://
        # to https:// when map-host-response matches URI in response.
        # (mod_proxy currently sends all backend requests as http)
  )

x-ref:
  "feature to remove part of the URI when passing along requests..."
  https://redmine.lighttpd.net/issues/152
6 years ago
Glenn Strauss ba679290f9 [core] inet_ntop_cache now 4-element cache
(round-robin replacement; fixes what was effectively a 1-element cache)
6 years ago
Glenn Strauss e33ec75999 [core] save connection-level proto in con->proto
The per-request scheme starts with con->proto (e.g. "http")
and can later be changed per-request by mod_extforward or mod_magnet
6 years ago
Glenn Strauss e116479731 [core] make stat_cache private to stat_cache.c 6 years ago
Glenn Strauss ba953cdf45 [core] include "fdevent.h" where needed
(instead of providing #include "fdevent.h" in base.h)
6 years ago
Glenn Strauss a8561450a5 [core] server.syslog-facility (fixes #2800)
server.syslog-facility = "daemon"

x-ref:
  "lighttpd needs config option for syslog facility"
  https://redmine.lighttpd.net/issues/2800
6 years ago
Glenn Strauss 76ad82434f [mod_proxy] remove use of con->got_response 6 years ago
Glenn Strauss 0ec41d6d80 [core] remove unused member con->in_joblist 6 years ago
Glenn Strauss 8f651a2b30 [core] config opt to intercept dynamic handler err (fixes #974)
new directive server.error-intercept = [ "enable" | "disable" ]
to intercept 4xx and 5xx responses from dynamic handlers
(e.g. CGI, FastCGI, SCGI, proxy)

Intercepted HTTP error status are then handled by one of
  server.error-handler
  server.error-handler-404
  server.errorfile-prefix
(if configured)

Do not use server.error-intercept with locations handled by mod_webdav!

x-ref:
  "would like something similar to nginx proxy_intercept_errors"
  https://redmine.lighttpd.net/issues/974
6 years ago
Glenn Strauss 3209f30d11 [core] handle if backend sends Transfer-Encoding (#2786)
It is still not a good idea for backend to send Transfer-Encoding unless
backend is mod_proxy, and mod_proxy should not currently receive chunked
response since mod_proxy sends HTTP/1.0 request.

If mod_proxy is changed to sent HTTP/1.1 request, then lighttpd would
need to check if client is HTTP/1.0 and would need to de-chunk and
remove any other transfer-codings if not supported by next-hop.

x-ref:
  "error 500 (mod_cgi.c.601) cgi died"
  https://redmine.lighttpd.net/issues/2786
6 years ago
Glenn Strauss acc37c1cbc [mod_openssl] use TLS SNI to set host-based certs
and then allow HTTP Host header to set con->uri.authority

remove con->tlsext_server_name
6 years ago
Glenn Strauss 8960633dc7 [mod_openssl] move openssl config into mod_openssl
move openssl data structures and config parsing into mod_openssl
6 years ago
Glenn Strauss cb7ed13621 [core] new plugin hooks to help isolate SSL
handle_request_env        (called on demand by handlers to populate env)
handle_connection_accept
handle_connection_shut_wr (was handle_connection_close)
handle_connection_close   (now occurs at socket close())
6 years ago
Glenn Strauss 2bc94dee82 [core] con interface for read/write; isolate SSL 6 years ago
Glenn Strauss b03c496298 [TLS] ssl.read-ahead = "disable" for low mem (fixes #2778)
new directive ssl.read-ahead = "enable"/"disable" to control
SSL_CTX_set_read_ahead().  Default "enable".  The "disable" setting
is intended for use on low memory systems with a slow CPU which is
unable to keep up with decryption of large request bodies.

x-ref:
  "larger memory usage for file uploads via SSL on embedded system"
  https://redmine.lighttpd.net/issues/2778
6 years ago
Glenn Strauss 4d7f5737f1 [core] support Transfer-Encoding: chunked req body (fixes #2156)
support Transfer-Encoding: chunked request body in conjunction with
  server.stream-request-body = 0

dynamic handlers will still return 411 Length Required if
  server.stream-request-body = 1 or 2 (!= 0)
  since CGI-like env requires CONTENT_LENGTH be set
  (and mod_proxy currently sends HTTP/1.0 requests to backends,
   and Content-Length recommended for robust interaction with backend)

x-ref:
  "request: support Chunked Transfer Coding for HTTP PUT"
  https://redmine.lighttpd.net/issues/2156
6 years ago
Glenn Strauss 544ccee5e1 [core] remove srv->entropy[]
unlikely to provide any real additional benefit as long as
PRNG has been appropriately initialized with random data
6 years ago
Glenn Strauss 1f3ad401ba [mod_deflate] skip deflate if loadavg too high (fixes #1505)
[mod_deflate] skip deflate if 1 min loadavg too high
deflate.max-loadavg  = "3.50"  # express value as string of float num

[mod_compress] skip compression if 1 min loadavg too high
compress.max-loadavg = "3.50"  # express value as string of float num

Feature available on BSD-like systems which have getloadavg() in libc

Note: load average calculations are different on different operating
systems and different types of system loads, so there is no value that
can be recommended for one-size-fits-all.

x-ref:
  "Enable mod_compress to abandon compression when load average is too high"
  https://redmine.lighttpd.net/issues/1505
7 years ago
Glenn Strauss 7f4e156e5f [core] rand.[ch] to use better RNGs when available
prefer RAND_pseudo_bytes() (openssl), arc4random() or jrand48(),
if available, over rand()

These are not necessarily cryptographically secure, but should be better
than rand()
7 years ago
Glenn Strauss 1018ff9922 [core] server.max-request-field-size (fixes #2130)
limits total size per request of request headers submitted by client

default limit set to 8k (prior lighttpd <= 1.4.41 hard-coded 64k limit)

(similar to Apache directive LimitRequestFieldSize)

x-ref:
  "limits the size of HTTP request header"
  https://redmine.lighttpd.net/issues/2130
7 years ago
Glenn Strauss 6ec66c4dce [core] better DragonFlyBSD support (fixes #2746)
(thx xenu)

x-ref:
  "[PATCH] better DragonFlyBSD support; fix crash"
  https://redmine.lighttpd.net/issues/2746
7 years ago
Glenn Strauss 5e76b284df [mod_accesslog] %a %A %C %D %k %{}t %{}T (fixes #1145, fixes #1415, fixes #2081)
add support for additional commonly-used accesslog format flags

x-ref:
  "mod_accesslog cookie field support %{VARNAME}C"
  https://redmine.lighttpd.net/issues/1145
  "access_log : %D time used in ms (not supported)"
  https://redmine.lighttpd.net/issues/1415
  "%{format}t support"
  https://redmine.lighttpd.net/issues/2081
7 years ago
Glenn Strauss fe02be7e34 [core] make server.max-request-size scopeable (fixes #1901)
x-ref:
  "make server.max-request-size scopeable"
  https://redmine.lighttpd.net/issues/1901
7 years ago
Glenn Strauss 695c8f4e07 [config] config options to stream request/response (#949, #376)
This allows admin to configure if response is collected in entirety
prior to sending data to client

For compatibility with existing configs, default is existing behavior:
  buffer entire response prior to sending data to client

The following are config options, though not all implemented yet

// default: buffer entire request body before connecting to backend
server.stream-request-body = 0

// stream request body to backend; buffer to temp files
server.stream-request-body = 1

// stream request body to backend; minimal buffering might block upload
server.stream-request-body = 2

// default: buffer entire response body before sending to client
server.stream-request-body = 0

// stream response body to client; buffer to temp files
server.stream-request-body = 1

// stream response body to client; minimal buffering might block backend
server.stream-request-body = 2

x-ref:
  "fastcgi, cgi, flush, php5 problem."
  https://redmine.lighttpd.net/issues/949
 "Reimplement upload (POST) handling to match apache/zeus/thttpd/boa functionality"
  https://redmine.lighttpd.net/issues/376
7 years ago
Glenn Strauss 4eeeb8fc76 [config] server.bsd-accept-filter option
BSD accept() filters

server.bsd-accept-filter = ""           (default)
server.bsd-accept-filter = "httpready"
server.bsd-accept-filter = "dataready"

Note: this is a behavior change from prior versions.
The default is now no additional accept() filter, whereas prior
versions unconditionally enabled "httpready" accept() filter

Additionally, server.defer-accept (Linux) is inherited from global scope
into $SERVER["socket"] blocks

github: closes #65
7 years ago
Glenn Strauss b47494d4cd [config] opts for http header parsing strictness (fixes #551, fixes #1086, fixes #1184, fixes #2143, #2258, #2281, fixes #946, fixes #1330, fixes #602, #1016)
server.http-parseopt-header-strict  = "enable"
server.http-parseopt-host-strict    = "enable"  (implies host-normalize)
server.http-parseopt-host-normalize = "disable"

defaults retain current behavior, which is strict header parsing
and strict host parsing, with enhancement to normalize IPv4 address
and port number strings.

For lighttpd tests, these need to be enabled (and are by default)
For marginally faster HTTP header parsing for benchmarks, disable these.

To allow
  - underscores in hostname
  - hypen ('-') at beginning of hostname
  - all-numeric TLDs
  server.http-parseopt-host-strict    = "disable"

x-ref:
  "lighttpd doesn't allow underscores in host names"
  https://redmine.lighttpd.net/issues/551
  "hyphen in hostname"
  https://redmine.lighttpd.net/issues/1086
  "a numeric tld"
  https://redmine.lighttpd.net/issues/1184
  "Numeric tld's"
  https://redmine.lighttpd.net/issues/2143
  "Bad Request"
  https://redmine.lighttpd.net/issues/2258
  "400 Bad Request when using Numeric TLDs"
  https://redmine.lighttpd.net/issues/2281

To allow a variety of numerical formats to be converted to IP addresses
  server.http-parseopt-host-strict    = "disable"
  server.http-parseopt-host-normalize = "enable"

x-ref:
  "URL encoding leads to "400 - Bad Request""
  https://redmine.lighttpd.net/issues/946
  "400 Bad Request when using IP's numeric value ("ip2long()")"
  https://redmine.lighttpd.net/issues/1330

To allow most 8-bit and 7-bit chars in headers
  server.http-parseopt-header-strict  = "disable"  (not recommended)

x-ref:
  "Russian letters not alowed?"
  https://redmine.lighttpd.net/issues/602
  "header Content-Disposition with russian '?' (CP1251, ascii code 255) causes error"
  https://redmine.lighttpd.net/issues/1016
7 years ago