Commit Graph

282 Commits (96324fbba0e1a98bc2616f5a60d737ebb4677928)

Author SHA1 Message Date
Stefan Bühler 5515bb2a08 scons: fix check environment
if os.environ is a special class the scons.Environment Clone() doesn't
work properly.

show builddir in prepare.sh so we can actually see it works.
3 years ago
Glenn Strauss de9e2f9881 [tests] mod_compress -> mod_deflate 3 years ago
Glenn Strauss 8dd33a72dd [mod_deflate] mod_deflate subsumes mod_compress
translate config server.modules "mod_compress" to "mod_deflate"
accept compress.* directives, but issue DEPRECATED warning trace

mod_deflate differences from mod_compress:
- mod_compress compress.filetype was exact match; deflate.mimetypes is
  prefix match (behavior change might compress longer mimetype matches,
  which are likely of similar type and compressability)
- mod_compress always sent entire (compressed) file for Range request
  mod_deflate will stream compress range result (not stored in cache)
- mod_compress would short-circuit request with 403 Forbidden error
  if request file did not exist (stat() failed) (This behavior was
  unfriendly to other handlers)
- mod_compress compress.cache-dir layout differs from deflate.cache-dir
  layout; file cache should be cleared (or renamed) when migrating from
  mod_compress to mod_deflate
- mod_deflate does not issue Vary: Accept-Encoding if request does not
  contain Accept-Encoding.  The identity response can be cache by
  proxies and served to clients.  Historically, some proxies disabled
  caching if any Vary: response was seen.  If the Vary header is
  desirable, mod_deflate code which checks for Accept-Encoding and
  compression type can be moved down a few lines to be below the
  setting of the Vary response header.
3 years ago
Glenn Strauss c75fd15d7a [build] remove tests/mod-userdir.t from builds
(replaced with src/t/test_mod_userdir.c)
3 years ago
Glenn Strauss c18f442a63 [multiple] add summaries to top of some modules 3 years ago
Glenn Strauss af836b94d3 [tests] fix skip count in mod-fastcgi w/o php-cgi
(fix test skip count when php-cgi is not available)
3 years ago
Glenn Strauss c8a1cba0c1 [tests] t/test_mod_userdir
create t/test_mod_userdir to replace sparse tests in tests/mod-userdir.t
remove tests/mod-userdir.t
3 years ago
Glenn Strauss 362ccedc41 [mod_indexfile] use config_plugin_values_init() 3 years ago
Glenn Strauss fa4ab19275 [core] reduce use of struct parse_header_state 3 years ago
Glenn Strauss 61f85d14ee [core] reject WS following header field-name (fixes #2985)
reject whitespace following request header field-name and before colon
Such whitespace is forbidden in RFC 7230 Section 3.2.4.

strict header parsing is enabled by default in lighttpd.  However,
if explicitly disabled in lighttpd.conf, lighttpd will continue to
accept (and re-format) such field-names before passing to any backend.
  UNSAFE: server.http-parseopts = ( "header-strict" => "disable" )
  This is NOT RECOMMENDED since doing so disables other protections
  provided by lighttpd strict http header parsing.

(thx fedormixalich)

x-ref:
  stricter request header parsing
  https://redmine.lighttpd.net/issues/2985
3 years ago
Glenn Strauss d5774fc6b3 [tests] skip mod-secdownload HMAC-SHA1,HMAC-SHA256
skip mod-secdownload.t HMAC-SHA1, HMAC-SHA256 tests if crypto algorithms
are not available (e.g. lighttpd build without openssl)
4 years ago
Glenn Strauss da6a94efbb [tests] has_feature() helper func
has_feature() helper func so that tests
can be skipped if support is not present
4 years ago
Glenn Strauss b0ae5fcbcd [core] support weak etags with If-None-Match
support weak etags with If-None-Match

(but require strong etags for Range requests)
4 years ago
Glenn Strauss 1cf68f79eb [core] behavior change: stricter URL normalization
behavior change: stricter URL normalization

Prior behavior can be obtained by configuring lighttpd.conf with:
      server.http-parseopts = (“url-normalize” => “disable” )
although this is not recommended.

This behavior change was pre-announced with the releases of
  lighttpd 1.4.52 (2018.11.28)
  lighttpd 1.4.53 (2019.01.27)

The recommended settings are:
      server.http-parseopts = (
        "header-strict"            => "enable",
        "host-strict"              => "enable",
        "host-normalize"           => "enable",
        "url-normalize"            => "enable",
        "url-normalize-unreserved" => "enable",
        "url-normalize-required"   => "enable",
        "url-ctrls-reject"         => "enable",
        "url-path-2f-decode"       => "enable",
        "url-path-backslash-trans" => "enable",
        "url-path-dotseg-remove"   => "enable",
        "url-query-20-plus"        => "enable"
      )

The lighttpd defaults with this commit are slightly less strict:
      server.http-parseopts = (
        "header-strict"            => "enable",
        "host-strict"              => "enable",
        "host-normalize"           => "enable",
        "url-normalize"            => "enable",
        "url-normalize-unreserved" => "enable",
       #"url-normalize-required"   => "enable",
        "url-ctrls-reject"         => "enable",
        "url-path-2f-decode"       => "enable",
       #"url-path-backslash-trans" => "enable",
        "url-path-dotseg-remove"   => "enable",
       #"url-query-20-plus"        => "enable"
      )
4 years ago
Glenn Strauss a1077d18cb [tests] more test config cleanup 5 years ago
Glenn Strauss 4e6ef76406 [tests] update skip count in mod-fastcgi.t 5 years ago
Glenn Strauss 07517ff30a [tests] some test config cleanup
including limiting use of php in tests to mod-fastcgi.t
5 years ago
Glenn Strauss f03e5e239d [tests] t/test_keyvalue
create t/test_keyvalue to replace sparse tests in
  tests/mod-redirect.t and tests/mod-rewrite.t
remove tests/mod-redirect.t and tests/mod-rewrite.t
5 years ago
Glenn Strauss 5a32d0f72a [mod_access] t/test_mod_access
create t/test_mod_access to test mod_access basic logic
remove tests/mod-access.t
5 years ago
Glenn Strauss 8aad091613 [mod_evhost] t/test_mod_evhost
create t/test_mod_evhost to test mod_evhost basic logic
remove tests/mod-evhost.t
5 years ago
Glenn Strauss b2a6239851 [mod_simple_vhost] t/test_mod_simple_vhost
create t/test_mod_simple_vhost to test mod_simple_vhost basic logic
remove tests/mod-simplevhost.t, which was not testing mod_simple_vhost
5 years ago
Glenn Strauss 90c30d5e90 [core] http_status_append() 5 years ago
Glenn Strauss d161f53de0 [core] security: use-after-free invalid Range req
(thx Marcus Wengelin)
5 years ago
Glenn Strauss 6ccccaaa38 [tests] move src/test_*.c to src/t/ 5 years ago
Glenn Strauss 1b62dc325c [tests] test_request unit tests
unit tests for request processing
collect existing request processing tests from Perl tests/*.t
(test_request.c runs *much* more quickly than Perl tests/*.t)
5 years ago
Glenn Strauss 1593190651 [core] buffer_append_string_encoded() uc hex
Use uc hex chars in buffer_append_string_encoded(), preferred in RFC3986

Preserve behavior using lc hex chars in buffer_append_string_c_escaped()
5 years ago
Glenn Strauss dc1675ea32 [core] fix POST with chunked request body (fixes #2854)
(thx the_jk)

x-ref:
  "chunked transfer encoding in request body only works for tiny chunks"
  https://redmine.lighttpd.net/issues/2854
5 years ago
Stefan Bühler 16c4530e61 [meson] new build system
Needed to extend lemon to take an output path parameter.
6 years ago
Glenn Strauss 371e1bf723 [mod_extforward] support Forwarded HTTP Extension (#2703)
enable with, e.g.:
extforward.headers = ( "Forwarded" )
or
extforward.headers = ( "Forwarded", "X-Forwarded-For" )
or
extforward.headers = ( "Forwarded", "X-Forwarded-For", "Forwarded-For" )

The default remains:
extforward.headers = ( "X-Forwarded-For", "Forwarded-For" )

Support for "Forwarded" is not enabled by default since intermediate
proxies might not be aware of Forwarded, and might therefore pass
spoofed Forwarded header received from client.

extforward.params = ( # overwrite "Host" with Forwarded value
                      #"host" => 1
                      # set REMOTE_USER with Forwarded value
                      #"remote_user" => 1
                    )
Note: be cautious configuring trusted proxies if enabling these options
since Forwarded header may be spoofed and passed along indescriminantly
by proxies which do not handle Forwarded.

To remove "Forwarded" from incoming requests, do not enable these
options and instead use mod_setenv to clear the request header:
  setenv.set-request-header = ( "Forwarded" => "" )

Other proxy-related headers which admin might evaluate to keep or clear:
  setenv.set-request-header = ( "X-Forwarded-For" => "",
                                "X-Forwarded-By" => "",
                                "X-Forwarded-Server" => "",
                                "X-Origin-IP" => "",
                                "Via" => "",
                                #...
                              )

x-ref:
  "Forwarded HTTP Extension"
  https://tools.ietf.org/html/rfc7239
  "Forward authenticated user to proxied requests"
  https://redmine.lighttpd.net/issues/2703
6 years ago
Glenn Strauss aa14493e47 [tests] reduce time waiting for backends to start
reduce time spent waiting for backends to start

tests check for active listening port before proceeding

test runs now complete in about 2/3 the time
6 years ago
Glenn Strauss 0a635fc8be [core] consolidate dynamic handler response parse
- consolidate dynamic handler HTTP response parsing code
- reduce string copies for CGI, FastCGI, SCGI, proxy response headers
- let read() signal EOF or EAGAIN instead of ioctl FIONREAD 0-data-ready
6 years ago
Glenn Strauss 57ab20ace5 [mod_cgi] cgi.local-redir = [enable|disable] (#2108, #2793)
new directive cgi.local-redir = [enable|disable]

*disable* RFC3875 6.2.2 local-redir by default.
(behavior change from when local-redir support added in lighttpd 1.4.40)

The reason for this behavior change is that CGI local-redir support
(RFC3875 6.2.2) is an optimization.  Absence of support may result in
additional latency in servicing a request due the additional round-trip
to the client, but that was the prior behavior (before lighttpd 1.4.40)
and is the behavior of web servers which do not support CGI local-redir.

However, enabling CGI local-redir by default may result in broken links
in the case where a user config (unaware of CGI local-redir behavior)
returns HTML pages containing *relative* paths (not root-relative paths)
which are relative to the location of the local-redir target document,
and the local-redir target document is located at a different URL-path
from the original CGI request.

x-ref:
  RFC3875 CGI 1.1 specification section 6.2.2 Local Redirect Response
  http://www.ietf.org/rfc/rfc3875
  "CGI local redirect not implemented correctly"
  https://redmine.lighttpd.net/issues/2108
  "1.4.40 regression: broken redirect (using Location) between url.rewrite-once URLs"
  https://redmine.lighttpd.net/issues/2793
6 years ago
Glenn Strauss ab077790f2 [tests] correct skip count for mod-scgi.t 6 years ago
Glenn Strauss 739ccb5de0 [tests] remove unused file depending on CGI.pm
lighttpd tests do not depend on CGI.pm.

remove *unused* file tests/docroot/www/404.fcgi
which used CGI::Fast, which depends on CGI.pm.
6 years ago
Glenn Strauss 4a68780e1b [mod_scgi] tests/mod-scgi.t unit tests
(copied from tests/mod-fastcgi.t fcgi-responder tests)
6 years ago
Glenn Strauss 37dac9a23c [core] support Expect: 100-continue with HTTP/1.1 (fixes #377, #1017, #1953, #2438)
support Expect: 100-continue with HTTP/1.1 requests

Ignore config option server.reject-expect-100-with-417;
server.reject-expect-100-with-417 will be removed in a future release.

x-ref:
  "Incorrect handling of the 100 (Continue) Status"
  https://redmine.lighttpd.net/issues/377
  "'Expect' header gives HTTP error 417"
  https://redmine.lighttpd.net/issues/1017
  "Improve DAV support to be able to handle git as a client"
  https://redmine.lighttpd.net/issues/1953
  "Change server.reject-expect-100-with-417 from flag to regular expression matching the URL"
  https://redmine.lighttpd.net/issues/2438
6 years ago
Glenn Strauss afce434e0b [mod_secdownload] new directives modify hash path (fixes #646, fixes #1904)
secdownload.path-segments = <number>
  include only given number of path segments in hash digest calculation

secdownload.hash-querystr = "enable" | "disable"
  include the query string in the hash digest calculation

x-ref:
  "secdownload.path_elements support"
  https://redmine.lighttpd.net/issues/646
  "mod_secdownload option to include url GET parameters in md5"
  https://redmine.lighttpd.net/issues/1904
6 years ago
Glenn Strauss 4d92366ab2 [mod_setenv] directives to overwrite/remove hdrs (fixes #650, fixes #2295)
directives to set value, rather than append values to headers, env
  setenv.set-request-header
  setenv.set-response-header
  setenv.set-environment
These directives take precedence over the setenv.add-* counterparts

Set a blank value for request or response header to remove the header
(blank value in environment will be set as the value; not removed)

setenv.*-environment is now deferred to handle_request_env hook.
setenv.*-response-header is now processed in handle_response_start hook.

x-ref:
  "setenv.add-or-replace-response-header"
  https://redmine.lighttpd.net/issues/650
  "set-request-header or remove-request-header support for mod_setenv"
  https://redmine.lighttpd.net/issues/2295
6 years ago
Glenn Strauss 83b2b71527 [tests] give time for periodic jobs to detect exit
give time for periodic jobs to detect backend exit
6 years ago
Glenn Strauss 16f171588f [tests] FCGI_Finish() final request before exit 6 years ago
Glenn Strauss 32443ea1c7 [tests] update test skip count for !fcgi-responder 6 years ago
Glenn Strauss 64df38aad8 [mod_fastcgi] detect child exit, restart proactively
(instead of detecting upon a subsequent HTTP request)

(for backends spawned by mod_fastcgi)
6 years ago
Glenn Strauss f57d8c54b4 [mod_cgi] skip local-redir handling if to self (fixes #2779, #2108)
Loosen local redirect handling in mod_cgi to skip handling as local
redirect if the Location matches con->uri.path, since if the request
is intended to redirect back to the same CGI using the same request
method, path info, and query string, the CGI would logically just
return the final intended response.  Loosening this handling avoids a
problem with applications (potentially) accessible through multiple
gateways, where the application is not aware of this specific handling
of Location in the Common Gateway Interface (CGI/1.1), the application
sends abs-path in the Location response header instead of absoluteURI,
and the application expects the client to receive this Location response
header instead of the server to process as a CGI local redirect.

One example of such an application is LuCI,
which sends Set-Cookie with Location: /abs-path
  https://github.com/openwrt/luci

(Note that this loose check for matching con->uri.path is not perfect
 and might not match if the CGI returned a path with a different case
 and the server is on a case-insensitive filesystem, or if the path
 returned by the CGI is rewritten elsewhere to a different con->uri.path
 before getting to mod_cgi.)

RFC3875 CGI 1.1 specification section 6.2.2 Local Redirect Response
http://www.ietf.org/rfc/rfc3875

x-ref:
  "CGI local-redir handling conflicts with LuCI redirect w/ Set-Cookie"
  https://redmine.lighttpd.net/issues/2779
  "CGI local redirect not implemented correctly"
  https://redmine.lighttpd.net/issues/2108
6 years ago
Glenn Strauss 15bc2313a4 [tests] mark tests/docroot/www/*.pl scripts a+x 7 years ago
Glenn Strauss 0f4cc0d697 [mod_evhost] fix an incorrect error trace 7 years ago
Glenn Strauss 4d7f5737f1 [core] support Transfer-Encoding: chunked req body (fixes #2156)
support Transfer-Encoding: chunked request body in conjunction with
  server.stream-request-body = 0

dynamic handlers will still return 411 Length Required if
  server.stream-request-body = 1 or 2 (!= 0)
  since CGI-like env requires CONTENT_LENGTH be set
  (and mod_proxy currently sends HTTP/1.0 requests to backends,
   and Content-Length recommended for robust interaction with backend)

x-ref:
  "request: support Chunked Transfer Coding for HTTP PUT"
  https://redmine.lighttpd.net/issues/2156
7 years ago
Glenn Strauss 7d339e21db comment out auth.backend.ldap.* in tests/*.conf
(mod_authn_ldap is not loaded in these test confs,
 so mod_authn_ldap directives are not available)
7 years ago
Glenn Strauss f664e77909 load mod_auth & mod_authn_file in sample/test.conf 7 years ago
Glenn Strauss 75040e9988 [mod_evhost] mod-evhost.t tests (#1194)
(thx Daniel-Brandt)

x-ref:
  "Partial matching in mod_evhost patterns"
  https://redmine.lighttpd.net/issues/1194
7 years ago
Glenn Strauss b11d059843 [cmake] build fcgi-auth, fcgi-responder for tests
Aside: must have cmake enable building openssl for tests to pass
due to tests/lighttpd.conf including config options requiring openssl
algorithms in mod_secdownload.c:
  (secdownload.algorithm       = "hmac-sha1")
  (secdownload.algorithm       = "hmac-sha256")

$ cmake -L .
$ cmake -DWITH_OPENSSL:BOOL=ON .
$ make -j 4 -k
$ make test

x-ref:
  https://blog.lighttpd.net/articles/2006/12/25/1-5-0-goes-cmake/
7 years ago