Commit Graph

4063 Commits (87b09d144ecbb5c9eea1bf604b9ec0d5a61875d0)
 

Author SHA1 Message Date
Glenn Strauss 87b09d144e [core] use CLOCK_MONOTONIC_COARSE where available
server.c uses monotonic clock to detect change in second,
so CLOCK_MONOTONIC_COARSE clock resolution (often ~1ms)
is more than sufficient.  (Obtaining CLOCK_MONOTONIC_COARSE
can be faster than CLOCK_MONOTONIC.)

reference:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_for_real_time/7/html/reference_guide/sect-posix_clocks
2 years ago
Glenn Strauss dbf7588147 [core] tune http_response_process_headers()
- rearrange some code for better CPU cache use
- use http_header_str_contains_token()
2 years ago
Glenn Strauss f4ff56e064 [mod_deflate] use http_header_str_contains_token 2 years ago
Glenn Strauss 309c1693ac [multiple] Y2038 32-bit signed time_t mitigations
Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
  $ lighttpd -V
    + Y2038 support                                    # Y2038-SAFE
  $ lighttpd -V
    - Y2038 support (unsafe 32-bit signed time_t)      # Y2038-UNSAFE

* Y2038: general platform info
  * Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
      - all major 64-bit platforms (known to this author) use 64-bit time_t
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
      - Linux x32 ABI (different from i686)
      - FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
      - NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
      - OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
      - Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
        Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
      - MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
      - e.g. OpenVMS (unknown if lighttpd builds on this platform)
  * Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
      - Linux 32-bit (including i686)
          - glibc 32-bit library support not yet available for 64-bit time_t
              - https://sourceware.org/glibc/wiki/Y2038ProofnessDesign
              - Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
                https://itsubuntu.com/linux-kernel-5-6-to-fix-the-year-2038-issue-unix-y2k/
              - https://www.gnu.org/software/libc/manual/html_node/64_002dbit-time-symbol-handling.html
                "Note: at this point, 64-bit time support in dual-time
                 configurations is work-in-progress, so for these
                 configurations, the public API only makes the 32-bit time
                 support available. In a later change, the public API will
                 allow user code to choose the time size for a given
                 compilation unit."
              - compiling with -D_TIME_BITS=64 currently has no effect
          - glibc recent (Jul 2021) mailing list discussion
              - https://public-inbox.org/bug-gnulib/878s2ozq70.fsf@oldenburg.str.redhat.com/T/
      - FreeBSD i386
      - DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
  * lighttpd prefers system monotonic clock instead of realtime clock
    in places where realtime clock is not required
  * lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
  * (lighttpd presumes that lighttpd will not encounter dates before 1970
    during normal operation.)
  * lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
    to convert negative timestamps for comparisions with 64-bit timestamps
    (treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
  * lighttpd provides unix_time64_t (int64_t) and
  * lighttpd provides struct unix_timespec64 (unix_timespec64_t)
    (struct timespec equivalent using unix_time64_t tv_sec member)
  * lighttpd provides gmtime64_r() and localtime64_r() wrappers
    for platforms 32-bit platforms using 32-bit time_t and
    lighttpd temporarily shifts the year in order to use
    gmtime_r() and localtime_r() (or gmtime() and localtime())
    from standard libraries, before readjusting year and passing
    struct tm to formatting functions such as strftime()
  * lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
    unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
  on 32-bit platforms using 32-bit signed time_t, underlying libraries and
  underlying filesystems might not behave properly after 32-bit signed time_t
  overflows (19 Jan 2038 03:14:08 GMT).  If a given 32-bit OS does not work
  properly using negative time_t values, then lighttpd likely will not work
  properly on that system.

* Other references and blogs
  - https://en.wikipedia.org/wiki/Year_2038_problem
  - https://en.wikipedia.org/wiki/Time_formatting_and_storage_bugs
  - http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/
2 years ago
Glenn Strauss 82c199db3f [autoconf] add AC_SYS_LARGEFILE for lfs
add AC_SYS_LARGEFILE for large file support
(in addition to manually defining macros for large file support
 on different platforms; bootstrap-and-suspenders)
2 years ago
Glenn Strauss 3ce554e18c [core] report Y2038 support in lighttpd -V
(report unsafe 32-bit signed time_t)

64-bit time_t (default with 64-bit builds) is safe beyond Y2038
2 years ago
Glenn Strauss 9c9cd37320 [tests] speed up mod-fastcgi and mod-scgi tests
On platforms where lighttpd gets SIGCHLD,
lighttpd can quickly detect exit of backend process
and can restart backend process
2 years ago
Glenn Strauss 5294d7e18f [tests] combine *.t using tests/condition.conf
combine tests/*.t using tests/condition.conf into tests/core-condition.t

Platforms which are horrifically slow starting processes (e.g. Windows)
take much more time to start and stop lighttpd many times for
independent *.t instances run through the Perl Test::More framework
2 years ago
Glenn Strauss f472611597 [tests] combine *.t using tests/lighttpd.conf
combine tests/*.t using tests/lighttpd.conf into tests/request.t

Platforms which are horrifically slow starting processes (e.g. Windows)
take much more time to start and stop lighttpd many times for
independent *.t instances run through the Perl Test::More framework
2 years ago
Glenn Strauss 91472ab768 [tests] t/test_mod_staticfile
move some tests from tests/request.t to src/t/test_mod_staticfile.c
2 years ago
Glenn Strauss a0a8cf821d [tests] mv tests from request.t to test_request.c
move some tests from tests/*.t to src/t/test_request.c
2 years ago
Glenn Strauss d4f43f7e1b [tests] update t/test_request.c
use http_header_request_get() in tests to retrieve request headers
2 years ago
Glenn Strauss f7bebe99a5 [tests] use generated date in HTTP If conditionals
replace hard-coded date in HTTP If conditional tests

x-ref:
  "Make tests pass after 2035"
  https://github.com/lighttpd/lighttpd1.4/pull/106

github: closes #106
2 years ago
Glenn Strauss 361b96476e [core] replace strncasecmp w/ buffer_eq_icase_ssn
replace strncasecmp() w/ buffer_clen() and buffer_eq_icase_ssn()
(portability; remove use of alt sys-strings.h portability header)
2 years ago
Glenn Strauss 75e9b2857a [core] better trace if TLS received on clear port 2 years ago
Glenn Strauss 6881f79c32 [mod_auth] detect and skip BWS (bad whitespace)
detect and skip BWS (bad whitespace) in Authorization
2 years ago
Glenn Strauss 7a913fb153 [mod_deflate] improve compress.*->deflate.* remap 2 years ago
Glenn Strauss a3603074c1 [mod_deflate] use brotli quality 5 by default
BROTLI_DEFAULT_QUALITY is 11, which may lead to a higher compression
ratio, but potentially at a cost of taking *many* multiples of the
time taken to compress at quality level 5.

x-ref:
  https://almanac.httparchive.org/en/2020/compression#fig-3
  https://blog.cloudflare.com/results-experimenting-brotli/
  https://blogs.akamai.com/2016/02/understanding-brotlis-potential.html
  https://expeditedsecurity.com/blog/nginx-brotli/
2 years ago
Glenn Strauss 3f248f0900 [mod_deflate] deflate.params per-encoder params
mechanism to define per-encoder params
parsed into structured data at startup

compression level is the initial target
deflate.params is a better solution to the deflate.compression-level,
which is a single range 1-9 that is overload and applied to all encoders
without any scaling, even though encoders might have different scales.

x-ref:
  "ModDeflate questions (possibly some feature requests too)"
  https://redmine.lighttpd.net/boards/2/topics/9786
2 years ago
Glenn Strauss 777d659c5c [mod_deflate] use ZSTD_c_strategy w/ compress lvl
map deflate.compression-level to ZSTD_c_strategy as the current
enum for ZSTD_strategy is in the range 1 .. 9, which matches the
gzip compression level 1 .. 9 range

x-ref:
  "ModDeflate questions (possibly some feature requests too)"
  https://redmine.lighttpd.net/boards/2/topics/9786
2 years ago
Glenn Strauss 7759660de5 [mod_deflate] use deflate.allowed-encodings order
use deflate.allowed-encodings ordering as server preference list
when choosing an encoding from list in client request Accept header

x-ref:
  "ModDeflate questions (possibly some feature requests too)"
  https://redmine.lighttpd.net/boards/2/topics/9786
2 years ago
Glenn Strauss 09aa794cae [core] refactor config_check_cond_nocache() flow 2 years ago
Glenn Strauss a3f5fa3ff6 [core] config_check_cond_nocache() xor return code 2 years ago
Glenn Strauss 9fe8fbaa72 [multiple] http_method_buf()
- http_method_buf() returns (const buffer *)
- comment out unused get_http_status_name()
- inline func for http_append_method()

config processing requires a persistent buffer for method on the
off-chance that the config performed a capturing regex match in
$HTTP["method"] condition and used it later (e.g. in mod_rewrite)
(Prior behavior using r->tmp_buf was undefined in this case)
2 years ago
Glenn Strauss 1de03a03ee [core] reduce $HTTP["host"] compare str scanning 2 years ago
Glenn Strauss 4b9da9f1e8 [core] parse $HTTP["remote-ip"] CIDR mask at start
parse $HTTP["remote-ip"] CIDR mask into structured data at startup

note: adds buffer_move() to configparser.y to reduce memory copying
for all config values, and is required for remote-ip to preserve the
structured data added after the config value string.  (Alternatively,
could have normalized the remote-ip value after copying into dc->string)
2 years ago
Glenn Strauss e34ce5f217 [core] const buffer * in config_check_cond_nocache
use (const buffer *) in config_check_cond_nocache()
2 years ago
Glenn Strauss 1ca721d479 [core] reduce excess cc inlining in http_chunk.c 2 years ago
Glenn Strauss 3a568c639c [core] http_date_time_append() convenience macro
append IMF-fixdate time string to (buffer *)
2 years ago
Glenn Strauss 9f90f33288 [core] write_all() simpler loop; better asm 2 years ago
Glenn Strauss 18e96334d6 [core] proxy_create_env() tweaks
reorder some code for better asm

proxy_set_Forwarded() sets multiple request headers, and does so prior
to walking all request headers to create request to backend.  This is
done so that specific already-existing request headers from client are
overwritten (intentionally) in proxy_set_Forwarded().

Expect header is handled, but not expected since client-sent Expect
header is handled (and unset) in connection_handle_read_post_state()
2 years ago
Glenn Strauss 9f82ba8fab [mod_proxy] proxy_response_headers load v earlier
proxy_response_headers() issue variable load slightly earlier
2 years ago
Glenn Strauss f2cdee2430 [core] order gw_host members for packing and usage 2 years ago
Glenn Strauss 50f88480a6 [core] order gw_proc members for packing and usage 2 years ago
Glenn Strauss e89d75136c [core] avoid inlining buffer_eq_icase_ssn()
(slightly better asm)
2 years ago
Glenn Strauss 2899d7f983 [core] construct file path after docroot hook
also remove some excess/duplicative trace during request processing
2 years ago
Glenn Strauss 5bc92071d7 [core] reset large path buffers from long URLs 2 years ago
Glenn Strauss f1e8a82f1a [multiple] inline struct in con->dst_addr_buf
(mod_extforward recently changed to use buffer_move() to save addr
 instead of swapping pointers)
2 years ago
Glenn Strauss e6f286ef5f [multiple] reduce use of BUFFER_INTLEN_PTR
reduce use of BUFFER_INTLEN_PTR where b->ptr known not to be NULL
2 years ago
Glenn Strauss 38a9f0ebdd [mod_access] remove excess trace
trace is still issued if access is denied
  and r->log_request_handling is set
2 years ago
Glenn Strauss 2d9bf8062d [mod_extforward] inline some more data structures 2 years ago
Glenn Strauss 4ccd86f981 [core] allocate initial request pool w/ srv->conns
allocate reusable h2 request pool when initial srv->conns pool allocated
2 years ago
Glenn Strauss 1cd73b08a6 [core] move backtrace and assert macros to ck.[ch] 2 years ago
Glenn Strauss 4e5b822b16 [mod_magnet] cache script objects at config time
always check at least mtime and size to trigger script reload
2 years ago
Glenn Strauss f490078d0f [multiple] buffer_copy_string_len_lc()
convenience wrapper combining
  buffer_copy_string_len()
  buffer_to_lower()
and making a single pass over string
2 years ago
Glenn Strauss 8c5acaae3c [core] lowercase r->http_host, r->uri.authority
always lowercase r->http_host, r->uri.authority for consistency
2 years ago
Glenn Strauss 84fa4b8d1b [core] buffer_realloc() using power-2 realloc 2 years ago
Glenn Strauss caff799ccb [build] look for port.h on Solaris, not sys/port.h 2 years ago
Glenn Strauss 0f90a9e320 [core] reduce memcmp in http_request_parse_header
extend http_header_parse_ctx to support enum http_header_h2_e
HTTP/2 pseudo-headers in hpctx->id before using memcmp()
2 years ago
Glenn Strauss 5beee8b2d4 [core] buffer_path_simplify() quick(er) path
scan to detect (potential) need for path simplification
(repeated '/' or "/.") before copying each char in string
2 years ago