Commit Graph

64 Commits (5a58f6963cc93801b49268bfe2b59d8e6bfe3f10)

Author SHA1 Message Date
Glenn Strauss 309c1693ac [multiple] Y2038 32-bit signed time_t mitigations
Most OS platforms have already provided solutions to
Y2038 32-bit signed time_t 5 - 10 years ago (or more!)
Notable exceptions are Linux i686 and FreeBSD i386.

Since 32-bit systems tend to be embedded systems,
and since many distros take years to pick up new software,
this commit aims to provide Y2038 mitigations for lighttpd
running on 32-bit systems with Y2038-unsafe 32-bit signed time_t

* Y2038: lighttpd 1.4.60 and later report Y2038 safety
  $ lighttpd -V
    + Y2038 support                                    # Y2038-SAFE
  $ lighttpd -V
    - Y2038 support (unsafe 32-bit signed time_t)      # Y2038-UNSAFE

* Y2038: general platform info
  * Y2038-SAFE: lighttpd 64-bit builds on platforms using 64-bit time_t
      - all major 64-bit platforms (known to this author) use 64-bit time_t
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 64-bit time_t
      - Linux x32 ABI (different from i686)
      - FreeBSD all 32-bit and 64-bit architectures *except* 32-bit i386
      - NetBSD 6.0 (released Oct 2012) all 32-bit and 64-bit architectures
      - OpenBSD 5.5 (released May 2014) all 32-bit and 64-bit architectures
      - Microsoft Windows XP and Visual Studio 2005 (? unsure ?)
        Another reference suggests Visual Studio 2015 defaults to 64-bit time_t
      - MacOS 10.15 Catalina (released 2019) drops support for 32-bit apps
  * Y2038-SAFE: lighttpd 32-bit builds on platforms using 32-bit unsigned time_t
      - e.g. OpenVMS (unknown if lighttpd builds on this platform)
  * Y2038-UNSAFE: lighttpd 32-bit builds on platforms using 32-bit signed time_t
      - Linux 32-bit (including i686)
          - glibc 32-bit library support not yet available for 64-bit time_t
              - Linux kernel 5.6 on 32-bit platforms does support 64-bit time_t
                "Note: at this point, 64-bit time support in dual-time
                 configurations is work-in-progress, so for these
                 configurations, the public API only makes the 32-bit time
                 support available. In a later change, the public API will
                 allow user code to choose the time size for a given
                 compilation unit."
              - compiling with -D_TIME_BITS=64 currently has no effect
          - glibc recent (Jul 2021) mailing list discussion
      - FreeBSD i386
      - DragonFlyBSD 32-bit

* Y2038 mitigations attempted on Y2038-UNSAFE platforms (32-bit signed time_t)
  * lighttpd prefers system monotonic clock instead of realtime clock
    in places where realtime clock is not required
  * lighttpd treats negative time_t values as after 19 Jan 2038 03:14:07 GMT
  * (lighttpd presumes that lighttpd will not encounter dates before 1970
    during normal operation.)
  * lighttpd casts struct stat st.st_mtime (and st.st_*time) through uint64_t
    to convert negative timestamps for comparisions with 64-bit timestamps
    (treating negative timestamp values as after 19 Jan 2038 03:14:07 GMT)
  * lighttpd provides unix_time64_t (int64_t) and
  * lighttpd provides struct unix_timespec64 (unix_timespec64_t)
    (struct timespec equivalent using unix_time64_t tv_sec member)
  * lighttpd provides gmtime64_r() and localtime64_r() wrappers
    for platforms 32-bit platforms using 32-bit time_t and
    lighttpd temporarily shifts the year in order to use
    gmtime_r() and localtime_r() (or gmtime() and localtime())
    from standard libraries, before readjusting year and passing
    struct tm to formatting functions such as strftime()
  * lighttpd provides TIME64_CAST() macro to cast signed 32-bit time_t to
    unsigned 32-bit and then to unix_time64_t

* Note: while lighttpd tries handle times past 19 Jan 2038 03:14:07 GMT
  on 32-bit platforms using 32-bit signed time_t, underlying libraries and
  underlying filesystems might not behave properly after 32-bit signed time_t
  overflows (19 Jan 2038 03:14:08 GMT).  If a given 32-bit OS does not work
  properly using negative time_t values, then lighttpd likely will not work
  properly on that system.

* Other references and blogs
2021-09-04 08:08:26 -04:00
Pavel Gulchuk 2d6c511d23 [core] fix kill workers and shutdown by signal
github: closes #105
2021-05-15 22:00:48 -04:00
Glenn Strauss dde9df4310 [multiple] mark addtl funcs attr returns_nonnull 2021-03-26 07:33:42 -04:00
Glenn Strauss 19bc88850e [multiple] add attrs from gcc -Wsuggest-attribute= 2021-03-26 07:33:41 -04:00
Glenn Strauss 29f50bdeb0 [multiple] fdevent_waitpid() wrapper 2020-12-24 16:13:17 -05:00
Glenn Strauss 2b4166fb74 [core] fdevent_dup_cloexec() 2020-10-11 12:19:27 -04:00
Glenn Strauss 771ebd39a9 [core] fdevent_load_file_bytes()
wrapper to load defined number of bytes from file at given offset
2020-10-11 12:19:27 -04:00
Glenn Strauss be6da785c5 [mod_deflate] deflate.cache-dir compressed cache
(achieves feature parity with mod_compress)
2020-08-02 06:46:22 -04:00
Glenn Strauss cc04468762 [core] fdevent_load_file() shared code 2020-07-08 22:51:32 -04:00
Glenn Strauss 8588772caa [core] isolate fdevent subsystem
fdevent.c no longer directly uses struct server *srv
srv->srvconf.max_fds (if set) is used to set rlimits
set max_conns in server.c after fdevent_init(), which sets srv->max_fds
  using srv->srvconf.max_fds (if set) as input hint
2020-07-08 19:54:29 -04:00
Glenn Strauss 48004c6a7c [core] remove unused arg to fdevent_fcntl_set_nb* 2020-07-08 19:54:29 -04:00
Glenn Strauss 05cc88ddb4 [multiple] omit passing srv to fdevent_handler 2020-07-08 19:54:29 -04:00
Glenn Strauss a78404cfbf [core] fdevent_mkstemp_append() (shared) 2019-03-16 01:38:29 -04:00
Glenn Strauss 37bd124ae4 [core] pass conf.follow_symlink in more places 2019-03-10 23:22:58 -04:00
Glenn Strauss 9113011d5b [core] pass (fdnode *) for registered fdevent fd
inline fdevent_fdnode_interest()
avoid array lookup for (fdnode *) for various ops
2019-03-07 00:32:17 -05:00
Glenn Strauss 41384a3657 [core] isolate fdevent processing 2019-03-07 00:32:17 -05:00
Glenn Strauss 800e9b7349 [core] remove fde_ndx member outside fdevents
(isolated to fdevent framework internals)
2019-03-07 00:32:17 -05:00
Glenn Strauss 0a46f7ec23 [core] some fdevent code streamlining
isolate fde_ndx to fdevent framework internals
2019-03-07 00:32:17 -05:00
Glenn Strauss fc914ae442 [core] map FDEVENT_* to OS system event frameworks
(avoid need to translate event flags between OS and lighttpd FDEVENT_*)
2019-02-17 18:35:05 -05:00
Glenn Strauss f5f221eda8 [core] dispatch events from within event framework
event framework now calls interface to dispatch events rather than
itself implementing multiple interfaces for fdevent_process() to
be able to dispatch events generically.
2019-02-13 21:34:59 -05:00
Glenn Strauss 653b0dac7c [core] fdevent_process()
process fdevents in fdevent.c
2019-02-04 03:01:33 -05:00
Glenn Strauss fb9b8ad8ae [core] mark startup/shutdown funcs cold 2019-02-04 02:25:48 -05:00
Glenn Strauss 7c8cc6f7c5 [core] option to propagate TCP FIN to backend host
(experimental support for mod_sockproxy)

"tcp-fin-propagate" = "enable" for each host in *.server backend defs
2018-08-12 14:43:22 -04:00
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
2018-04-08 22:22:23 -04:00
Glenn Strauss 210b57708e [core] fix rare race condition from backends (fixes #2878)
fix rare race condition from backends with

(thx abelbeck)

  "fastcgi and stream-response-body=2 hangs on last chunk"
2018-03-18 19:01:32 -04:00
Glenn Strauss 84b5064dc4 [core] discard from socket using recv MSG_TRUNC
discard from socket using recv MSG_TRUNC on Linux TCP SOCK_STREAM socket

Currently, lighttpd supports only TCP SOCK_STREAM.  If UDP SOCK_DGRAM
were to be supported in the future, then socket type will need to be
stored so that MSG_TRUNC is used appropriately for the desired effect.

To find out socket type on arbitrary socket fd:
  getsockopt(..., SOL_SOCKET, SO_TYPE, ...)
but better to store it with each listening socket.
2017-12-11 21:35:31 -05:00
Glenn Strauss d5d0258362 [core] support POLLRDHUP, where available (#2743)
  "mod_cgi, lighty not killing CGI if connection in the other end is closed"
  "1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
2017-11-19 12:01:09 -05:00
Glenn Strauss 8f3bbd7f13 [core] isolate backend fdevent handler defs 2017-11-03 23:02:08 -04:00
Glenn Strauss 168f67a1b1 [core] perf: more efficient fdevent_sched_run()
perf: more efficient fdevent_sched_run() over fds pending close
2017-10-21 12:34:17 -04:00
Glenn Strauss 93e91954a7 [core] fdevent setsockopt() helper functions
2017-09-23 10:30:08 -04:00
Glenn Strauss 9030cfaecf [core] SIGCHLD handle_waitpid hook for modules
centralize most waitpid() handling in core server, with hooks for
modules to be informed of pid and status when a process exits.

This enables faster discovery (and restart) of exited processes,
and also allows for lighttpd to manage backend processes in the
parent (master) process when server.max-worker > 0.
2017-09-10 16:24:34 -04:00
Glenn Strauss d4a37a3bbb [core] remove fdevent fcntl_set hook
(could have been removed in 2010 with commit 38f2d1dd
 which removed fdevent_linux_rtsig.c)
2017-07-29 00:36:48 -04:00
Glenn Strauss adeec956c3 [core] restart piped loggers if they exit (fixes #1393)
  "access log pipe writer should restart child process if it exits"
2017-07-23 19:02:31 -04:00
Glenn Strauss ad7d6a6b15 [core] fdevent_cycle_logger()
fdevent_cycle_logger() re-opens log files before closing existing fd
2017-07-23 19:02:31 -04:00
Glenn Strauss 91d14acfc3 [core] fdevent_connect_status() shared code 2017-07-15 22:42:15 -04:00
Glenn Strauss 685cab057e [core] rename fd_close_on_exec()
rename fd_close_on_exec() to fdevent_setfd_cloexec()
2017-07-15 22:42:15 -04:00
Glenn Strauss a9970fec23 [core] consolidate fork()/execve() code (#1393)
(refactoring work to address issue #1393)

  "access log pipe writer should restart child process if it exits"
2017-07-15 22:42:15 -04:00
Glenn Strauss 8641d1b03f [core] no SOCK_CLOEXEC on Linux kernel < 2.6.27
Linux kernels < 2.6.27 (old!) might return EINVAL if SOCK_CLOEXEC used

2017-04-15 17:46:25 -04:00
Glenn Strauss 4796313efc [core] collect ioctl FIONREAD code
include <sys/ioctl.h> in files which use ioctl()
  instead of exposing header in local header "sys-socket.h"
2017-03-28 02:17:33 -04:00
Glenn Strauss ba953cdf45 [core] include "fdevent.h" where needed
(instead of providing #include "fdevent.h" in base.h)
2017-03-28 02:17:33 -04:00
Glenn Strauss 93afda9c8e performance: use Linux extended syscalls and flags
reduce syscalls on Linux using extended syscalls and flags,
e.g. accept4(), pipe2(), O_CLOEXEC, SOCK_CLOEXEC, SOCK_NONBLOCK

github: closes #2
2016-09-24 02:23:49 -04:00
Glenn Strauss 40f16d52db [core] fix crash if ready events on abandoned fd (fixes #2748)
  "1.4.40/1.4.41 uploads to CGI may cause crash (SIGABRT)"
2016-08-24 15:30:11 -04:00
Glenn Strauss 4bc06bfc0b [core] check if client half-closed TCP if POLLHUP (#2743)
Check if client half-closed TCP connection if POLLHUP is received.
This more robustly handles if client called shutdown(fd, SHUT_WR).

This patch reverts commit:ab05eb7c which should now be handled properly.
(Time will tell.)

  "1.4.40/41 mod_proxy, mod_scgi may trigger POLLHUP on *BSD,Darwin"
2016-08-06 04:28:45 -04:00
Glenn Strauss 5ce7b2df5c [core] add declarations to fdevent.h (#2373)
(follow-up to 79bcfab0)

  "FreeBSD build with libev broken"
2016-06-29 21:12:21 -04:00
Glenn Strauss 879a282de7 fix errors detected by Coverity Scan
fd leak in mod_dirlisting.c
use after free in error condition in mod_proxy.c
NULL pointer dereference in error message in chunk.c

additional minor code changes made to quiet other coverity warnings
2016-06-21 03:53:50 -04:00
Glenn Strauss 18a7b2be37 [core] option to stream response body to client (fixes #949, #760, #1283, #1387)
Set = 1 or = 2
to have lighttpd stream response body to client as it arrives from the
backend (CGI, FastCGI, SCGI, proxy).

default: buffer entire response body before sending response to client.
(This preserves existing behavior for now, but may in the future be
 changed to stream response to client, which is the behavior more
 commonly expected.)

  "fastcgi, cgi, flush, php5 problem."
  "Random crashing on FreeBSD 6.1"
  "Memory usage increases when proxy+ssl+large file"
  "lighttpd+fastcgi memory problem"
2016-06-19 23:34:16 -04:00
Glenn Strauss 695c8f4e07 [config] config options to stream request/response (#949, #376)
This allows admin to configure if response is collected in entirety
prior to sending data to client

For compatibility with existing configs, default is existing behavior:
  buffer entire response prior to sending data to client

The following are config options, though not all implemented yet

// default: buffer entire request body before connecting to backend = 0

// stream request body to backend; buffer to temp files = 1

// stream request body to backend; minimal buffering might block upload = 2

// default: buffer entire response body before sending to client = 0

// stream response body to client; buffer to temp files = 1

// stream response body to client; minimal buffering might block backend = 2

  "fastcgi, cgi, flush, php5 problem."
 "Reimplement upload (POST) handling to match apache/zeus/thttpd/boa functionality"
2016-06-19 23:34:15 -04:00
Glenn Strauss 4b0c822ed0 always poll for client POLLHUP/POLLERR events (fixes #399)
to detect client disconnect.  Do so even when waiting on backend,
and not polling for POLLRD or POLLWR on client connection.

This reduces unnecessary load on backends when backends are slow
to respond and client has given up waiting.

  FastCGI performance on high load
2016-04-18 06:19:03 -04:00
Glenn Strauss 8abd06a7ff consistent inclusion of config.h at top of files (fixes #2073)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:14:35 +00:00
Stefan Bühler bfce99aacc [bitset] unused -> remove
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:51 +00:00