Commit Graph

265 Commits

Author SHA1 Message Date
Glenn Strauss f69bd9cdb8 [core] perf: simple, quick buffer_clear()
quickly clear buffer instead of buffer_string_set_length(b, 0) or
buffer_reset(b).  Avoids free() of large buffers about to be reused,
or buffers that are module-scoped, persistent, and reused.

(buffer_reset() should still be used with buffers in connection *con
 when the data in the buffers is supplied by external, untrusted source)
2018-11-23 00:37:38 -05:00
Glenn Strauss 19b5fbdd3d [mod_fastcgi] transfer chunks minus packet padding
(thx avij)
2018-11-19 20:52:34 -05:00
Glenn Strauss b5cd216781 [mod_fastcgi] minor: copy packet without padding 2018-11-18 17:04:02 -05:00
Glenn Strauss 9f6a46738d [multiple] dynamic handlers hint backend header sz 2018-11-12 21:56:05 -05:00
Glenn Strauss 6dcdc1af18 [mod_fastcgi,mod_scgi] perf: env accumulation 2018-11-12 21:56:05 -05:00
Glenn Strauss 3d8d56d8de [multiple] perf: reuse large buffers w/ backend 2018-11-12 21:56:05 -05:00
Glenn Strauss c79bc31609 [mod_fastcgi] perf: reduce data copies
http_chunk_transfer_cqlen()
2018-11-12 08:25:05 -05:00
Glenn Strauss 2df8f9ebf4 [mod_fastcgi] perf: reduce data copies
reduce data copies while reading FastCGI packets from FastCGI backend
2018-11-12 08:25:05 -05:00
Glenn Strauss f410431b58 [multiple] perf: use larger initial backend buffer
use larger initial backend buffer if the client request is large
2018-11-12 08:25:01 -05:00
Glenn Strauss 7e8beee577 [multiple] perf: power-2 alloc large headers
allocate large headers by power-2 as headers to backend is generated
(mod_proxy, mod_fastcgi, mod_scgi)
2018-10-29 09:00:46 -04:00
Glenn Strauss 610d5c03ab [multiple] perf: write headers to backend write cq
write headers directly to dynamic backend write chunkqueue
(mod_proxy, mod_fastcgi, mod_scgi)
2018-10-29 09:00:46 -04:00
Glenn Strauss 4992c4de10 [mod_fastcgi,mod_scgi] error on oversized request (fixes #2905)
regression in lighttpd 1.4.49 and lighttpd 1.4.50

(thx slawomir.pryczek)

x-ref
  "oversized fcgi requests should fail gracefully"
  https://redmine.lighttpd.net/issues/2905
2018-09-23 18:01:58 -04:00
Glenn Strauss eb429c9c19 [mod_fastcgi] fix memleak with FastCGI auth,resp (fixes #2894)
fix memleak in mod_fastcgi when FastCGI is used for both authentication
and response on the same request

(thx rschmid)

x-ref:
  "Memory leak if two fcgi calls with one request (authentication and response)"
  https://redmine.lighttpd.net/issues/2894
2018-08-12 14:43:22 -04:00
Glenn Strauss 3efaff973f [mod_fastcgi,mod_scgi] fastcgi.balance,scgi.balance (fixes #2882)
x-ref:
  "fastcgi.balance not working"
  https://redmine.lighttpd.net/issues/2882
2018-08-05 03:44:15 -04:00
Glenn Strauss 04d76e7afd [core] some header cleanup
provide standard types in first.h instead of base.h
provide lighttpd types in base_decls.h instead of settings.h
reduce headers exposed by headers for core data structures
  do not expose <pcre.h> or <stdlib.h> in headers
move stat_cache_entry to stat_cache.h
reduce use of "server.h" and "base.h" in headers
2018-04-08 22:22:23 -04:00
Glenn Strauss 210b57708e [core] fix rare race condition from backends (fixes #2878)
fix rare race condition from backends with server.stream-response-body=2

(thx abelbeck)

x-ref:
  "fastcgi and stream-response-body=2 hangs on last chunk"
  https://redmine.lighttpd.net/issues/2878
2018-03-18 19:01:32 -04:00
Glenn Strauss 9030cfaecf [core] SIGCHLD handle_waitpid hook for modules
centralize most waitpid() handling in core server, with hooks for
modules to be informed of pid and status when a process exits.

This enables faster discovery (and restart) of exited processes,
and also allows for lighttpd to manage backend processes in the
parent (master) process when server.max-worker > 0.
2017-09-10 16:24:34 -04:00
Glenn Strauss 748fc860fb [mod_fastcgi] fix return when streaming min buffer
fix incorrect return value when server.stream-response-body = 2
2017-08-12 12:59:58 -04:00
Glenn Strauss 45b970e69b [core] shared code for socket backends
common codebase for socket backends, based off mod_fastcgi with
some features added for mod_proxy

(mostly intended to reduce code duplication and enhance code isolation)

mod_fastcgi and mod_scgi can now use fastcgi.balance and scgi.balance
  for similar behavior as proxy.balance, but the balancing is per-host
  and not per-proc.  proxy.balance is also per-host and not per-proc.

mod_proxy and mod_scgi can now use proxy.map-extensions and
scgi.map-extensions, similar to fastcgi.map-extensions.

mod_fastcgi behavior change (affects only mod_status):
- statistics tags have been renamed from "fastcgi.*" to "gw.*"
  "fastcgi.backend.*"       -> "gw.backend.*"
  "fastcgi.active-requests" -> "gw.active-requests"
  ("fastcgi.requests" remains "fastcgi.requests")
  ("proxy.requests" is new)
  ("scgi.requests" is new)

mod_scgi behavior change (likely minor):
- removed scgi_proclist_sort_down() and scgi_proclist_sort_up().
  procs now chosen based on load as measured by num socket connnections

Note:
modules using gw_backend.[ch] are currently still independent modules.
If it had been written as a single module with fastcgi, scgi, proxy
implementations, then there would have been a chance of breaking some
existing user configurations where module ordering made a difference
for which module handled a given request, though for most people, this
would have made no difference.

Details about mod_fastcgi code transformations:
unsigned int debug -> int debug
fastcgi_env member removed from plugin_config
renamed "fcgi" and "fastcgi" to "gw", and "FCGI" to "GW"
reorganize routines for high-level and lower-level interfaces
some lower-level internal interfaces changed to use host,proc,debug
  args rather than knowing about higher-level (app) hctx and plugin_data
tabs->spaces and reformatting
2017-07-23 19:02:00 -04:00
Glenn Strauss 81b8fffd31 [mod_fastcgi,mod_scgi] use temp buffer for cgi_env 2017-07-15 22:42:15 -04:00
Glenn Strauss 6e1c02f386 [mod_fastcgi,mod_scgi] parse bin_path at startup 2017-07-15 22:42:15 -04:00
Glenn Strauss 35c4bf5857 [mod_fastcgi,mod_scgi] struct member consistency 2017-07-15 22:42:15 -04:00
Glenn Strauss c0a9020935 [mod_fastcgi,mod_scgi] consistent connect() error
more consistent connect() error handling

NOTE: behavior change in mod_scgi:
  "disable-time" default is now 1 second (was 60 seconds)
  The new behavior matches the default in mod_fastcgi
  (and is a much saner default disable time).
2017-07-15 22:42:15 -04:00
Glenn Strauss d836c72751 [mod_fastcgi] move delayed connect() into switch()
move delayed connect() handling into switch()
2017-07-15 22:42:15 -04:00
Glenn Strauss d26837d509 [mod_fastcgi] set request_id in fcgi_create_env()
set hctx->request_id in fcgi_create_env()
2017-07-15 22:42:15 -04:00
Glenn Strauss 9cfa9dc3d4 [mod_fastcgi] consolidate connect() error handling 2017-07-15 22:42:15 -04:00
Glenn Strauss b65bdb5540 [mod_fastcgi] slightly simplify counters 2017-07-15 22:42:15 -04:00
Glenn Strauss 91d14acfc3 [core] fdevent_connect_status() shared code 2017-07-15 22:42:15 -04:00
Glenn Strauss 1002574692 [core] continue collecting use of netdb.h
continue collecting use of netdb.h into inet_ntop_cache.[ch]
2017-07-15 22:42:15 -04:00
Glenn Strauss cafb091cd4 [core] remove unused includes of stat_cache.h 2017-07-15 22:42:15 -04:00
Glenn Strauss a9970fec23 [core] consolidate fork()/execve() code (#1393)
(refactoring work to address issue #1393)

x-ref:
  "access log pipe writer should restart child process if it exits"
  https://redmine.lighttpd.net/issues/1393
2017-07-15 22:42:15 -04:00
Gaël PORTAY e8498bbfcc [core] silence compiler warnings if !HAVE_FORK
silence compiler warnings if HAVE_FORK is not set

However, if HAVE_FORK is not set, then -Werror was probably passed to
./configure, which is currently a mistake.  lighttpd can successfully
compiles src/ with -Werror on many platforms, but ./configure tests
should not be run with -Werror. [gstrauss]

github: closes #81

x-ref:
   "Fix warnings"
   https://github.com/lighttpd/lighttpd1.4/pull/81
2017-05-15 22:02:33 -04:00
Glenn Strauss 7a27d5eff5 [core] buffer to disk streaming to slow backends
buffer input to disk when streaming request body to slow backends
2017-05-15 22:02:33 -04:00
Glenn Strauss 86bb8be2c8 [core] perf: skip redundant strlen() if len known
performance: skip redundant strlen() if length is already known

introduce array_get_element_klen() to take key and klen params
2017-05-15 22:02:33 -04:00
Glenn Strauss 316e959b4d [core] prep mod transitions to transparent proxy
prep mod_proxy,mod_fastcgi,mod_scgi for transition to transparent proxy
2017-05-14 00:09:23 -04:00
Glenn Strauss a48d65c8a5 [core] remove redundant resets of fde_ndx
after initialization, value of fde_ndx should be managed by fdevent.c
2017-05-14 00:09:23 -04:00
Glenn Strauss 77509ed087 [mod_fastcgi] consolidate backend read code
Use same funcs as other dynamic handlers to recv data from backend.
Add hook for fastcgi to process FastCGI packets (and other future
dynamic handlers may hook this in order to handle custom data framing)
2017-05-10 23:21:15 -04:00
Glenn Strauss 1465cf80f9 [core] calloc plugin_config for consistent init 2017-04-30 20:38:03 -04:00
Glenn Strauss 8641d1b03f [core] no SOCK_CLOEXEC on Linux kernel < 2.6.27
Linux kernels < 2.6.27 (old!) might return EINVAL if SOCK_CLOEXEC used

x-ref:
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=529929
  http://www.linksysinfo.org/index.php?threads/lighttpd-no-longer-starts-toastman-1-28-0510-7.73132/
2017-04-15 17:46:25 -04:00
Glenn Strauss 4796313efc [core] collect ioctl FIONREAD code
include <sys/ioctl.h> in files which use ioctl()
  instead of exposing header in local header "sys-socket.h"
2017-03-28 02:17:33 -04:00
Glenn Strauss a53f662a30 [core] remove some unused header includes
remove exposure of stdio.h in buffer.h for print_backtrace(), now static
2017-03-28 02:17:33 -04:00
Glenn Strauss aa14493e47 [tests] reduce time waiting for backends to start
reduce time spent waiting for backends to start

tests check for active listening port before proceeding

test runs now complete in about 2/3 the time
2017-03-19 23:48:50 -04:00
Glenn Strauss 0a635fc8be [core] consolidate dynamic handler response parse
- consolidate dynamic handler HTTP response parsing code
- reduce string copies for CGI, FastCGI, SCGI, proxy response headers
- let read() signal EOF or EAGAIN instead of ioctl FIONREAD 0-data-ready
2017-03-19 23:48:50 -04:00
Glenn Strauss e4bb56222f [mod_cgi,fastcgi,scgi,proxy] fix streaming response (fixes #2796)
fix streaming response when server.stream-response-body = 2
and client catches up to stream from backend

(thx horgh)

x-ref:
  "mod_fastcgi can fail to read entire response from server"
  https://redmine.lighttpd.net/issues/2796
2017-03-13 13:49:17 -04:00
Glenn Strauss bd77abe0f8 [config] more specific checks for array lists
More specific checks on contents of array lists.  Each module using
lists now does better checking on the types of values in the list
(strings, integers, arrays/lists)

This helps prevent misconfiguration of things like cgi.assign,
fastcgi.server, and scgi.server, where source code might be
served as static files if parenthesis are misplaced.

x-ref:
  https://redmine.lighttpd.net/boards/2/topics/6571
2017-03-08 11:42:59 -05:00
Glenn Strauss 1dd5cce3bc [mod_fastcgi,mod_scgi] consolidate backend process accounting (#2788)
consolidate backend process accounting for consistency

x-ref:
  "FreeBSD/1.4.45/SSL: requests getting stuck in handle-req state occasionally"
  https://redmine.lighttpd.net/issues/2788
2017-02-25 12:41:21 -05:00
Glenn Strauss 23c3535efc [mod_scgi] fix potential repeated use of proc->id
host->max_id is now number of procs allocated
proc->id is used when generated unix socket name or port for proc
2017-02-25 12:41:21 -05:00
Glenn Strauss fb6b4027c8 [mod_fastcgi,mod_scgi] consistent waitpid handling (fixes #2791)
more consistent waitpid() handling, consolidate similar code

If ECHILD received for a given pid, do not retry waitpid() for that pid

x-ref:
  "mod_fastcgi : pid {pid} 1 not found: No child processes"
  https://redmine.lighttpd.net/issues/2791
2017-02-25 12:41:18 -05:00
Glenn Strauss aa923e05f6 [mod_fastcgi,mod_scgi] backend spawn EINTR retry (#2788)
When spawning backends, retry blocking connect() to backend if EINTR
received when attempting to see if backend is already running.  EINTR
might be received if a HUP or USR1 signal is received while connecting
(or SIGCHLD on systems without SA_RESTART)

(expected to occur extremely rarely, but simple to handle properly)

x-ref:
  "FreeBSD/1.4.45/SSL: requests getting stuck in handle-req state occasionally"
  https://redmine.lighttpd.net/issues/2788
2017-02-11 12:35:49 -05:00
Glenn Strauss 3209f30d11 [core] handle if backend sends Transfer-Encoding (#2786)
It is still not a good idea for backend to send Transfer-Encoding unless
backend is mod_proxy, and mod_proxy should not currently receive chunked
response since mod_proxy sends HTTP/1.0 request.

If mod_proxy is changed to sent HTTP/1.1 request, then lighttpd would
need to check if client is HTTP/1.0 and would need to de-chunk and
remove any other transfer-codings if not supported by next-hop.

x-ref:
  "error 500 (mod_cgi.c.601) cgi died"
  https://redmine.lighttpd.net/issues/2786
2017-02-11 11:06:31 -05:00