Commit Graph

60 Commits

Author SHA1 Message Date
Glenn Strauss 24680a9142 [core] array_init() arg for initial size 2020-07-08 18:08:52 -04:00
Glenn Strauss e3dc34d142 [core] array a->sorted[] as ptrs rather than pos
While slightly more memory use in 64-bit (though same memory use as
prior versions of lighttpd), avoids bouncing through second array
when searching in sorted list.  Most use of arrays in lighttpd is to
build a list once, and elements are not removed from the list.
2020-05-23 17:59:29 -04:00
Glenn Strauss a762402da5 [core] keep a->data[] sorted (REVERT)
This reverts commit 2260a8062ee599ecf28d9b52b981603fd2084aff.

original ordering of array elements is significant
e.g. in lighttpd.conf lists where first match to request is applied
2020-02-24 11:15:32 -05:00
Glenn Strauss c2238256e2 [core] inline array as part of data_array value
(instead of value being (array *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 601c572c39 [core] inline buffer as part of data_string value
(instead of value being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss b7942c58cc [core] (data_unset *) from array_get_element_klen
return (data_unset *) from array_get_element_klen() to prep for
putting buffer into data_string for value member, rather than as
(buffer *)

(allow in-place modification of these buffer values)
2020-02-24 11:15:32 -05:00
Glenn Strauss ad9b7e009b [core] inline buffer as part of DATA_UNSET key
(instead of key being (buffer *))
2020-02-24 11:15:32 -05:00
Glenn Strauss 83535bbef3 [core] differentiate array_get_* for ro and rw
array_get_element_klen() is now intended for read-only access
array_get_data_unset() is used by config processing for r/w access
array_get_buf_ptr() is used for r/w access to ds->value (string buffer)
2020-02-24 11:15:32 -05:00
Glenn Strauss c9f1b612f1 [core] keep a->data[] sorted; remove a->sorted[] 2020-02-24 11:15:32 -05:00
Glenn Strauss 61785d868f [core] array keys are non-empty in key-value list 2020-02-24 11:15:32 -05:00
Glenn Strauss db5ff222e4 [core] short-circuit path to clear request.headers
short-circuit path to clear con->request.headers if entire size of
all request headers is <= BUFFER_MAX_REUSE_SIZE

clear (reset) data_string key and value upon reuse
2020-02-24 11:15:32 -05:00
Glenn Strauss ddb78f75ee [core] remove unused array_reset() 2020-02-24 11:15:32 -05:00
Glenn Strauss b2991c686d [core] perf: array.c performance enhancements
mark array_get_index() as hot, rewrite to be pure and return sorted pos

mark routines as pure, as appropriate

mark routines as cold if used only at startup for config processing

mark params const, as appropriate

array_get_buf_ptr() for modifiable value buffer after insert into array

uint32_t used and size members instead of size_t

remove a->unique_ndx member; simply add to end of array for value lists
remove du->is_index_key member; simply check buffer_is_empty(du->key)

array_insert_key_value() used to be a hint that lookup could be skipped,
but the state from array_get_index() is now saved and reused internally,
so the distinction is no longer needed.  Use array_set_key_value().
2020-02-24 11:15:32 -05:00
Glenn Strauss e20b5318d5 [core] use buffer_eq_icase_ssn func
specialized buffer_eq_icase_ssn func replace strncasecmp()
in cases where string lengths are known to be at least as
large as the len being compared case-insensitively
2019-06-06 02:48:43 -04:00
Glenn Strauss ca059d580d [core] array-specialized buffer_caseless_compare()
specialize buffer_caseless_compare() for array.c
2019-06-06 02:48:43 -04:00
Glenn Strauss 9459c05468 [core] fix mixed use of srv->split_vals array (fixes #2932)
regression in mod_evhost in lighttpd 1.4.53
regression in mod_flv_streaming in lighttpd 1.4.51 - lighttpd 1.4.53

(thx moisseev)

  "[regression] %0 pattern does not match hostnames without the domain part"
2019-02-13 19:54:22 -05:00
Glenn Strauss d28bac32fe [multiple] reduce code dup in list resizing
reduce code duplication in list resizing
realloc() of NULL ptr has behavior similar to malloc()

Note that if initial size == 0, then code used to adjust size
must be += x to ensure the size is non-zero for reallocation.
(Multiplying 0 * x, e.g. power-2 resizing, will result in 0.)
2019-02-12 22:36:04 -05:00
Glenn Strauss 758174ecbc [core] perf: specialized func for array sorting
specialized func for array sorting for use in binary search
2018-10-28 02:38:10 -04:00
Glenn Strauss 062089ff14 [core] perf: array_reset_data_strings()
array_reset_data_strings() specialization
2018-10-22 20:28:53 -04:00
Glenn Strauss 8c7f1dfb03 [core] more memory-efficient fn table for data_*
save 40 bytes (64-bit), or 16 bytes (32-bit) per data_* element
at the cost of going through indirect function pointer to execute
methods.  At runtime, the reset() method is most used among them.
2018-09-23 18:01:58 -04:00
Glenn Strauss 002a4c524d [core] array_get_int_ptr() 2018-09-23 18:01:58 -04:00
Glenn Strauss 66ff05db8f [tests] t/test_array.c
(more tests should be added, but starting with something has benefits)
2018-09-23 18:01:58 -04:00
Glenn Strauss 810109cc34 [multiple] code reuse: using array_*() funcs 2018-09-23 18:01:58 -04:00
Glenn Strauss e6741acd4e [core] code reuse array_match_*() routines 2018-09-23 18:01:58 -04:00
Glenn Strauss a7c27c9f99 [core] code reuse with array_insert_key_value()
code reuse with array_insert_key_value() and related array manipulation
2018-09-16 05:18:05 -04:00
Glenn Strauss a46bc4f5de [core] remove proc_open.[ch], reduce stdio.h use 2018-08-05 03:44:15 -04:00
Glenn Strauss 2496c1af4c [core] pass array_get_element_klen() const array * 2018-02-02 06:22:33 -05:00
Glenn Strauss 86bb8be2c8 [core] perf: skip redundant strlen() if len known
performance: skip redundant strlen() if length is already known

introduce array_get_element_klen() to take key and klen params
2017-05-15 22:02:33 -04:00
Glenn Strauss bd77abe0f8 [config] more specific checks for array lists
More specific checks on contents of array lists.  Each module using
lists now does better checking on the types of values in the list
(strings, integers, arrays/lists)

This helps prevent misconfiguration of things like cgi.assign,
fastcgi.server, and scgi.server, where source code might be
served as static files if parenthesis are misplaced.

2017-03-08 11:42:59 -05:00
Glenn Strauss 381aaae363 remove unused array type TYPE_COUNT data_count
(unused type, and very similar to TYPE_INTEGER data_integer,
 differing only in initial value and how dup inserts are merged)
2016-09-22 19:54:57 -04:00
Glenn Strauss acad2c903a fix some warnings reported by cppcheck
fix some warnings reported by cppcheck and
change mod_skeleton.c to use buffer_string_length()
2016-07-17 16:13:31 -04:00
Stefan Bühler 5c68caa6d7 [core] replace array weakref with vector
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:27:38 +00:00
Glenn Strauss 8abd06a7ff consistent inclusion of config.h at top of files (fixes #2073)
From: Glenn Strauss <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-19 15:14:35 +00:00
Stefan Bühler 68e4a416cc [core] provide array_extract_element and use it
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:56:02 +00:00
Stefan Bühler 224bf545c1 [core] refactor array search; raise array size limit to SSIZE_MAX
- raise array size limit from INT_MAX to SSIZE_MAX. INT_MAX already
  is way to high to be hit in any sane scenario, but SSIZE_MAX can
  *obviously* not be hit due to memory constraints.
- use size_t for array indices instead of int
- use binary search instead of next_power_of_2 hack; document invariants
  and check them in debug mode (asserts).
- return the actual insert position instead of something near

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:41:57 +00:00
Stefan Bühler be121a638b [core] rename variable in array.c
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:41:54 +00:00
Stefan Bühler 8d8ae9cbc8 [core] improve array API to prevent theoretical memory leaks
- refactor insert into array_find_or_insert; if the element already
  exists the caller must resolve the conflict manually:
  - array_replace frees the old element
  - array_insert_unique calls "insert_dup"
  both have no return value anymore
- fix usages of array_replace; they now don't need to delete the old
  entry anymore; usage in configparser was probably broken, as it
  possibly deleted the old element before calling array_replace

This should fix a lot of the issues reported in "Fortify Open Review
Project - lighttpd 1.4.39" (usually hitting the array limit):
when the array size limit was hit "new" entries leaked instead of
getting added.

On 32-bit INT_MAX entries cannot actually be reached (each entry
requires at least 48 bytes, leading to a total of 96GB memory).

On 64-bit INT_MAX entries would require 224GB memory, so it would be
theoretically possible. But it would need 2^27 reallocations of two
C-arrays of up to 16GB size.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-03-15 18:26:57 +00:00
Stefan Bühler 566cf8decb add force_assert for more allocation results
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2016-01-30 13:59:07 +00:00
Stefan Bühler ad3e93ea96 Use buffer API to read and modify "used" member
- a lot of code tried to handle manually adding terminating zeroes and
  keeping track of the correct "used" count.
  Replaced all "external" usages with simple wrapper functions:
  * buffer_string_is_empty (used <= 1), buffer_is_empty (used == 0);
    prefer buffer_string_is_empty
  * buffer_string_set_length
  * buffer_string_length
  * CONST_BUF_LEN() macro
- removed "static" buffer hacks (buffers pointing to constant/stack
  memory instead of malloc()ed data)
- buffer_append_strftime(): refactor buffer+strftime uses
- li_tohex(): no need for a buffer for binary-to-hex conversion:
  the output data length is easy to predict
- remove "-Winline" from extra warnings: the "inline" keyword just
  supresses the warning about unused but defined (static) functions;
  don't care whether it actually gets inlined or not.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 19:10:44 +00:00
Stefan Bühler 6afad87d2e fix buffer, chunk and http_chunk API
* remove unused structs and functions
    (buffer_array, read_buffer)
  * change return type from int to void for many functions,
    as the return value (indicating error/success) was never checked,
    and the function would only fail on programming errors and not on
    invalid input; changed functions to use force_assert instead of
    returning an error.
  * all "len" parameters now are the real size of the memory to be read.
    the length of strings is given always without the terminating 0.
  * the "buffer" struct still counts the terminating 0 in ->used,
    provide buffer_string_length() to get the length of a string in a
    unset config "strings" have used == 0, which is used in some places
    to distinguish unset values from "" (empty string) values.
  * most buffer usages should now use it as string container.
  * optimise some buffer copying by "moving" data to other buffers
  * use (u)intmax_t for generic int-to-string functions
  * remove unused enum values: UNUSED_CHUNK, ENCODING_UNSET
  * converted BUFFER_APPEND_SLASH to inline function (no macro feature
  * refactor: create chunkqueue_steal: moving (partial) chunks into another
  * http_chunk: added separate function to terminate chunked body instead of
    magic handling in http_chunk_append_mem().
    http_chunk_append_* now handle empty chunks, and never terminate the
    chunked body.

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2015-02-08 12:37:10 +00:00
Stefan Bühler 07dd0bd0a5 add force_assert() to enforce assertions as simple assert()s are disabled by -DNDEBUG (fixes #2546)
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2014-02-16 13:08:20 +00:00
Stefan Bühler 524e0a30a0 add two asserts to help parfait (#2530)
From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2014-01-10 12:05:09 +00:00
Stefan Bühler 6f208cfde1 fix/silence bugs reported by ccc-analyzer (clang)
These should all be non critical:
 * memory leaks on startup in error cases (which lead to
   immediate shutdowns anyway)
 * http_auth/ldap: passing uninitialized "ret" to ldap_err2string
 * sizeof(T) not matching the target pointer in malloc/calloc calls;
   those cases were either:
   * T being the wrong pointer type - shouldn't matter as long as all
     pointers have same size
   * T being larger than the type needed
 * mod_accesslog: direct use after free in cleanup (server shutdown);
   could crash before "clean" shutdown
 * some false positives (mod_compress, mod_expire)
 * assert(srv->config_context->used > 0); - this is always the case,
   as there is always a global config block

From: Stefan Bühler <>

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2013-11-13 11:43:26 +00:00
Stefan Bühler 38f2d1ddd7 cleanup fdevent code, removed linux-rtsig handler, replaced some fprintf calls
* use log functions
 * convert flags
 * fix handler callback prototype

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2010-08-06 21:57:15 +00:00
Stefan Bühler 12f375f3b1 array.c: improve array_get_unused_element to check data type; fix mem leak if unused_element didn't find a matching entry (fixes #2145)
- the "mem leak" could only be triggered if you use different entry
   types in the same array (this wasn't supported by
   array_get_unused_element) or didn't call array_get_unused_element
   before creating new entries.

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2010-08-05 21:08:23 +00:00
Stefan Bühler 22e8b456a9 Fix header inclusion order, always include "config.h" before any system header
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2009-10-11 14:31:42 +00:00
Stefan Bühler 4f4d40f3f7 Add "lighty.req_env" table to mod_magnet for setting/getting environment values for cgi (fixes #1967, thx presbrey)
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2009-06-10 13:08:15 +00:00
Stefan Bühler 52861d77df Replace buffer_{append,copy}_string with the _len variant where possible (#1732, thx crypt)

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2008-07-30 19:38:32 +00:00
Marcus Rückert 8cd1471cb3 - white space cleanup part 2 this time 1.4 ;)
i hope it helps with merging stuff back to 1.5

git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2006-10-04 13:26:23 +00:00
Jan Kneschke dd0abe5aba print 'lighttpd -p -f ...' to stdout instead of stderr
git-svn-id: svn:// 152afb58-edef-0310-8abb-c4023f1b3aa9
2006-09-20 14:37:15 +00:00