Test::Nginx::Socket - Socket-backed test scaffold for the Nginx C modules and Nginx/OpenResty-based libraries and applications
use Test::Nginx::Socket; repeat_each(2); plan tests => repeat_each() * 3 * blocks(); no_shuffle(); run_tests(); __DATA__ === TEST 1: sanity --- config location /echo { echo_before_body hello; echo world; } --- request GET /echo --- response_body hello world --- error_code: 200 === TEST 2: set Server --- config location /foo { echo hi; more_set_headers 'Server: Foo'; } --- request GET /foo --- response_headers Server: Foo --- response_body hi === TEST 3: clear Server --- config location /foo { echo hi; more_clear_headers 'Server: '; } --- request GET /foo --- response_headers_like Server: nginx.* --- response_body hi === TEST 3: chunk size too small --- config chunkin on; location /main { echo_request_body; } --- more_headers Transfer-Encoding: chunked --- request eval "POST /main 4\r hello\r 0\r \r " --- error_code: 400 --- response_body_like: 400 Bad Request
This module provides a test scaffold based on non-blocking IO::Socket for automated testing in Nginx C module development.
This class inherits from Test::Base, thus bringing all its declarative power to the Nginx C module testing practices.
You need to terminate or kill any Nginx processes before running the test suite if you have changed the Nginx server binary. Normally it's as simple as
killall nginx PATH=/path/to/your/nginx-with-memc-module:$PATH prove -r t
This module will create a temporary server root under t/servroot/ of the current working directory and starts and uses the nginx executable in the PATH environment.
You will often want to look into t/servroot/logs/error.log when things go wrong ;)
You can find a comprehensive user guide on this test framework in my upcoming book "Programming OpenResty":
https://openresty.gitbooks.io/programming-openresty/content/testing/index.html
All the features of Test::Base are inherited since it is an ancestor class of this module anyway.
Still we would highlight some of the inherited features here for those unfamiliar with Test::Base.
--- ONLY
Runs the surrounding test block only. You need to remember removing --- ONLY before committing your changes though. Don't worry, the test scaffold would warn you loudly on the console when you left a --- ONLY in some test file.
It is also very intuitive for the developer's workflow. One does not have to specify a (unique) test name on the command-line; just find the block in the editor, insert a --- ONLY line right away, and run the current test file immediately (for Vim users, the final step is as simple as entering :!prove % where :! is the Vim way of running an external shell command and % would get substituted with the current file being edited in Vim's buffer).
:!prove %
:!
%
This is definitely one of the most useful and frequently used features.
--- SKIP
Skips the surrounding test block unconditionally. You can use --- skip_nginx and --- skip_nginx2 providied by this module (see their documentation below) to conditionally skip tests according to the current NGINX server versions. You can also use --- skip_openssl (see its documentation below) to conditionally skip tests according to the current OpenSSL version.
--- skip_nginx
--- skip_nginx2
--- skip_openssl
We can use filters to preprocess the values of our blocks, which can make specifying special values much easier.
For example, we could chop off the last new-line character (if any) of the current section value by specifying the chomp filter, like this:
chomp
--- response_body chomp Hello world!
Without the chomp filter, the value of the response_body section would take a trailing new line.
response_body
We list some of the common filters below (please keep in mind that one can define custom filters!)
Remove the last character if it is a newline.
chop
Remove the last character no matter what it is.
eval
Treat the section value as a Perl source code snippet, evaluate it right away, and use the returned value of the Perl code snippet (usually being the value of the last expression).
This is very useful for specifying non-printable characters in section values, as in
--- response_body eval "I don't know what \0 is.\n"
The following Perl functions are exported by default:
This is the main entry point of the test scaffold. Calling this Perl function before __DATA__ makes all the tests run. Other configuration Perl functions must be called before calling this run_tests function.
__DATA__
run_tests
By default, the test scaffold always shuffles the order of the test blocks automatically. Calling this function before calling run_tests will disable the shuffling.
Calling this function before calling run_tests will make the current test scaffold behave as if TEST_NGINX_USE_HUP was set to 1.
TEST_NGINX_USE_HUP
By default, failed string equality test will use the Test::LongString module to generate the error message. Calling this function before calling run_tests will turn this off.
When the no_long_string function is called, the Text::Diff module will be used to generate a diff for failed string equality test. Calling this no_diff function before calling run_tests will turn this diff output format off and just generate the raw "got" text and "expected" text.
no_long_string
Text::Diff
no_diff
Call this function before calling run_tests to set the Nginx's worker_connections configuration value. For example,
worker_connections
worker_connections(1024); run_tests();
Default to 64.
Call this function with an integer argument before run_tests() to ask the test scaffold to run the specified number of duplicate requests for each test block. When it is called without argument, it returns the current setting.
run_tests()
Default to 1.
You can use this section to check the error log generated during nginx exit.
For example,
--- shutdown_error_log cleanup resolver
or an example for using an array value,
--- shutdown_error_log eval ["cleanup", "resolver"]
WARNING: skip the shutdown_error_log tests under the HUP reload mode.
Very much like the --- shutdown_error_log section, but does the opposite test, i.e., pass only when the specified patterns of lines do not appear in the error.log file at all.
--- shutdown_error_log
Here is an example:
--- no_shutdown_error_log [error]
This test will fail when any of the line in the error.log file contains the string "[error]".
"[error]"
Specify additional system environmnt variables to be passed into the nginx server.
env_to_nginx("foo", "bar=123", "baz=hello world"); run_tests();
will result in the following lines to be inserted into the resulting nginx.conf file generated by the test scaffold:
env foo; env bar=123; env 'baz=hello world';
The latter two are examples of setting values directly to the environments. You can also set values directly on the Perl land, before calling this env_to_nginx function, for instance,
env_to_nginx
$ENV{baz} = 'hello world'; env_to_nginx("baz");
If you just want to pass certain environments to a particular test case (or test block), you can just use the --- main_config secion directly. For example,
--- main_config
--- main_config env foo; env bar=123;
You can check out nginx's official document on its env directive below:
env
http://nginx.org/r/env
By default, only the following environments are passed:
MOCKEAGAIN_VERBOSE
MOCKEAGAIN
MOCKEAGAIN_WRITE_TIMEOUT_PATTERN
LD_PRELOAD
LD_LIBRARY_PATH
DYLD_INSERT_LIBRARIES
DYLD_FORCE_FLAT_NAMESPACE
ASAN_OPTIONS
MOCKNOEAGAIN_VERBOSE
MOCKNOEAGAIN
Call this function before run_tests() to configure Nginx's worker_processes directive's value. For example,
worker_processes
workers(2);
Call this function before run_tests() to turn on the Nginx master process.
By default, the master process is not enabled unless in the "HUP reload" testing mode.
Call this function before run_tests() to set the default error log filtering level in Nginx.
This global setting can be overridden by the per-test-block --- log_level sections.
--- log_level
Default to debug.
debug
Make --- error_log and --- no_error_log check accumulated error log across duplicate requests controlled by repeat_each. By default, only the error logs belonging to the individual repeat_each request is tested.
--- error_log
--- no_error_log
repeat_each
By default, the Nginx configuration file generated by the test scaffold automatically emits a location /. Calling this function before run_tests() disables this behavior such that the test blocks can have their own root locations.
location /
Aborting the whole test session (not just the current test file) with a specified message.
This function will also do all the necessary cleanup work. So always use this function instead of calling Test::More::BAIL_OUT() directly.
Test::More::BAIL_OUT()
bail_out("something bad happened!");
Register custom cleanup handler for the current perl/prove process by specifying a Perl subroutine object as the argument.
add_cleanup_handler(sub { kill INT => $my_own_child_pid; $my_own_socket->close() });
Add a custom Perl preprocessor to each test block by specifying a Perl subroutine object as the argument.
The processor subroutine is always run right before processing the test block.
This mechanism can be used to add custom sections or modify existing ones.
add_block_preprocessor(sub { my $block = shift; # use "--- req_headers" for "--- more_Headers": $block->set_value("more_headers", $block->req_headers); # initialize external dependencies like memcached services here... });
We can leverage this feature to specify a default value for one or more sections in a single test file. For instance,
use Test::Nginx::Socket 'no_plan'; add_block_preprocessor(sub { my $block = shift; if (!defined $block->config) { $block->set_value("config", <<'_END_'); location = /t { echo $arg_a; } _END_ } }); run_tests(); __DATA__ === TEST 1: --- request GET /t?a=3 --- response_body 3 === TEST 2: --- request GET /t?a=blah --- response_body blah === TEST 3: --- config location = /t { echo ok; } --- request GET /t?a=blah --- response_body ok
Here all the test blocks in this file have a default --- config section configured. Some of the test blocks can still specify its own --- config section to override the default, as in the `TEST 3` test block above.
--- config
You can also make the defaults applicable to all the test files. Just create a subclass of Test::Nginx::Socket (or one of its subclasses like Test::Nginx::Socket::Lua, as in,
package t::MyTester; use Test::Nginx::Socket -Base; add_block_preprocessor(sub { my $block = shift; if (!defined $block->config) { $block->set_value("config", <<'_END_'); location = /t { echo $arg_a; } _END_ } }); 1;
Save this as file t/MyTester.pm. And then in one of your test file:
use t::MyTester 'no_plan'; run_tests(); __DATA__ === TEST 1: --- request GET /t?a=3 --- response_body 3 === TEST 2: --- request GET /t?a=blah --- response_body blah
You can do the same with the --- http_config section, or even inventing your own new sections. This is very powerful.
--- http_config
Add custom checks for testing response bodies by specifying a Perl subroutine object as the argument.
Below is an example for doing HTML title checks:
add_response_body_check(sub { my ($block, $body, $req_idx, $repeated_req_idx, $dry_run) = @_; my $name = $block->name; my $expected_title = $block->resp_title; if ($expected_title && !ref $expected_title) { $expected_title =~ s/^\s*|\s*$//gs; } if (defined $expected_title) { SKIP: { skip "$name - resp_title - tests skipped due to $dry_run", 1 if $dry_run; my $title; if ($body =~ m{<\s*title\s*>\s*(.*?)<\s*/\s*title\s*>}) { $title = $1; $title =~ s/\s*$//s; } is_str($title, $expected_title, "$name - resp_title (req $repeated_req_idx)" ); } } });
Performs intelligent string comparison subtests which honors both no_long_string and regular expression references in the "expected" argument.
The following sections are supported:
Enforces the test scaffold to use the HTTP/2 wire protocol to send the test request.
Under the hood, the test scaffold uses the `curl` command-line utility to do the wire communication with the NGINX server. The `curl` utility must be recent enough to support both the --http2 and --http2-prior-knowledge command-line options.
--http2
--http2-prior-knowledge
WARNING: not all the sections and features are supported when this --- http2 section is specified. For example, this section cannot be used with --- pipelined_requests or --- raw_request.
--- http2
--- pipelined_requests
--- raw_request
See also the TEST_NGINX_USE_HTTP2 system environment for the "http2" test mode.
Set protocol (such as http/https) when sending request using 'curl'.
Add extra command line options when using 'curl' to send request.
Below is an example for sending an insecure https request using 'curl':
--- http2 --- curl_options: -k --- curl_protocol: https --- request GET /ping
The expected curl error. --- curl_error
Content of this section will be included in the "server" part of the generated config file. This is the place where you want to put the "location" directive enabling the module you want to test. Example:
location /echo { echo_before_body hello; echo world; }
Sometimes you simply don't want to bother copying ten times the same configuration for the ten tests you want to run against your module. One way to do this is to write a config section only for the first test in your .t file. All subsequent tests will re-use the same config. Please note that this depends on the order of test, so you should run prove with variable TEST_NGINX_NO_SHUFFLE=1 (see below for more on this variable).
.t
prove
TEST_NGINX_NO_SHUFFLE=1
Please note that config section goes through environment variable expansion provided the variables to expand start with TEST_NGINX. So, the following is a perfectly legal (provided TEST_NGINX_HTML_DIR is set correctly):
TEST_NGINX_HTML_DIR
location /main { echo_subrequest POST /sub -f $TEST_NGINX_HTML_DIR/blah.txt; }
Content of this section will be included in the "http" part of the generated config file. This is the place where you want to put the "upstream" directive you might want to test. Example:
upstream database { postgres_server 127.0.0.1:$TEST_NGINX_POSTGRESQL_PORT dbname=ngx_test user=ngx_test password=wrong_pass; }
As you guessed from the example above, this section goes through environment variable expansion (variables have to start with TEST_NGINX).
Content of this section will be included in the "main" part (or toplevel) of the generated config file. This is very rarely used, except if you are testing nginx core itself. Everything in --- main_config will be put before the http {} block generated automatically by the test scaffold.
http {}
This section goes through environment variable expansion (variables have to start with TEST_NGINX).
Similar to main_config, but the content will be put after the http {} block generated by this module.
main_config
Specify a custom server name (via the "server_name" nginx config directive) for the current test block. Default to "localhost".
Run a piece of Perl code specified as the content of this --- init section before running the tests for the blocks. Note that it is only run once before *all* the repeated requests for this test block.
--- init
This is probably the most important section. It defines the request(s) you are going to send to the nginx server. It offers a pretty powerful grammar which we are going to walk through one example at a time.
In its most basic form, this section looks like that:
--- request GET
This will just do a GET request on the root (i.e. /) of the server using HTTP/1.1.
Of course, you might want to test something else than the root of your web server and even use a different version of HTTP. This is possible:
--- request GET /foo HTTP/1.0
Please note that specifying HTTP/1.0 will not prevent Test::Nginx from sending the Host header. Actually Test::Nginx always sends 2 headers: Host (with value localhost) and Connection (with value close for simple requests and keep-alive for all but the last pipelined_requests).
Host
Connection
close
You can also add a content to your request:
--- request POST /foo Hello world
Test::Nginx will automatically calculate the content length and add the corresponding header for you.
This being said, as soon as you want to POST real data, you will be interested in using the more_headers section and using the power of Test::Base filters to urlencode the content you are sending. Which gives us a slightly more realistic example:
--- more_headers Content-type: application/x-www-form-urlencoded --- request eval use URI::Escape; "POST /rrd/foo value=".uri_escape("N:12345")
Sometimes a test is more than one request. Typically you want to POST some data and make sure the data has been taken into account with a GET. You can do it using arrays:
--- request eval ["POST /users name=foo", "GET /users/foo"]
This way, REST-like interfaces are pretty easy to test.
When you develop nifty nginx modules you will eventually want to test things with buffers and "weird" network conditions. This is where you split your request into network packets:
--- request eval [["POST /users\nna", "me=foo"]]
Here, Test::Nginx will first send the request line, the headers it automatically added for you and the first two letters of the body ("na" in our example) in ONE network packet. Then, it will send the next packet (here it's "me=foo"). When we talk about packets here, this is not exactly correct as there is no way to guarantee the behavior of the TCP/IP stack. What Test::Nginx can guarantee is that this will result in two calls to syswrite.
syswrite
A good way to make almost sure the two calls result in two packets is to introduce a delay (let's say 2 seconds)before sending the second packet:
--- request eval [["POST /users\nna", {value => "me=foo", delay_before => 2}]]
Of course, everything can be combined till your brain starts boiling ;) :
--- request eval use URI::Escape; my $val="value=".uri_escape("N:12346"); [["POST /rrd/foo ".substr($val, 0, 6), {value => substr($val, 6, 5), delay_before=>5}, substr($val, 11)], "GET /rrd/foo"]
Adding comments before the actual request spec is also supported, for example,
--- request # this request contains the URI args # "foo" and "bar": GET /api?foo=1&bar=2
Use of this section is deprecated and tests using it should replace it with a request section with an eval filter. More explicitly:
request
--- request_eval "POST /echo_body hello\x00\x01\x02 world\x03\x04\xff"
should be replaced by:
--- request eval "POST /echo_body hello\x00\x01\x02 world\x03\x04\xff"
Specify pipelined requests that use a single keep-alive connection to the server.
Here is an example from ngx_lua's test suite:
=== TEST 7: discard body --- config location = /foo { content_by_lua ' ngx.req.discard_body() ngx.say("body: ", ngx.var.request_body) '; } location = /bar { content_by_lua ' ngx.req.read_body() ngx.say("body: ", ngx.var.request_body) '; } --- pipelined_requests eval ["POST /foo hello, world", "POST /bar hiya, world"] --- response_body eval ["body: nil\n", "body: hiya, world\n"]
Adds the content of this section as headers to the request being sent. Example:
--- more_headers X-Foo: blah
This will add X-Foo: blah to the request (on top of the automatically generated headers like Host, Connection and potentially Content-Length).
X-Foo: blah
Content-Length
When this section is specified, the test scaffold will try generating a curl command line for the (first) test request.
curl
--- request GET /foo/bar?baz=3 --- more_headers X-Foo: 3 User-Agent: openresty --- curl
will produce the following line (to stderr) while running this test block:
stderr
# curl -i -sS -H 'X-Foo: 3' -A openresty 'http://127.0.0.1:1984/foo/bar?baz=3'
You need to remember to set the TEST_NGINX_NO_CLEAN environment to 1 to prevent the nginx and other processes from quitting automatically upon test exits.
TEST_NGINX_NO_CLEAN
Transforms the value of the actual response body data through a series of filters, before being matched against the expected response body data specified by the response_body or response_body_like sections.
response_body_like
The filters can be specified either as names (for builtin filters) or as arbitrary Perl subroutine references.
The following builtin filter names are supported:
Their meanings are self-explanatory.
=== TEST 1: --- config location = /t { echo hello; } --- request GET /t --- response_body_filters uc --- response_body HELLO
Here the actual response body data, hello, will go through the fitler, uc, to become all-upper-case, before getting matched against the expected pattern specified by the response_body section, HELLO.
hello
uc
HELLO
The example above can be rewritten by using raw Perl subroutine reference values:
=== TEST 1: --- config location = /t { echo hello; } --- request GET /t --- response_body_filters eval \&CORE::uc --- response_body HELLO
To reference builtin Perl functions like \&CORE::uc and \&CORE::lc, you need at least perl 5.16.
\&CORE::uc
\&CORE::lc
Multiple builtin filter names can be specified at the same time and they will be applied in order. For example,
=== TEST 2: --- config location = /t { echo hello; } --- request GET /hello --- response_body_filters uc lc --- response_body hello
If the response_body_filters value can also be an array reference, mostly useful for specifying multiple Perl subroutine references as the filters:
=== TEST 3: --- config location = /t { echo hello; } --- request GET /hello --- response_body_filters eval [\&CORE::uc, \&CORE::lc] --- response_body hello
If the response_body_filters value can also be an two-dimensional array reference, it means the actual response body data will be isolatedly applied by the indexed array's filters:
isolatedly
=== TEST 4: --- config location = /t { echo hello; } --- request eval ['GET /t', 'GET /t'] --- response_body_filters eval [[\&CORE::uc, \&CORE::lc], [\&CORE::uc]] --- response_body eval ['hello', 'HELLO']
The expected value for the body of the submitted request.
--- response_body hello
If the test is made of multiple requests, then the response_body MUST be an array and each request MUST return the corresponding expected body:
--- request eval ["GET /hello", "GET /world"] --- response_body eval ["hello", "world"]
Use of this section is deprecated and tests using it should replace it with a request section with an eval filter. Therefore:
--- response_body_eval "hello\x00\x01\x02 world\x03\x04\xff"
--- response_body eval "hello\x00\x01\x02 world\x03\x04\xff"
The body returned by the request MUST match the pattern provided by this section. Example:
--- response_body_like ^elapsed 0\.00[0-5] sec\.$
If the test is made of multiple requests, then response_body_like MUST be an array and each request MUST match the corresponding pattern.
Just like response_body_like but this test only pass when the specified pattern does not match the actual response body data.
The headers specified in this section are in the response sent by nginx.
--- response_headers Content-Type: application/x-resty-dbd-stream
Of course, you can specify many headers in this section:
--- response_headers X-Resty-DBD-Module: Content-Type: application/x-resty-dbd-stream
The test will be successful only if all headers are found in the response with the appropriate values.
If the test is made of multiple requests, then response_headers MUST be an array and each element of the array is checked against the response to the corresponding request.
The value of the headers returned by nginx match the patterns.
--- response_headers_like X-Resty-DBD-Module: ngx_drizzle \d+\.\d+\.\d+ Content-Type: application/x-resty-dbd-stream
This will check that the response's Content-Type is application/x-resty-dbd-stream and that the X-Resty-DBD-Module matches ngx_drizzle \d+\.\d+\.\d+.
Content-Type
X-Resty-DBD-Module
ngx_drizzle \d+\.\d+\.\d+
The test will be successful only if all headers are found in the response and if the values match the patterns.
If the test is made of multiple requests, then response_headers_like MUST be an array and each element of the array is checked against the response to the corresponding request.
Checks the headers part of the response against this pattern. This is particularly useful when you want to write tests of redirect functions that are not bound to the value of the port your nginx server (under test) is listening to:
--- raw_response_headers_like: Location: http://localhost(?::\d+)?/foo\r\n
As usual, if the test is made of multiple requests, then raw_response_headers_like MUST be an array.
Just like raw_response_headers_like but the subtest only passes when the regex does not match the raw response headers string.
raw_response_headers_like
The expected value of the HTTP response code. If not set, this is assumed to be 200. But you can expect other things such as a redirect:
--- error_code: 302
If the test is made of multiple requests, then error_code MUST be an array with the expected value for the response status of each request in the test.
Just like error_code, but accepts a Perl regex as the value, for example:
error_code
--- error_code_like: ^(?:500)?$
If the test is made of multiple requests, then error_code_like MUST be an array with the expected value for the response status of each request in the test.
Specify the timeout value (in seconds) for the HTTP client embedded into the test scaffold. This has nothing to do with the server side configuration. When the timeout expires, the test scaffold will immediately close the socket for connecting to the Nginx server being tested.
Note that, just as almost all the timeout settings in the Nginx world, this timeout also specifies the maximum waiting time between two successive I/O events on the same socket handle, rather than the total waiting time for the current socket operation.
When the timeout setting expires, a test failure will be triggered with the message "ERROR: client socket timed out - TEST NAME", unless you have specified --- abort at the same time.
--- abort
=== TEST 1: test timeout --- location location = /t { echo_sleep 1; echo ok; } --- request GET /t --- response_body ok --- timeout: 1.5
An optional time unit can be specified, for example,
--- timeout: 50ms
Acceptable time units are s (seconds) and ms (milliseconds). If no time unit is specified, then default to seconds.
s
ms
Default to 3s.
Specify the global error log file for the current test block only.
Right now, it will not affect the --- error_log section and etc accordingly.
Checks if the pattern or multiple patterns all appear in lines of the error.log file.
=== TEST 1: matched with j --- config location /re { content_by_lua ' m = ngx.re.match("hello, 1234", "([0-9]+)", "j") if m then ngx.say(m[0]) else ngx.say("not matched!") end '; } --- request GET /re --- response_body 1234 --- error_log: pcre JIT compiling result: 1
Then the substring "pcre JIT compiling result: 1" must appear literally in a line of error.log.
Multiple patterns are also supported, for example:
--- error_log eval ["abc", qr/blah/]
then the substring "abc" must appear literally in a line of error.log, and the regex qr/blah must also match a line in error.log.
qr/blah
By default, only the part of the error logs corresponding to the current request is checked. You can make it check accumulated error logs by calling the check_accum_error_log Perl function before calling run_tests in the boilerplate Perl code above the __DATA__ line.
check_accum_error_log
Similar to the error_log section, but for asserting appearance of patterns in the nginx access log file.
Below is an example:
=== TEST 1: check access log --- config location /t { content_by_lua_block { ngx.say("hello") } } --- request GET /t --- response_body hello --- access_log GET /t
Similar to the no_error_log section, but for asserting appearance of patterns in the nginx access log file.
=== TEST 1: check access log --- config location /t { content_by_lua_block { ngx.say("hello") } } --- request GET /t --- response_body hello --- no_access_log GET /p
Makes the test scaffold not to treat --- timeout expiration as a test failure.
--- timeout
Perform a shutdown() operation on the client socket connecting to Nginx as soon as sending out all the request data. This section takes an (optional) integer value for the argument to the shutdown function call. For example,
shutdown
--- shutdown: 1
will make the connection stop sending data, which is the default.
Very much like the --- error_log section, but does the opposite test, i.e., pass only when the specified patterns of lines do not appear in the error.log file at all.
--- no_error_log [error]
Just like the --- error_log section, one can also specify multiple patterns:
--- no_error_log eval ["abc", qr/blah/]
Then if any line in error.log contains the string "abc" or match the Perl regex qr/blah/, then the test will fail.
"abc"
qr/blah/
This section specifies the Perl regex pattern for filtering out the Nginx error logs.
You can specify a verbatim substring being matched in the error log messages, as in
--- grep_error_log chop some thing we want to see
or specify a Perl regex object to match against the error log message lines, as in
--- grep_error_log eval qr/something should be: \d+/
All the matched substrings in the error log messages will be concatenated by a newline character as a whole to be compared with the value of the --- grep_error_log_out section.
--- grep_error_log_out
This section contains the expected output for the filtering operations specified by the --- grep_error_log section.
--- grep_error_log
If the filtered output varies among the repeated requests (specified by the repeat_each function, then you can specify a Perl array as the value, as in
--- grep_error_log_out eval ["output for req 0", "output for req 1"]
Overrides the default error log level for the current test block.
For example:
--- log_level: debug
The default error log level can be specified in the Perl code by calling the log_level() function, as in
log_level()
use Test::Nginx::Socket; repeat_each(2); plan tests => repeat_each() * (3 * blocks()); log_level('warn'); run_tests(); __DATA__ ...
The exact request to send to nginx. This is useful when you want to test some behaviors that are not available with "request" such as an erroneous Content-Length header or splitting packets right in the middle of headers:
--- raw_request eval ["POST /rrd/taratata HTTP/1.1\r Host: localhost\r Connection: close\r Content-Type: application/", "x-www-form-urlencoded\r Content-Length:15\r\n\r\nvalue=N%3A12345"]
This can also be useful to tests "invalid" request lines:
--- raw_request GET /foo HTTP/2.0 THE_FUTURE_IS_NOW
Specifies that the HTTP 0.9 protocol is used. This affects how Test::Nginx::Socket parses the response.
Test::Nginx::Socket
Below is an example from ngx_headers_more module's test suite:
=== TEST 38: HTTP 0.9 (set) --- config location /foo { more_set_input_headers 'X-Foo: howdy'; echo "x-foo: $http_x_foo"; } --- raw_request eval "GET /foo\r\n" --- response_headers ! X-Foo --- response_body x-foo: --- http09
Do not attempt to parse the response or run the response related subtests.
Allows the NGINX HUP reload fails, which means that the server will still use the previous test block's nginx configuration.
This only makes sense in the HUP reload testing mode.
With this section you can create a file that will be copied in the html directory of the nginx server under test. For example:
--- user_files >>> blah.txt Hello, world
will create a file named blah.txt in the html directory of the nginx server tested. The file will contain the text "Hello, world".
blah.txt
Multiple files are supported, for example,
--- user_files >>> foo.txt Hello, world! >>> bar.txt Hello, heaven!
An optional last modified timestamp (in elpased seconds since Epoch) is supported, for example,
--- user_files >>> blah.txt 199801171935.33 Hello, world
It's also possible to specify a Perl data structure for the user files to be created, for example,
--- user_files eval [ [ "foo.txt" => "Hello, world!", 199801171935.33 ], [ "bar.txt" => "Hello, heaven!" ], ]
Skip the specified number of subtests (in the current test block) if the result of running a piece of Perl code is true.
The format for this section is
--- skip_eval <subtest-count>: <perl-code>
For example, to skip 3 subtests when the current operating system is not Linux:
--- skip_eval 3: $^O ne 'linux'
or equivalently,
--- skip_eval: 3: $^O ne 'linux'
Skip the specified number of subtests (in the current test block) for the specified version range of nginx.
--- skip_nginx <subtest-count>: <op> <version>
The <subtest-count> value must be a positive integer. The <op> value could be either >, >=, <, or <=. the <version> part is a valid nginx version number, like 1.0.2.
>
>=
<
<=
1.0.2
An example is
=== TEST 1: sample --- config location /t { echo hello; } --- request GET /t --- response_body --- skip_nginx 2: < 0.8.54
That is, skipping 2 subtests in this test block for nginx versions older than 0.8.54.
This skip_nginx section only allows you to specify one boolean expression as the skip condition. If you want to use two boolean expressions, you should use the skip_nginx2 section instead.
skip_nginx
skip_nginx2
This section is similar to skip_nginx, but the skip condition consists of two boolean expressions joined by the operator and or or.
and
or
--- skip_nginx2 <subtest-count>: <op> <version> and|or <op> <version>
=== TEST 1: sample --- config location /t { echo hello; } --- request GET /t --- response_body --- skip_nginx2 2: < 0.8.53 and >= 0.8.41
Skip the specified number of subtests (in the current test block) for the specified version range of OpenSSL.
--- skip_openssl <subtest-count>: <op> <version>
The <subtest-count> value must be a positive integer. The <op> value could be either >, >=, <, or <=. The <version> part is a valid OpenSSL version number, like 1.1.1 or 1.1.0h.
1.1.1
1.1.0h
=== TEST 1: sample --- config location /t { echo hello; } --- request GET /t --- response_body --- skip_openssl 2: < 1.1.1
That is, skipping 2 subtests in this test block for OpenSSL versions older than 1.1.1.
This skip_openssl section only allows you to specify one boolean expression as the skip condition.
skip_openssl
Mark tests as todo. Currently they are not used but they should be.
--- todo <subtest-count>: <reason>
The <subtest-count> value must be a positive integer.
<reason> is logged when you run tests with --directives.
This section is used to specify user systemtap script file (.stp file)
Here's an example:
=== TEST 1: stap sample --- config location /t { echo hello; } --- stap probe process("nginx").function("ngx_http_finalize_request") { printf("finalize %s?%s\n", ngx_http_req_uri($r), ngx_http_req_args($r)) } --- stap_out finalize /test?a=3&b=4 --- request GET /test?a=3&b=4 --- response_body hello
There's some macros that can be used in the "--- stap" section value. These macros will be expanded by the test scaffold automatically.
F(function_name)
This expands to probe process("nginx").function("function_name"). For example, the sample above can be rewritten as
probe process("nginx").function("function_name")
=== TEST 1: stap sample --- config location /t { echo hello; } --- stap F(ngx_http_finalize_request) { printf("finalize %s?%s\n", ngx_http_req_uri($r), ngx_http_req_args($r)) } --- stap_out finalize /test?a=3&b=4 --- request GET /test?a=3&b=4 --- response_body hello
T()
This macro will be expanded to println("Fire ", pp()).
println("Fire ", pp())
M(static-probe-name)
This macro will be expanded to probe process("nginx").mark("static-probe-name").
probe process("nginx").mark("static-probe-name")
M(http-subrequest-start) { ... }
will be expanded to
probe process("nginx").mark("http-subrequest-start") { ... }
This section specifies the expected literal output of the systemtap script specified by stap.
stap
Just like stap_out, but specify a Perl regex pattern instead.
stap_out
Just like stap_like, but the subtest only passes when the specified pattern does not match the output of the systemtap script.
stap_like
Takes an integer value for the seconds of time to wait right after processing the Nginx response and before performing the error log and/or systemtap output checks.
Instantiates a UDP server listening on the port specified in the background for the test case to access. The server will be started and shut down at each iteration of the test case (if repeat_each is set to 3, then there are 3 iterations).
The UDP server will first read and discard a datagram and then send back a datagram with the content specified by the udp_reply section value.
udp_reply
=== TEST 1: udp access --- config location = /t { content_by_lua ' local udp = ngx.socket.udp() udp:setpeername("127.0.0.1", 19232) udp:send("blah") local data, err = udp:receive() ngx.say("received: ", data) '; } --- udp_listen: 19232 --- udp_reply: hello world --- request GET /t --- response_body received: hello world
Datagram UNIX domain socket is also supported if a path name ending with ".sock" is given to this directive. For instance,
=== TEST 2: datagram unix domain socket access --- config location = /t { content_by_lua ' local udp = ngx.socket.udp() udp:setpeername("unix:a.sock") udp:send("blah") local data, err = udp:receive() ngx.say("received: ", data) '; } --- udp_listen: a.sock --- udp_reply: hello world --- request GET /t --- response_body received: hello world
This section specifies the datagram reply content for the UDP server created by the udp_listen section.
udp_listen
You can also specify a delay time before sending out the reply via the udp_reply_delay section. By default, there is no delay.
udp_reply_delay
An array value can be specified to make the embedded UDP server to send multiple replies as specified, for example:
--- udp_reply eval [ "hello", "world" ]
This section also accepts a Perl subroutine value that can be used to generate dynamic response packet or packets based on the actual query, for example:
--- udp_reply eval sub { my $req = shift; return "hello, $req"; }
The custom Perl subroutine can also return an array reference, for example,
--- udp_reply eval sub { my $req = shift; return ["hello, $req", "hiya, $req"]; }
See the udp_listen section for more details.
This section specifies the delay time before sending out the reply specified by the udp_reply section.
It is 0 delay by default.
0
--- udp_reply_delay: 50ms
Tests whether the UDP query sent to the embedded UDP server is equal to what is specified by this directive.
=== TEST 1: udp access --- config location = /t { content_by_lua ' local udp = ngx.socket.udp() udp:setpeername("127.0.0.1", 19232) udp:send("blah") local data, err = udp:receive() ngx.say("received: ", data) '; } --- udp_listen: 19232 --- udp_reply: hello world --- request GET /t --- udp_query: hello world --- response_body received: hello world
Just like udp_listen, but starts an embedded TCP server listening on the port specified. For example,
--- tcp_listen: 12345
Stream-typed unix domain socket is also supported. Just specify the path to the socket file, as in
--- tcp_listen: /tmp/my-socket.sock
When this section is present, the embedded TCP server (if any) will not close the current TCP connection.
Just like udp_reply_delay, but for the embedded TCP server.
Just like udp_reply, but for the embedded TCP server.
Like the udp_reply section, this section also accepts a Perl subroutine value that can be used to generate dynamic response packet or packets based on the actual query, for example:
--- tcp_reply eval sub { my $req = shift; return "hello, $req"; }
--- tcp_reply eval sub { my $req = shift; return ["hello, $req", "hiya, $req"]; }
Just like udp_query, but for the embedded TCP server.
udp_query
Specifies the expected TCP query received by the embedded TCP server.
If tcp_query is specified, tcp_query_len defaults to the length of the value of tcp_query.
tcp_query
tcp_query_len
Shuts down the reading part, writing part, or both in the embedded TCP server as soon as a new connection is established. Its value specifies which part to shut down: 0 for read part only, 1 for write part only, and 2 for both directions.
Delay in sec between sending successive packets in the "raw_request" array value. Also used when a request is split in packets.
Skip the tests in the current test block in the "check leak" testing mode (i.e, with TEST_NGINX_CHECK_LEAK=1).
TEST_NGINX_CHECK_LEAK
Test the cases that Nginx must die right after starting. If a value is specified, the exit code must match the specified value.
Normal request and response cycle is not done. But you can still use the error_log section to check if there is an error message to be seen.
error_log
This is meant to test bogus configuration is noticed and given proper error message. It is normal to see stderr error message when running these tests.
=== TEST 1: bad "return" directive --- config location = /t { return a b c; } --- request GET /t --- must_die --- error_log invalid number of arguments in "return" directive --- no_error_log [error]
This configuration ignores TEST_NGINX_USE_VALGRIND TEST_NGINX_USE_STAP or TEST_NGINX_CHECK_LEAK since there is no point to check other things when the nginx is expected to die right away.
TEST_NGINX_USE_VALGRIND
TEST_NGINX_USE_STAP
This directive is handled before checking TEST_NGINX_IGNORE_MISSING_DIRECTIVES.
TEST_NGINX_IGNORE_MISSING_DIRECTIVES
This section specifies the server address Test::Nginx will connect to. If server_addr_for_client is not set, then 127.0.0.1 is used.
All environment variables starting with TEST_NGINX_ are expanded in the sections used to build the configuration of the server that tests automatically starts. The following environment variables are supported by this module:
TEST_NGINX_
When this environment is set to a true value, the test scaffold would add the "resuseport" parameter to the "listen" directive automatically generated in nginx.conf.
Enables the "http2" test mode by enforcing using the (plain text) HTTP/2 protocol to send the test request.
WARNING: not all the sections and features are supported in the "http2" test mode. For example, the pipelined_requests and raw_request will still use the HTTP/1 protocols even in the "http2" test mode. Similarly, test blocks explicitly require the HTTP 1.0 protocol will still use HTTP 1.0.
One can enable HTTP/2 mode for an individual test block by specifying the http2 section, as in
One can disable HTTP/2 mode for an individual test block by specifying the no_http2 section, as in
--- no_http2
Enables the "http3" test mode by enforcing using the HTTP/3 protocol to send the test request.
Under the hood, the test scaffold uses the `curl` command-line utility to do the wire communication with the NGINX server. The `curl` utility must be recent enough to support both the --http3 command-line options.
--http3
WARNING: not all the sections and features are supported in the "http3" test mode. For example, the pipelined_requests and raw_request will still use the HTTP/1 protocols even in the "http3" test mode. Similarly, test blocks explicitly require the HTTP 1.0 protocol will still use HTTP 1.0.
One can enable HTTP/3 mode for an individual test block by specifying the http3 section, as in
--- http3
One can disable HTTP/3 mode for an individual test block by specifying the no_http3 section, as in
--- no_http3
When running in http3 mode, you need to specify the default certificate.
When running in http3 mode, you need to specify the default key.
HTTP3 connections are not closed when the requests finished. When reload nginx, the older nginx will not exit unitl the older connections idle timeout reach. The default idle timeout is 60 seconds which is too long for the test scaffold.
Change the idle timeout value by environment var TEST_NGINX_QUIC_IDLE_TIMEOUT. Default idle timeout value is 0.6s if not set.
export TEST_NGINX_QUIC_IDLE_TIMEOUT=0.1
Controls whether to output verbose debugging messages in Test::Nginx. Default to empty.
When set to an non-empty and non-zero value, then the test scaffold enters the benchmarking testing mode by invoking weighttp (for HTTP 1.1 requests) and ab (for HTTP 1.0 requests) to run each test case with the test request repeatedly.
weighttp
ab
When specifying a positive number as the value, then this number is used for the total number of repeated requests. For example,
export TEST_NGINX_BENCHMARK=1000
will result in 1000 repeated requests for each test block. Default to 100000.
100000
When a second number is specified (separated from the first number by spaces), then this second number is used for the concurrency level for the benchmark. For example,
export TEST_NGINX_BENCHMARK='1000 10'
will result in 1000 repeated requests over 10 concurrent connections for each test block. The default concurrency level is 2 (or 1 if the number of requests is 1).
The "benchmark" testing mode will also output to stderr the actual "ab" or "weighttp" command line used by the test scaffold. For example,
weighttp -c2 -k -n2000 -H 'Host: foo.com' http://127.0.0.1:1984/t
See also the TEST_NGINX_BENCHMARK_WARMUP environment.
TEST_NGINX_BENCHMARK_WARMUP
This testing mode requires the unbuffer command-line utility from the expect package.
unbuffer
expect
Specify the number of "warm-up" requests performed before the actual benchmark requests for each test block.
The latencies of the warm-up requests never get included in the final benchmark results.
Only meaningful in the "benchmark" testing mode.
See also the TEST_NGINX_BENCHMARK environment.
TEST_NGINX_BENCHMARK
When set to 1, the test scaffold performs the most general memory leak test by means of calling weighttpd/ab and ps.
weighttpd
ps
Specifically, it starts weighttp (for HTTP 1.1 GET requests) or ab (for HTTP 1.0 requests) to repeatedly hitting Nginx for seconds in a sub-process, and then after about 1 second, it will start sampling the RSS value of the Nginx process by calling the ps utility every 20 ms. Finally, it will output all the sample point data and the line slope of the linear regression result on the 100 sample points.
GET
One typical output for non-leaking test cases:
t/075-logby.t .. 3/17 TEST 2: log_by_lua_file LeakTest: [2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176 2176] LeakTest: k=0.0
and here is an example of leaking:
TEST 5: ngx.ctx available in log_by_lua (not defined yet) LeakTest: [4396 4440 4476 4564 4620 4708 4752 4788 4884 4944 4996 5032 5080 5132 5188 5236 5348 5404 5464 5524 5596 5652 5700 5776 5828 5912 5964 6040 6108 6108 6316 6316 6584 6672 6672 6752 6820 6912 6912 6980 7064 7152 7152 7240 7340 7340 7432 7508 7508 7600 7700 7700 7792 7896 7896 7992 7992 8100 8100 8204 8296 8296 8416 8416 8512 8512 8624 8624 8744 8744 8848 8848 8968 8968 9084 9084 9204 9204 9324 9324 9444 9444 9584 9584 9704 9704 9832 9832 9864 9964 9964 10096 10096 10488 10488 10488 10488 10488 11052 11052] LeakTest: k=64.1
Even very small leaks can be amplified and caught easily by this testing mode because their slopes will usually be far above 1.0.
1.0
One can configure the number of sample points via the TEST_NGINX_CHECK_LEAK_COUNT system environment, for example, to sample 1000 data points, we can set the following environment before running the test:
export TEST_NGINX_CHECK_LEAK_COUNT=1000
For now, only GET, POST, PUT, and HEAD requests are supported (due to the limited HTTP support in both ab and weighttp). Other methods specified in the test cases will turn to GET with force.
POST
PUT
HEAD
The tests in this mode will always succeed because this mode also enforces the "dry-run" mode.
Test blocks carrying the "--- no_check_leak" directive will be skipped in this testing mode.
Takes a number value which controls how many data points to be sampled in the "check leak" test mode. See TEST_NGINX_CHECK_LEAK for more details.
Defaults to 100.
When set to 1, the test scaffold will try to send HUP signal to the Nginx master process to reload the config file between successive test blocks (but not successive repeat_each sub-tests within the same test block). When this environment is set to 1, it will also enforce the "master_process on" config line in the nginx.conf file, because Nginx is buggy in processing HUP signal when the master process is off.
HUP
This environment takes a list of dynamic NGINX module files' paths. The test scaffold generates a series of load_module directives in the top-level scope of nginx.conf.
load_module
For example, when this environment takes the value ../../work/nginx/modules/ngx_http_headers_more_filter_module.so ../../work/nginx/modules/ngx_http_lua_module.so , then the following snippet will be generated in nginx.conf:
../../work/nginx/modules/ngx_http_headers_more_filter_module.so ../../work/nginx/modules/ngx_http_lua_module.so
load_module ../../work/nginx/modules/ngx_http_headers_more_filter_module.so; load_module ../../work/nginx/modules/ngx_http_lua_module.so;
This requires at least NGINX 1.9.11 to work.
Defaults to empty. This environment takes positive integer numbers as its value and it will cause the auto-generated nginx.conf file to have a "postpone_output" setting in the http {} block.
For example, setting TEST_NGINX_POSTPONE_OUTPUT to 1 will have the following line in nginx.conf's http {} block:
postpone_output 1;
and it will effectively disable the write buffering in nginx's ngx_http_write_module.
When this environment is set to 1, it will prevent the test scaffold from quitting the Nginx server at the end of the run. This is very useful when you want to use other tools like gdb or curl inspect the Nginx server manually afterwards.
Defaults to 0. If set to 1, Test::Nginx module will not manage (configure/start/stop) the nginx process. Can be useful to run tests against an already configured (and running) nginx server.
nginx
Defaults to 0. If set to 1, Test::Nginx module will stop nginx process with SIGTERM.
Defaults to 0. If set to 1, will make sure the tests are run in the order they appear in the test file (and not in random order).
If set, Test::Nginx will start nginx with valgrind with the the value of this environment as the options.
Nginx is actually started with valgrind -q $TEST_NGINX_USE_VALGRIND --gen-suppressions=all --suppressions=valgrind.suppress, the suppressions option being used only if there is actually a valgrind.suppress file.
valgrind -q $TEST_NGINX_USE_VALGRIND --gen-suppressions=all --suppressions=valgrind.suppress
If this environment is set to the number 1 or any other non-zero numbers, then it is equivalent to taking the value --tool=memcheck --leak-check=full.
1
--tool=memcheck --leak-check=full
Uses Mozilla rr to record the execution of the nginx server run by the test scaffold.
This feature is experimental.
When set to true values (like 1), the test scaffold will use systemtap to instrument the nginx process.
You can specify the stap script in the stap section.
Note that you need to use the stap-nginx script from the nginx-dtrace project.
stap-nginx
nginx-dtrace
You can specify the output file for the systemtap tool. By default, a random file name under the system temporary directory is generated.
It's common to specify TEST_NGINX_STAP_OUT=/dev/stderr when debugging.
TEST_NGINX_STAP_OUT=/dev/stderr
The command to start nginx. Defaults to nginx. Can be used as an alternative to setting PATH to run a specific nginx instance.
PATH
Value of the last argument of the error_log configuration directive. Defaults to debug.
Value of the master_process configuration directive. Defaults to off.
master_process
off
Value of the port the server started by Test::Nginx will listen to. If not set, TEST_NGINX_PORT is used. If TEST_NGINX_PORT is not set, then 1984 is used. See below for typical use.
TEST_NGINX_PORT
1984
Value of the port Test::Nginx will direct requests to. If not set, TEST_NGINX_PORT is used. If TEST_NGINX_PORT is not set, then 1984 is used. A typical use of this feature is to test extreme network conditions by adding a "proxy" between Test::Nginx and nginx itself. This is described in the etcproxy integration section of this module README.
etcproxy integration
A shortcut for setting both TEST_NGINX_CLIENT_PORT and TEST_NGINX_SERVER_PORT.
TEST_NGINX_CLIENT_PORT
TEST_NGINX_SERVER_PORT
How much time (in seconds) should Test::Nginx sleep between two calls to syswrite when sending request data. Defaults to 0.015 (seconds).
Defaults to 1. If set to 0, Test::Nginx will not restart the nginx server when the config does not change between two tests.
The root of the nginx "hierarchy" (where you find the conf, *_tmp and logs directories). This value will be used with the -p option of nginx. Defaults to appending t/servroot to the current directory.
-p
t/servroot
If set to 1 will SKIP all tests which config sections resulted in a unknown directive when trying to start nginx. Useful when you want to run tests on a build of nginx that does not include all modules it should. By default, these tests will FAIL.
config
unknown directive
This environment can be used to specify a event API type to be used by Nginx. Possible values are epoll, kqueue, select, rtsig, poll, and others.
epoll
kqueue
select
rtsig
poll
$ TEST_NGINX_EVENT_TYPE=select prove -r t
Error log files from all tests will be appended to the file specified with this variable. There is no default value which disables the feature. This is very useful when debugging. By default, each test triggers a start/stop cycle for nginx. All logs are removed before each restart, so you can only see the logs for the last test run (which you usually do not control except if you set TEST_NGINX_NO_SHUFFLE=1). With this, you accumulate all logs into a single file that is never cleaned up by Test::Nginx.
When set, the test scaffold forces the use of random server listening port numbers as well as random t/servroot_XXXX/ directories. This can help test suite run in multiple parallel jobs via prove -jN where N is an integer bigger than 1. For instance, prove -j8 -r t runs the test suite under t/ in 8 parallel jobs, utilizing up to 8 (logical) CPU cores in the same machine.
t/servroot_XXXX/
prove -jN
N
prove -j8 -r t
Note that only test suite without external shared and writable service dependencies (like Memcached, Redis or MySQL) can run in parallel in this way, obviously.
Sets the user account used to run the nginx worker processes when the master process is enabled. This requires root access to run the nginx master process. For instance,
export TEST_NGINX_WORKER_USER='agentzh'
Defaults to the `root` when the master is run by `root` also. Otherwise defaults to the current user.
One can also add an optional user group separated by spaces, as in
export TEST_NGINX_WORKER_USER='agentzh wheel'
Variables like TEST_NGINX_RAND_PORT_XXX are expanded to random and unused unprivileged ports numbers to build the configuration of the server, where XXX is an integer from 1985 to 65535.
TEST_NGINX_RAND_PORT_XXX
XXX
For instance, TEST_NGINX_RAND_PORT_1 will be expanded to an port number, such as 1986, and TEST_NGINX_RAND_PORT_2 will be expanded to another port number, such as 65535.
TEST_NGINX_RAND_PORT_1
TEST_NGINX_RAND_PORT_2
Test::Nginx has integrated support for valgrind (http://valgrind.org) even though by default it does not bother running it with the tests because valgrind will significantly slow down the test suite.
First ensure that your valgrind executable visible in your PATH env. And then run your test suite with the TEST_NGINX_USE_VALGRIND env set to true:
TEST_NGINX_USE_VALGRIND=1 prove -r t
If you see false alarms, you do have a chance to skip them by defining a ./valgrind.suppress file at the root of your module source tree, as in
https://github.com/chaoslawful/drizzle-nginx-module/blob/master/valgrind.suppress
This is the suppression file for ngx_drizzle. Test::Nginx will automatically use it to start nginx with valgrind memcheck if this file does exist at the expected location.
If you do see a lot of "Connection refused" errors while running the tests this way, then you probably have a slow machine (or a very busy one) that the default waiting time is not sufficient for valgrind to start. You can define the sleep time to a larger value by setting the TEST_NGINX_SLEEP env:
TEST_NGINX_SLEEP
TEST_NGINX_SLEEP=1 prove -r t
The time unit used here is "second". The default sleep setting just fits my ThinkPad (Core2Duo T9600).
Core2Duo T9600
Applying the no-pool patch to your nginx core is recommended while running nginx with valgrind:
https://github.com/shrimp/no-pool-nginx
The nginx memory pool can prevent valgrind from spotting lots of invalid memory reads/writes as well as certain double-free errors. We did find a lot more memory issues in many of our modules when we first introduced the no-pool patch in practice ;)
There's also more advanced features in Test::Nginx that have never documented. I'd like to write more about them in the near future ;)
WARNING: use etcproxy is no longer recommended because the mockeagain is way more effective and efficient:
https://github.com/openresty/mockeagain
The default settings in etcproxy (https://github.com/chaoslawful/etcproxy) makes this small TCP proxy split the TCP packets into bytes and introduce 1 ms latency among them.
There's usually various TCP chains that we can put etcproxy into, for example
$ ./etcproxy 1234 1984
Here we tell etcproxy to listen on port 1234 and to delegate all the TCP traffic to the port 1984, the default port that Test::Nginx makes nginx listen to.
And then we tell Test::Nginx to test against the port 1234, where etcproxy listens on, rather than the port 1984 that nginx directly listens on:
$ TEST_NGINX_CLIENT_PORT=1234 prove -r t/
Then the TCP chain now looks like this:
Test::Nginx <=> etcproxy (1234) <=> nginx (1984)
So etcproxy can effectively emulate extreme network conditions and exercise "unusual" code paths in your nginx server by your tests.
In practice, *tons* of weird bugs can be captured by this setting. Even ourselves didn't expect that this simple approach is so effective.
We first start the memcached server daemon on port 11211:
memcached -p 11211 -vv
and then we another etcproxy instance to listen on port 11984 like this
$ ./etcproxy 11984 11211
Then we tell our t/foo.t test script to connect to 11984 rather than 11211:
# foo.t use Test::Nginx::Socket; repeat_each(1); plan tests => 2 * repeat_each() * blocks(); $ENV{TEST_NGINX_MEMCACHED_PORT} ||= 11211; # make this env take a default value run_tests(); __DATA__ === TEST 1: sanity --- config location /foo { set $memc_cmd set; set $memc_key foo; set $memc_value bar; memc_pass 127.0.0.1:$TEST_NGINX_MEMCACHED_PORT; } --- request GET /foo --- response_body_like: STORED --- error_code: 201
The Test::Nginx library will automatically expand the special macro $TEST_NGINX_MEMCACHED_PORT to the environment with the same name. You can define your own $TEST_NGINX_BLAH_BLAH_PORT macros as long as its prefix is TEST_NGINX_ and all in upper case letters.
$TEST_NGINX_MEMCACHED_PORT
$TEST_NGINX_BLAH_BLAH_PORT
And now we can run your test script against the etcproxy port 11984:
TEST_NGINX_MEMCACHED_PORT=11984 prove t/foo.t
Then the TCP chains look like this:
Test::Nginx <=> nginx (1984) <=> etcproxy (11984) <=> memcached (11211)
If TEST_NGINX_MEMCACHED_PORT is not set, then it will take the default value 11211, which is what we want when there's no etcproxy configured:
TEST_NGINX_MEMCACHED_PORT
Test::Nginx <=> nginx (1984) <=> memcached (11211)
This approach also works for proxied mysql and postgres traffic. Please see the live test suite of ngx_drizzle and ngx_postgres for more details.
Usually we set both TEST_NGINX_CLIENT_PORT and TEST_NGINX_MEMCACHED_PORT (and etc) at the same time, effectively yielding the following chain:
Test::Nginx <=> etcproxy (1234) <=> nginx (1984) <=> etcproxy (11984) <=> memcached (11211)
as long as you run two separate etcproxy instances in two separate terminals.
It's easy to verify if the traffic actually goes through your etcproxy server. Just check if the terminal running etcproxy emits outputs. By default, etcproxy always dump out the incoming and outgoing data to stdout/stderr.
This module has a Git repository on Github, which has access for all:
https://github.com/openresty/test-nginx
If you want a commit bit, feel free to drop me a line.
The openresty-en mailing list is for English speakers: https://groups.google.com/group/openresty-en
openresty-en
The openresty mailing list is for Chinese speakers: https://groups.google.com/group/openresty
openresty
Yichun "agentzh" Zhang (章亦春) <agentzh@gmail.com>, OpenResty Inc.
<agentzh@gmail.com>
Antoine BONAVITA <antoine.bonavita@gmail.com>
<antoine.bonavita@gmail.com>
Copyright (c) 2009-2016, Yichun Zhang <agentzh@gmail.com>, OpenResty Inc.
Copyright (c) 2011-2012, Antoine BONAVITA <antoine.bonavita@gmail.com>.
This module is licensed under the terms of the BSD license.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
Neither the name of the authors nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Test::Nginx::Lua, Test::Nginx::Lua::Stream, Test::Nginx::LWP, Test::Base.
To install Test::Nginx, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Test::Nginx
CPAN shell
perl -MCPAN -e shell install Test::Nginx
For more information on module installation, please visit the detailed CPAN module installation guide.