Name Date Size #Lines LOC

..25-Jun-2020-

unit-tests/H01-May-2019-

Config.ccH A D06-Jun-20207.8 KiB287233

Config.hH A D06-Jun-20202.4 KiB7537

ContentRange.ccH A D11-Jan-20191.5 KiB5629

ContentRange.hH A D06-Jun-20201.6 KiB5621

Data.hH A D06-Jun-20203.2 KiB12379

HttpHeader.ccH A D09-Jun-20209.7 KiB420306

HttpHeader.hH A D06-Jun-20206.1 KiB235123

Makefile.incH A D29-Apr-20202.6 KiB7144

Makefile.tsxsH A D21-Mar-20201.6 KiB6638

README.mdH A D20-Nov-20196.2 KiB166127

Range.ccH A D21-Mar-20204.5 KiB197146

Range.hH A D06-Jun-20202.3 KiB7921

Stage.hH A D21-Mar-20204 KiB191150

client.ccH A D06-Jun-20206.7 KiB192134

client.hH A D21-Mar-20201.2 KiB376

intercept.ccH A D21-Mar-20202.8 KiB8347

intercept.hH A D11-Jan-2019893 243

response.ccH A D20-Apr-20193.6 KiB10871

response.hH A D11-Jan-20191 KiB296

server.ccH A D09-Jun-202022.6 KiB694477

server.hH A D11-Jan-20191.4 KiB384

slice.ccH A D06-Jun-202011 KiB359263

slice.hH A D06-Jun-20202.7 KiB9352

slice_test.ccH A D30-Apr-20195.7 KiB196144

transfer.ccH A D21-Mar-20204.3 KiB152102

transfer.hH A D21-Mar-20201.4 KiB385

util.ccH A D06-Jun-20204.4 KiB157103

util.hH A D21-Mar-20201.2 KiB377

README.md

1### Apache Traffic Server - Slicer Plugin
2
3The purpose of this plugin is to slice full file or range based requests
4into deterministic chunks.  This allows a large file to be spread across
5multiple cache stripes and allows range requests to be satisfied by
6stitching these chunks together.
7
8Deterministic chunks are requested from a parent cache or origin server
9using a preconfigured block byte size.
10
11The plugin is an example of an intercept handler which takes a single
12incoming request (range or whole asset), breaks it into a sequence
13of block requests and assembles those blocks into a client response.
14The plugin uses TSHttpConnect to delegate each block request to
15cache_range_requests.so which handles all cache and parent interaction.
16
17To enable the plugin, specify the plugin library via @plugin at the end
18of a remap line as follows (default 1MB slice in this example):
19
20```
21map http://ats-cache/ http://parent/ @plugin=slice.so @plugin=cache_range_requests.so
22map https://ats-cache/ http://parent/ @plugin=slice.so @plugin=cache_range_requests.so
23```
24
25alternatively (2MB slice block)
26
27```
28map http://ats-cache/ http://parent/ @plugin=slice.so @pparam=-b @pparam=2M @plugin=cache_range_requests.so
29map https://ats-cache/ http://parent/ @plugin=slice.so @pparam=--blockbytes=2M @plugin=cache_range_requests.so
30```
31
32Options for the slice plugin (typically last one wins):
33```
34--blockbytes=<number bytes> (optional)
35  Slice block size.
36  Default is 1m or 1048576 bytes.
37  also -b <num bytes>
38  Suffix k,m,g supported.
39  Limited to 32k and 32m inclusive.
40  For backwards compatibility blockbytes:<num bytes> is also supported.
41
42--blockbytes-test=<number bytes> (optional)
43  Slice block size for testing.
44  also -t <num bytes>
45  Suffix k,m,g supported.
46  Limited to any positive number.
47  Ignored if --blockbytes is provided.
48
49--remap-host=<loopback hostname> (optional)
50  Uses effective url with given host and port 0 for remapping.
51  Requires setting up an intermediate loopback remap rule.
52  -r for short
53
54--pace-errorlog=<second(s)> (optional)
55  Limit stitching error logs to every 'n' second(s)
56  Default is to log all errors (no pacing).
57  also -e <seconds>
58
59--disable-errorlog (optional)
60  Disable writing stitching errors to the error log.
61  also -d
62```
63
64By default the plugin uses the pristine url to loopback call back
65into the same rule as each range slice is issued.  The effective url
66with loopback remap host may be used by adding the '-r <hostname>' or
67'--remap-host=<hostname>' plugin option.
68
69Using the `--remap-host` option splits the plugin chain into 2 remap rules.
70One remap rule for all the incoming requests and the other for just the block
71range requests.  This allows for easier trouble shooting via logs and
72also allows for more effecient plugin rules.  The default pristine method
73runs the remap plugins twice, one for the incoming request and one for
74eace slice.  Splitting the rules allows for plugins like URI signing to
75be done on the client request only.
76
77NOTE: Requests NOT handled by the slice plugin (ie: HEAD requests) are
78handled as with a typical remap rule.  GET requests intercepted by the
79slice plugin are virtually reissued into ATS and are forward proxied
80through the cache_range_requests plugin.
81
82```
83map http://ats/ http://parent/ @plugin=slice.so @pparam=--blockbytes=512k @pparam=--remap-host=loopback
84map https://ats/ https://parent/ @plugin=slice.so @pparam=--blockbytes=512k @pparam=--remap-host=loopback
85
86# Virtual forward proxy for slice range blocks
87map http://loopback/ http://parent/ @plugin=cache_range_requests.so
88map https://loopback/ http://parent/ @plugin=cache_range_requests.so
89```
90
91**Note**: For default pristine behavior cache_range_requests **MUST**
92follow slice.so Put these plugins at the end of the plugin list
93
94**Note**: blockbytes is defined in bytes. Postfix for 'K', 'M' and 'G'
95may be used.  1048576 (1MB) is the default.
96
97For testing purposes an unchecked value of "blockbytes-test" is also available.
98
99Debug output can be enable by setting the debug tag: **slice**.  If debug
100is enabled all block stitch errors will log to diags.log
101
102The slice plugin is susceptible to block stitching errors caused by
103mismatched blocks.  For these cases special detailed error logs are
104provided to help with debugging.  Below is a sample error log entry::
105
106```
107[Apr 19 20:26:13.639] [ET_NET 17] ERROR: [slice] 1555705573.639 reason="Non 206 internal block response" uri="http://localhost:18080/%7Ep.tex/%7Es.50M/%7Eui.20000/" uas="curl/7.29.0" req_range="bytes=1000000-" norm_range="bytes 1000000-52428799/52428800" etag_exp="%221603934496%22" lm_exp="Fri, 19 Apr 2019 18:53:20 GMT" blk_range="21000000-21999999" status_got="400" cr_got="" etag_got="" lm_got="" cc="no-store" via=""
108```
109
110Current error types logged:
111```
112    Mismatch block Etag
113    Mismatch block Last-Modified
114    Non 206 internal block response
115    Mismatch/Bad block Content-Range
116```
117
118
119With slice error logs disabled these type errors can typically be detected
120by observing crc=ERR_READ_ERROR and pscl=0 in normal logs.
121
122At the current time only single range requests or the first part of a
123multi part range request of the forms:
124```
125Range: bytes=<begin>-<end>
126Range: bytes=<begin>-
127Range: bytes=-<last N bytes>
128```
129are supported as multi part range responses are non trivial to implement.
130This matches with the cache_range_requests.so plugin capability.
131---
132
133Important things to note:
134
135Any first block server response that is not a 206 is passed down to
136the client.
137
138Only the first server response block is used to evaluate any "If-"
139headers.  Subsequent server slice block requests remove these headers.
140
141If a client aborts mid transaction the current slice block is completed
142to ensure that the block is written to cache.
143
144The only 416 case this plugin handles itself is if the requested range
145is inside the end slice block but past the content length.  Otherwise
146parents seem to properly issue 416 responses themselves.
147
148---
149
150To manually build the plugin use the "tsxs" executable that installs with
151traffic_server.
152
153Running the following command will build the plugin
154
155```
156tsxs -v -o slice.so *.cc
157```
158
159Running the following command will build and install the plugin.
160Beware this may crash a running system if the plugin is loaded
161and the OS uses memory paging with plugins.
162
163```
164tsxs -v -i -o slice.so *.cc
165```
166