1 ====================================
2 SLOW WORK ITEM EXECUTION THREAD POOL
3 ====================================
5 By: David Howells <dhowells@redhat.com>
7 The slow work item execution thread pool is a pool of threads for performing
8 things that take a relatively long time, such as making mkdir calls.
9 Typically, when processing something, these items will spend a lot of time
10 blocking a thread on I/O, thus making that thread unavailable for doing other
13 The standard workqueue model is unsuitable for this class of work item as that
14 limits the owner to a single thread or a single thread per CPU. For some
15 tasks, however, more threads - or fewer - are required.
17 There is just one pool per system. It contains no threads unless something
18 wants to use it - and that something must register its interest first. When
19 the pool is active, the number of threads it contains is dynamic, varying
20 between a maximum and minimum setting, depending on the load.
27 This pool support two classes of work items:
31 (*) Very slow work items.
33 The former are expected to finish much quicker than the latter.
35 An operation of the very slow class may do a batch combination of several
36 lookups, mkdirs, and a create for instance.
38 An operation of the ordinarily slow class may, for example, write stuff or
39 expand files, provided the time taken to do so isn't too long.
41 Operations of both types may sleep during execution, thus tying up the thread
44 A further class of work item is available, based on the slow work item class:
46 (*) Delayed slow work items.
48 These are slow work items that have a timer to defer queueing of the item for
52 THREAD-TO-CLASS ALLOCATION
53 --------------------------
55 Not all the threads in the pool are available to work on very slow work items.
56 The number will be between one and one fewer than the number of active threads.
57 This is configurable (see the "Pool Configuration" section).
59 All the threads are available to work on ordinarily slow work items, but a
60 percentage of the threads will prefer to work on very slow work items.
62 The configuration ensures that at least one thread will be available to work on
63 very slow work items, and at least one thread will be available that won't work
64 on very slow work items at all.
71 Firstly, a module or subsystem wanting to make use of slow work items must
72 register its interest:
74 int ret = slow_work_register_user(struct module *module);
76 This will return 0 if successful, or a -ve error upon failure. The module
77 pointer should be the module interested in using this facility (almost
78 certainly THIS_MODULE).
81 Slow work items may then be set up by:
83 (1) Declaring a slow_work struct type variable:
85 #include <linux/slow-work.h>
87 struct slow_work myitem;
89 (2) Declaring the operations to be used for this item:
91 struct slow_work_ops myitem_ops = {
92 .get_ref = myitem_get_ref,
93 .put_ref = myitem_put_ref,
94 .execute = myitem_execute,
97 [*] For a description of the ops, see section "Item Operations".
99 (3) Initialising the item:
101 slow_work_init(&myitem, &myitem_ops);
105 delayed_slow_work_init(&myitem, &myitem_ops);
109 vslow_work_init(&myitem, &myitem_ops);
111 depending on its class.
113 A suitably set up work item can then be enqueued for processing:
115 int ret = slow_work_enqueue(&myitem);
117 This will return a -ve error if the thread pool is unable to gain a reference
118 on the item, 0 otherwise, or (for delayed work):
120 int ret = delayed_slow_work_enqueue(&myitem, my_jiffy_delay);
123 The items are reference counted, so there ought to be no need for a flush
124 operation. But as the reference counting is optional, means to cancel
125 existing work items are also included:
127 cancel_slow_work(&myitem);
128 cancel_delayed_slow_work(&myitem);
130 can be used to cancel pending work. The above cancel function waits for
131 existing work to have been executed (or prevent execution of them, depending
135 When all a module's slow work items have been processed, and the
136 module has no further interest in the facility, it should unregister its
139 slow_work_unregister_user(struct module *module);
141 The module pointer is used to wait for all outstanding work items for that
142 module before completing the unregistration. This prevents the put_ref() code
143 from being taken away before it completes. module should almost certainly be
151 The slow-work facility provides a function by which it can be determined
152 whether or not an item is queued for later execution:
154 bool queued = slow_work_is_queued(struct slow_work *work);
156 If it returns false, then the item is not on the queue (it may be executing
157 with a requeue pending). This can be used to work out whether an item on which
158 another depends is on the queue, thus allowing a dependent item to be queued
166 Each work item requires a table of operations of type struct slow_work_ops.
167 Only ->execute() is required; the getting and putting of a reference and the
168 describing of an item are all optional.
170 (*) Get a reference on an item:
172 int (*get_ref)(struct slow_work *work);
174 This allows the thread pool to attempt to pin an item by getting a
175 reference on it. This function should return 0 if the reference was
176 granted, or a -ve error otherwise. If an error is returned,
177 slow_work_enqueue() will fail.
179 The reference is held whilst the item is queued and whilst it is being
180 executed. The item may then be requeued with the same reference held, or
181 the reference will be released.
183 (*) Release a reference on an item:
185 void (*put_ref)(struct slow_work *work);
187 This allows the thread pool to unpin an item by releasing the reference on
188 it. The thread pool will not touch the item again once this has been
193 void (*execute)(struct slow_work *work);
195 This should perform the work required of the item. It may sleep, it may
196 perform disk I/O and it may wait for locks.
198 (*) View an item through /proc:
200 void (*desc)(struct slow_work *work, struct seq_file *m);
202 If supplied, this should print to 'm' a small string describing the work
203 the item is to do. This should be no more than about 40 characters, and
204 shouldn't include a newline character.
206 See the 'Viewing executing and queued items' section below.
213 The slow-work thread pool has a number of configurables:
215 (*) /proc/sys/kernel/slow-work/min-threads
217 The minimum number of threads that should be in the pool whilst it is in
218 use. This may be anywhere between 2 and max-threads.
220 (*) /proc/sys/kernel/slow-work/max-threads
222 The maximum number of threads that should in the pool. This may be
223 anywhere between min-threads and 255 or NR_CPUS * 2, whichever is greater.
225 (*) /proc/sys/kernel/slow-work/vslow-percentage
227 The percentage of active threads in the pool that may be used to execute
228 very slow work items. This may be between 1 and 99. The resultant number
229 is bounded to between 1 and one fewer than the number of active threads.
230 This ensures there is always at least one thread that can process very
231 slow work items, and always at least one thread that won't.
234 ==================================
235 VIEWING EXECUTING AND QUEUED ITEMS
236 ==================================
238 If CONFIG_SLOW_WORK_PROC is enabled, a proc file is made available:
242 through which the list of work items being executed and the queues of items to
243 be executed may be viewed. The owner of a work item is given the chance to
244 add some information of its own.
246 The contents look something like the following:
248 THR PID ITEM ADDR FL MARK DESC
249 === ===== ================ == ===== ==========
250 0 3005 ffff880023f52348 a 952ms FSC: OBJ17d3: LOOK
251 1 3006 ffff880024e33668 2 160ms FSC: OBJ17e5 OP60d3b: Write1/Store fl=2
252 2 3165 ffff8800296dd180 a 424ms FSC: OBJ17e4: LOOK
253 3 4089 ffff8800262c8d78 a 212ms FSC: OBJ17ea: CRTN
254 4 4090 ffff88002792bed8 2 388ms FSC: OBJ17e8 OP60d36: Write1/Store fl=2
255 5 4092 ffff88002a0ef308 2 388ms FSC: OBJ17e7 OP60d2e: Write1/Store fl=2
256 6 4094 ffff88002abaf4b8 2 132ms FSC: OBJ17e2 OP60d4e: Write1/Store fl=2
257 7 4095 ffff88002bb188e0 a 388ms FSC: OBJ17e9: CRTN
258 vsq - ffff880023d99668 1 308ms FSC: OBJ17e0 OP60f91: Write1/EnQ fl=2
259 vsq - ffff8800295d1740 1 212ms FSC: OBJ16be OP4d4b6: Write1/EnQ fl=2
260 vsq - ffff880025ba3308 1 160ms FSC: OBJ179a OP58dec: Write1/EnQ fl=2
261 vsq - ffff880024ec83e0 1 160ms FSC: OBJ17ae OP599f2: Write1/EnQ fl=2
262 vsq - ffff880026618e00 1 160ms FSC: OBJ17e6 OP60d33: Write1/EnQ fl=2
263 vsq - ffff880025a2a4b8 1 132ms FSC: OBJ16a2 OP4d583: Write1/EnQ fl=2
264 vsq - ffff880023cbe6d8 9 212ms FSC: OBJ17eb: LOOK
265 vsq - ffff880024d37590 9 212ms FSC: OBJ17ec: LOOK
266 vsq - ffff880027746cb0 9 212ms FSC: OBJ17ed: LOOK
267 vsq - ffff880024d37ae8 9 212ms FSC: OBJ17ee: LOOK
268 vsq - ffff880024d37cb0 9 212ms FSC: OBJ17ef: LOOK
269 vsq - ffff880025036550 9 212ms FSC: OBJ17f0: LOOK
270 vsq - ffff8800250368e0 9 212ms FSC: OBJ17f1: LOOK
271 vsq - ffff880025036aa8 9 212ms FSC: OBJ17f2: LOOK
273 In the 'THR' column, executing items show the thread they're occupying and
274 queued threads indicate which queue they're on. 'PID' shows the process ID of
275 a slow-work thread that's executing something. 'FL' shows the work item flags.
276 'MARK' indicates how long since an item was queued or began executing. Lastly,
277 the 'DESC' column permits the owner of an item to give some information.