Based on kernel version 2.6.27. Page generated on 2008-10-13 09:53 EST.
1 2 The Resource Counter 3 4 The resource counter, declared at include/linux/res_counter.h, 5 is supposed to facilitate the resource management by controllers 6 by providing common stuff for accounting. 7 8 This "stuff" includes the res_counter structure and routines 9 to work with it. 10 11 12 13 1. Crucial parts of the res_counter structure 14 15 a. unsigned long long usage 16 17 The usage value shows the amount of a resource that is consumed 18 by a group at a given time. The units of measurement should be 19 determined by the controller that uses this counter. E.g. it can 20 be bytes, items or any other unit the controller operates on. 21 22 b. unsigned long long max_usage 23 24 The maximal value of the usage over time. 25 26 This value is useful when gathering statistical information about 27 the particular group, as it shows the actual resource requirements 28 for a particular group, not just some usage snapshot. 29 30 c. unsigned long long limit 31 32 The maximal allowed amount of resource to consume by the group. In 33 case the group requests for more resources, so that the usage value 34 would exceed the limit, the resource allocation is rejected (see 35 the next section). 36 37 d. unsigned long long failcnt 38 39 The failcnt stands for "failures counter". This is the number of 40 resource allocation attempts that failed. 41 42 c. spinlock_t lock 43 44 Protects changes of the above values. 45 46 47 48 2. Basic accounting routines 49 50 a. void res_counter_init(struct res_counter *rc) 51 52 Initializes the resource counter. As usual, should be the first 53 routine called for a new counter. 54 55 b. int res_counter_charge[_locked] 56 (struct res_counter *rc, unsigned long val) 57 58 When a resource is about to be allocated it has to be accounted 59 with the appropriate resource counter (controller should determine 60 which one to use on its own). This operation is called "charging". 61 62 This is not very important which operation - resource allocation 63 or charging - is performed first, but 64 * if the allocation is performed first, this may create a 65 temporary resource over-usage by the time resource counter is 66 charged; 67 * if the charging is performed first, then it should be uncharged 68 on error path (if the one is called). 69 70 c. void res_counter_uncharge[_locked] 71 (struct res_counter *rc, unsigned long val) 72 73 When a resource is released (freed) it should be de-accounted 74 from the resource counter it was accounted to. This is called 75 "uncharging". 76 77 The _locked routines imply that the res_counter->lock is taken. 78 79 80 2.1 Other accounting routines 81 82 There are more routines that may help you with common needs, like 83 checking whether the limit is reached or resetting the max_usage 84 value. They are all declared in include/linux/res_counter.h. 85 86 87 88 3. Analyzing the resource counter registrations 89 90 a. If the failcnt value constantly grows, this means that the counter's 91 limit is too tight. Either the group is misbehaving and consumes too 92 many resources, or the configuration is not suitable for the group 93 and the limit should be increased. 94 95 b. The max_usage value can be used to quickly tune the group. One may 96 set the limits to maximal values and either load the container with 97 a common pattern or leave one for a while. After this the max_usage 98 value shows the amount of memory the container would require during 99 its common activity. 100 101 Setting the limit a bit above this value gives a pretty good 102 configuration that works in most of the cases. 103 104 c. If the max_usage is much less than the limit, but the failcnt value 105 is growing, then the group tries to allocate a big chunk of resource 106 at once. 107 108 d. If the max_usage is much less than the limit, but the failcnt value 109 is 0, then this group is given too high limit, that it does not 110 require. It is better to lower the limit a bit leaving more resource 111 for other groups. 112 113 114 115 4. Communication with the control groups subsystem (cgroups) 116 117 All the resource controllers that are using cgroups and resource counters 118 should provide files (in the cgroup filesystem) to work with the resource 119 counter fields. They are recommended to adhere to the following rules: 120 121 a. File names 122 123 Field name File name 124 --------------------------------------------------- 125 usage usage_in_<unit_of_measurement> 126 max_usage max_usage_in_<unit_of_measurement> 127 limit limit_in_<unit_of_measurement> 128 failcnt failcnt 129 lock no file :) 130 131 b. Reading from file should show the corresponding field value in the 132 appropriate format. 133 134 c. Writing to file 135 136 Field Expected behavior 137 ---------------------------------- 138 usage prohibited 139 max_usage reset to usage 140 limit set the limit 141 failcnt reset to zero 142 143 144 145 5. Usage example 146 147 a. Declare a task group (take a look at cgroups subsystem for this) and 148 fold a res_counter into it 149 150 struct my_group { 151 struct res_counter res; 152 153 <other fields> 154 } 155 156 b. Put hooks in resource allocation/release paths 157 158 int alloc_something(...) 159 { 160 if (res_counter_charge(res_counter_ptr, amount) < 0) 161 return -ENOMEM; 162 163 <allocate the resource and return to the caller> 164 } 165 166 void release_something(...) 167 { 168 res_counter_uncharge(res_counter_ptr, amount); 169 170 <release the resource> 171 } 172 173 In order to keep the usage value self-consistent, both the 174 "res_counter_ptr" and the "amount" in release_something() should be 175 the same as they were in the alloc_something() when the releasing 176 resource was allocated. 177 178 c. Provide the way to read res_counter values and set them (the cgroups 179 still can help with it). 180 181 c. Compile and run :)