[Kernel] CGroups

3 min readJan 10, 2022

Cgroups

# check cgroup version
grep cgroup /proc/filesystems# installation
sudo apt install cgroup-tools# copy conf
sudo cp /usr/share/doc/cgroup-tools/examples/cgred.conf /etc/

Configuration

Metric explanation

# tunable list of cpu
ls -l /sys/fs/cgroup/cpuacct/# list current cgroups for cpu
ls /sys/fs/cgroup/cpu,cpuacct# /etc/cgconfig.conf
group app1{
     cpu {
         cpu.cfs_quota_us=10000; // 10% of cpu single core
         cpu.cfs_period_us=100000; // 10% of cpu single core (default of 100 ms)
         //  this begins with the root / cgroup with 1024 shares and 100% of CPU resources.
         // https://www.redhat.com/sysadmin/cgroups-part-two
         //user.slice, system.slice each one uses 1024
         cpu.shares="1024"; // Equally share the cpu     }
     memory {
         memory.limit_in_bytes = 1024m;
     }
}# /etc/cgrules.conf
#<user>    <controllers>           <destination>
app_user       cpu,memory              app1

multicore limit

cpu.cfs_period_us
Specifies a period in microseconds (represented by us for µs) to indicate how often a cgroup’s access to CPU resources should be reallocated. For example, if you want tasks in a cgroup to have access to a single CPU for 0.2 seconds in a 1 second window, set cpu.cfs_quota_us to 200000 and cpu.cfs_period_us to 1000000. The upper limit of the cpu.cfs_quota_us parameter is 1 second and the lower limit is 1000 microseconds.
cpu.cfs_quota_us
Specifies the total amount of runtime in microseconds (represented by us for µs), for which all tasks in the Drill cgroup can run during one period (as defined by cpu.cfs_period_us). When tasks in the Drill cgroup use up all the time specified by the quota, the tasks are throttled for the remainder of the time specified by the period and they cannot run until the next period. For example, if tasks in the Drill cgroup can access a single CPU for 0.2 seconds out of every 1 second, set cpu.cfs_quota_us to 200000 and cpu.cfs_period_us to 1000000. Setting the cpu.cfs_quota_us value to -1 indicates that the group does not have any restrictions on CPU. This is the default value for every cgroup, except for the root cgroup.
cpu.shares: Default is 1024. setting shares to 1024 implies equally split cpu share in CFS. Check the reference from design of CFS for more details.
cpuset.cpus: Specify the cpu set is permitted to access

#  CFS quotas will permit a group to execute for up to cpu.cfs_quota_us microseconds within a period of cpu.cfs_period_us microseconds, (default of 100 ms.)// use 7 out of 8 cores in a machine
cpu.cfs_quota_us=700000; // use 7 out of 8 cores in total
cpu.cfs_period_us=100000; // a second window
cpu.shares=6144 // 6144/(1024+1024+6144) ~= 75%
// 8 cores, 16g mem examplegroup app1 {
     cpu {
         cpu.shares=8192;
         cpu.cfs_quota_us=800000;
         cpu.cfs_period_us=100000;     
     }
     memory {
         memory.limit_in_bytes = 16g;
     }
}

Testing conf

sudo /usr/sbin/cgconfigparser -l /etc/cgconfig.conf
sudo /usr/sbin/cgrulesengd -vvv

Check cgroups is working or not

cat /sys/fs/cgroup/cpu/supplyframeapp/tasks
cat /sys/fs/cgroup/memory/supplyframeapp/tasks

cgconfigparser.service

[Unit]
Description=cgroup config parser
After=network.target

[Service]
User=root
Group=root
ExecStart=/usr/sbin/cgconfigparser -l /etc/cgconfig.conf
Type=oneshot

[Install]
WantedBy=multi-user.target

cgrulesgend.service

sudo service cgrulesgend start # a daemon to distribute processes to control groups[Unit]
Description=cgroup rules generator
After=network.target cgconfigparser.service

[Service]
User=root
Group=root
Type=forking
EnvironmentFile=-/etc/cgred.conf
ExecStart=/usr/sbin/cgrulesengd
Restart=on-failure

[Install]
WantedBy=multi-user.target

How to use systemd services

Update /etc/cgconfig.conf
sudo service cgconfigparser start // reload new conf
when cgrulesgend.service is running, it will sudo distribute processes to control groups

References

[Kernel] CGroups

Cgroups

Written by Brian Pan