maxcpus=2 will only boot 2. You can choose to bring the
other cpus later online, read FAQ's for more info.
-additional_cpus=n [x86_64, s390 only] use this to limit hotpluggable cpus.
- This option sets
+additional_cpus=n (*) Use this to limit hotpluggable cpus. This option sets
cpu_possible_map = cpu_present_map + additional_cpus
-ia64 and x86_64 use the number of disabled local apics in ACPI tables MADT
-to determine the number of potentially hot-pluggable cpus. The implementation
-should only rely on this to count the #of cpus, but *MUST* not rely on the
-apicid values in those tables for disabled apics. In the event BIOS doesnt
-mark such hot-pluggable cpus as disabled entries, one could use this
-parameter "additional_cpus=x" to represent those cpus in the cpu_possible_map.
+cede_offline={"off","on"} Use this option to disable/enable putting offlined
+ processors to an extended H_CEDE state on
+ supported pseries platforms.
+ If nothing is specified,
+ cede_offline is set to "on".
+(*) Option valid only for following architectures
+- ia64
-possible_cpus=n [s390 only] use this to set hotpluggable cpus.
+ia64 uses the number of disabled local apics in ACPI tables MADT to
+determine the number of potentially hot-pluggable cpus. The implementation
+should only rely on this to count the # of cpus, but *MUST* not rely
+on the apicid values in those tables for disabled apics. In the event
+BIOS doesn't mark such hot-pluggable cpus as disabled entries, one could
+use this parameter "additional_cpus=x" to represent those cpus in the
+cpu_possible_map.
+
+possible_cpus=n [s390,x86_64] use this to set hotpluggable cpus.
This option sets possible_cpus bits in
cpu_possible_map. Thus keeping the numbers of bits set
constant even if the machine gets rebooted.
- This option overrides additional_cpus.
CPU maps and such
-----------------
You really dont need to manipulate any of the system cpu maps. They should
be read-only for most use. When setting up per-cpu resources almost always use
-cpu_possible_map/for_each_cpu() to iterate.
+cpu_possible_map/for_each_possible_cpu() to iterate.
Never use anything other than cpumask_t to represent bitmap of CPUs.
-#include <linux/cpumask.h>
+ #include <linux/cpumask.h>
-for_each_cpu - Iterate over cpu_possible_map
-for_each_online_cpu - Iterate over cpu_online_map
-for_each_present_cpu - Iterate over cpu_present_map
-for_each_cpu_mask(x,mask) - Iterate over some random collection of cpu mask.
+ for_each_possible_cpu - Iterate over cpu_possible_map
+ for_each_online_cpu - Iterate over cpu_online_map
+ for_each_present_cpu - Iterate over cpu_present_map
+ for_each_cpu_mask(x,mask) - Iterate over some random collection of cpu mask.
-#include <linux/cpu.h>
-lock_cpu_hotplug() and unlock_cpu_hotplug():
+ #include <linux/cpu.h>
+ get_online_cpus() and put_online_cpus():
-The above calls are used to inhibit cpu hotplug operations. While holding the
-cpucontrol mutex, cpu_online_map will not change. If you merely need to avoid
-cpus going away, you could also use preempt_disable() and preempt_enable()
-for those sections. Just remember the critical section cannot call any
+The above calls are used to inhibit cpu hotplug operations. While the
+cpu_hotplug.refcount is non zero, the cpu_online_map will not change.
+If you merely need to avoid cpus going away, you could also use
+preempt_disable() and preempt_enable() for those sections.
+Just remember the critical section cannot call any
function that can sleep or schedule this process away. The preempt_disable()
will work as long as stop_machine_run() is used to take a cpu down.
CPU Hotplug - Frequently Asked Questions.
-Q: How to i enable my kernel to support CPU hotplug?
+Q: How to enable my kernel to support CPU hotplug?
A: When doing make defconfig, Enable CPU hotplug support
"Processor type and Features" -> Support for Hotpluggable CPUs
Check if sysfs is mounted, using the "mount" command. You should notice
an entry as shown below in the output.
-....
-none on /sys type sysfs (rw)
-....
+ ....
+ none on /sys type sysfs (rw)
+ ....
-if this is not mounted, do the following.
+If this is not mounted, do the following.
-#mkdir /sysfs
-#mount -t sysfs sys /sys
+ #mkdir /sysfs
+ #mount -t sysfs sys /sys
-now you should see entries for all present cpu, the following is an example
+Now you should see entries for all present cpu, the following is an example
in a 8-way system.
-#pwd
-#/sys/devices/system/cpu
-#ls -l
-total 0
-drwxr-xr-x 10 root root 0 Sep 19 07:44 .
-drwxr-xr-x 13 root root 0 Sep 19 07:45 ..
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu0
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu1
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu2
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu3
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu4
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu5
-drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu6
-drwxr-xr-x 3 root root 0 Sep 19 07:48 cpu7
+ #pwd
+ #/sys/devices/system/cpu
+ #ls -l
+ total 0
+ drwxr-xr-x 10 root root 0 Sep 19 07:44 .
+ drwxr-xr-x 13 root root 0 Sep 19 07:45 ..
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu0
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu1
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu2
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu3
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu4
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu5
+ drwxr-xr-x 3 root root 0 Sep 19 07:44 cpu6
+ drwxr-xr-x 3 root root 0 Sep 19 07:48 cpu7
Under each directory you would find an "online" file which is the control
file to logically online/offline a processor.
Q: Does hot-add/hot-remove refer to physical add/remove of cpus?
A: The usage of hot-add/remove may not be very consistently used in the code.
-CONFIG_CPU_HOTPLUG enables logical online/offline capability in the kernel.
+CONFIG_HOTPLUG_CPU enables logical online/offline capability in the kernel.
To support physical addition/removal, one would need some BIOS hooks and
the platform should have something like an attention button in PCI hotplug.
CONFIG_ACPI_HOTPLUG_CPU enables ACPI support for physical add/remove of CPUs.
Q: How do i logically offline a CPU?
A: Do the following.
-#echo 0 > /sys/devices/system/cpu/cpuX/online
+ #echo 0 > /sys/devices/system/cpu/cpuX/online
-once the logical offline is successful, check
+Once the logical offline is successful, check
-#cat /proc/interrupts
+ #cat /proc/interrupts
-you should now not see the CPU that you removed. Also online file will report
+You should now not see the CPU that you removed. Also online file will report
the state as 0 when a cpu if offline and 1 when its online.
-#To display the current cpu state.
-#cat /sys/devices/system/cpu/cpuX/online
+ #To display the current cpu state.
+ #cat /sys/devices/system/cpu/cpuX/online
Q: Why cant i remove CPU0 on some systems?
A: Some architectures may have some special dependency on a certain CPU.
A: The following happen, listed in no particular order :-)
- A notification is sent to in-kernel registered modules by sending an event
- CPU_DOWN_PREPARE
-- All process is migrated away from this outgoing CPU to a new CPU
+ CPU_DOWN_PREPARE or CPU_DOWN_PREPARE_FROZEN, depending on whether or not the
+ CPU is being offlined while tasks are frozen due to a suspend operation in
+ progress
+- All processes are migrated away from this outgoing CPU to new CPUs.
+ The new CPU is chosen from each process' current cpuset, which may be
+ a subset of all online CPUs.
- All interrupts targeted to this CPU is migrated to a new CPU
- timers/bottom half/task lets are also migrated to a new CPU
- Once all services are migrated, kernel calls an arch specific routine
__cpu_disable() to perform arch specific cleanup.
- Once this is successful, an event for successful cleanup is sent by an event
- CPU_DEAD.
+ CPU_DEAD (or CPU_DEAD_FROZEN if tasks are frozen due to a suspend while the
+ CPU is being offlined).
"It is expected that each service cleans up when the CPU_DOWN_PREPARE
notifier is called, when CPU_DEAD is called its expected there is nothing
departure, how to i arrange for proper notification?
A: This is what you would need in your kernel code to receive notifications.
- #include <linux/cpu.h>
- static int __cpuinit foobar_cpu_callback(struct notifier_block *nfb,
+ #include <linux/cpu.h>
+ static int __cpuinit foobar_cpu_callback(struct notifier_block *nfb,
unsigned long action, void *hcpu)
{
unsigned int cpu = (unsigned long)hcpu;
switch (action) {
case CPU_ONLINE:
+ case CPU_ONLINE_FROZEN:
foobar_online_action(cpu);
break;
case CPU_DEAD:
+ case CPU_DEAD_FROZEN:
foobar_dead_action(cpu);
break;
}
return NOTIFY_OK;
}
- static struct notifier_block foobar_cpu_notifer =
+ static struct notifier_block __cpuinitdata foobar_cpu_notifer =
{
.notifier_call = foobar_cpu_callback,
};
+You need to call register_cpu_notifier() from your init function.
+Init functions could be of two types:
+1. early init (init function called when only the boot processor is online).
+2. late init (init function called _after_ all the CPUs are online).
-In your init function,
+For the first case, you should add the following to your init function
register_cpu_notifier(&foobar_cpu_notifier);
+For the second case, you should add the following to your init function
+
+ register_hotcpu_notifier(&foobar_cpu_notifier);
+
You can fail PREPARE notifiers if something doesn't work to prepare resources.
This will stop the activity and send a following CANCELED event back.
A: Yes, CPU notifiers are called only when new CPUs are on-lined or offlined.
If you need to perform some action for each cpu already in the system, then
- for_each_online_cpu(i) {
+ for_each_online_cpu(i) {
foobar_cpu_callback(&foobar_cpu_notifier, CPU_UP_PREPARE, i);
- foobar_cpu_callback(&foobar-cpu_notifier, CPU_ONLINE, i);
- }
+ foobar_cpu_callback(&foobar_cpu_notifier, CPU_ONLINE, i);
+ }
Q: If i would like to develop cpu hotplug support for a new architecture,
what do i need at a minimum?
Q: I need to ensure that a particular cpu is not removed when there is some
work specific to this cpu is in progress.
-A: First switch the current thread context to preferred cpu
-
- int my_func_on_cpu(int cpu)
- {
- cpumask_t saved_mask, new_mask = CPU_MASK_NONE;
- int curr_cpu, err = 0;
-
- saved_mask = current->cpus_allowed;
- cpu_set(cpu, new_mask);
- err = set_cpus_allowed(current, new_mask);
-
- if (err)
- return err;
-
- /*
- * If we got scheduled out just after the return from
- * set_cpus_allowed() before running the work, this ensures
- * we stay locked.
- */
- curr_cpu = get_cpu();
-
- if (curr_cpu != cpu) {
- err = -EAGAIN;
- goto ret;
- } else {
- /*
- * Do work : But cant sleep, since get_cpu() disables preempt
- */
- }
- ret:
- put_cpu();
- set_cpus_allowed(current, saved_mask);
- return err;
- }
+A: There are two ways. If your code can be run in interrupt context, use
+ smp_call_function_single(), otherwise use work_on_cpu(). Note that
+ work_on_cpu() is slow, and can fail due to out of memory:
+ int my_func_on_cpu(int cpu)
+ {
+ int err;
+ get_online_cpus();
+ if (!cpu_online(cpu))
+ err = -EINVAL;
+ else
+#if NEEDS_BLOCKING
+ err = work_on_cpu(cpu, __my_func_on_cpu, NULL);
+#else
+ smp_call_function_single(cpu, __my_func_on_cpu, &err,
+ true);
+#endif
+ put_online_cpus();
+ return err;
+ }
Q: How do we determine how many CPUs are available for hotplug.
A: There is no clear spec defined way from ACPI that can give us that
git://ftp.safe.ca / safe / jmp / linux-2.6 / blobdiff