-.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions
- are met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in
- the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Intel Corporation nor the names of its
- contributors may be used to endorse or promote products derived
- from this software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-VM Power Management Application
-===============================
-
-Introduction
-------------
-
-Applications running in Virtual Environments have an abstract view of
-the underlying hardware on the Host, in particular applications cannot see
-the binding of virtual to physical hardware.
-When looking at CPU resourcing, the pinning of Virtual CPUs(vCPUs) to
-Host Physical CPUs(pCPUS) is not apparent to an application
-and this pinning may change over time.
-Furthermore, Operating Systems on virtual machines do not have the ability
-to govern their own power policy; the Machine Specific Registers (MSRs)
-for enabling P-State transitions are not exposed to Operating Systems
-running on Virtual Machines(VMs).
-
-The Virtual Machine Power Management solution shows an example of
-how a DPDK application can indicate its processing requirements using VM local
-only information(vCPU/lcore) to a Host based Monitor which is responsible
-for accepting requests for frequency changes for a vCPU, translating the vCPU
-to a pCPU via libvirt and affecting the change in frequency.
-
-The solution is comprised of two high-level components:
-
-#. Example Host Application
-
- Using a Command Line Interface(CLI) for VM->Host communication channel management
- allows adding channels to the Monitor, setting and querying the vCPU to pCPU pinning,
- inspecting and manually changing the frequency for each CPU.
- The CLI runs on a single lcore while the thread responsible for managing
- VM requests runs on a second lcore.
-
- VM requests arriving on a channel for frequency changes are passed
- to the librte_power ACPI cpufreq sysfs based library.
- The Host Application relies on both qemu-kvm and libvirt to function.
-
-#. librte_power for Virtual Machines
-
- Using an alternate implementation for the librte_power API, requests for
- frequency changes are forwarded to the host monitor rather than
- the APCI cpufreq sysfs interface used on the host.
-
- The l3fwd-power application will use this implementation when deployed on a VM
- (see Chapter 11 "L3 Forwarding with Power Management Application").
-
-.. _figure_24:
-
-**Figure 24. Highlevel Solution**
-
-|vm_power_mgr_highlevel|
-
-Overview
---------
-
-VM Power Management employs qemu-kvm to provide communications channels
-between the host and VMs in the form of Virtio-Serial which appears as
-a paravirtualized serial device on a VM and can be configured to use
-various backends on the host. For this example each Virtio-Serial endpoint
-on the host is configured as AF_UNIX file socket, supporting poll/select
-and epoll for event notification.
-In this example each channel endpoint on the host is monitored via
-epoll for EPOLLIN events.
-Each channel is specified as qemu-kvm arguments or as libvirt XML for each VM,
-where each VM can have a number of channels up to a maximum of 64 per VM,
-in this example each DPDK lcore on a VM has exclusive access to a channel.
-
-To enable frequency changes from within a VM, a request via the librte_power interface
-is forwarded via Virtio-Serial to the host, each request contains the vCPU
-and power command(scale up/down/min/max).
-The API for host and guest librte_power is consistent across environments,
-with the selection of VM or Host Implementation determined at automatically
-at runtime based on the environment.
-
-Upon receiving a request, the host translates the vCPU to a pCPU via
-the libvirt API before forwarding to the host librte_power.
-
-.. _figure_25:
-
-**Figure 25. VM request to scale frequency**
-
-|vm_power_mgr_vm_request_seq|
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2010-2014 Intel Corporation.
+
+Virtual Machine Power Management Application
+============================================
+
+Applications running in virtual environments have an abstract view of
+the underlying hardware on the host. Specifically, applications cannot
+see the binding of virtual components to physical hardware. When looking
+at CPU resourcing, the pinning of Virtual CPUs (vCPUs) to Physical CPUs
+(pCPUs) on the host is not apparent to an application and this pinning
+may change over time. In addition, operating systems on Virtual Machines
+(VMs) do not have the ability to govern their own power policy. The
+Machine Specific Registers (MSRs) for enabling P-state transitions are
+not exposed to the operating systems running on the VMs.
+
+The solution demonstrated in this sample application shows an example of
+how a DPDK application can indicate its processing requirements using
+VM-local only information (vCPU/lcore, and so on) to a host resident VM
+Power Manager. The VM Power Manager is responsible for:
+
+- **Accepting requests for frequency changes for a vCPU**
+- **Translating the vCPU to a pCPU using libvirt**
+- **Performing the change in frequency**
+
+This application demonstrates the following features:
+
+- **The handling of VM application requests to change frequency.**
+ VM applications can request frequency changes for a vCPU. The VM
+ Power Management Application uses libvirt to translate that
+ virtual CPU (vCPU) request to a physical CPU (pCPU) request and
+ performs the frequency change.
+
+- **The acceptance of power management policies from VM applications.**
+ A VM application can send a policy to the host application. The
+ policy contains rules that define the power management behaviour
+ of the VM. The host application then applies the rules of the
+ policy independent of the VM application. For example, the
+ policy can contain time-of-day information for busy/quiet
+ periods, and the host application can scale up/down the relevant
+ cores when required. See :ref:`sending_policy` for information on
+ setting policy values.
+
+- **Out-of-band monitoring of workloads using core hardware event counters.**
+ The host application can manage power for an application by looking
+ at the event counters of the cores and taking action based on the
+ branch miss/hit ratio. See :ref:`enabling_out_of_band`.
+
+ **Note**: This functionality also applies in non-virtualised environments.
+
+In addition to the ``librte_power`` library used on the host, the
+application uses a special version of ``librte_power`` on each VM, which
+directs frequency changes and policies to the host monitor rather than
+the APCI ``cpufreq`` ``sysfs`` interface used on the host in non-virtualised
+environments.
+
+.. _figure_vm_power_mgr_highlevel:
+
+.. figure:: img/vm_power_mgr_highlevel.*
+
+ Highlevel Solution
+
+In the above diagram, the DPDK Applications are shown running in
+virtual machines, and the VM Power Monitor application is shown running
+in the host.
+
+**DPDK VM Application**
+
+- Reuse ``librte_power`` interface, but uses an implementation that
+ forwards frequency requests to the host using a ``virtio-serial`` channel
+- Each lcore has exclusive access to a single channel
+- Sample application reuses ``l3fwd_power``
+- A CLI for changing frequency from within a VM is also included
+
+**VM Power Monitor**
+
+- Accepts VM commands over ``virtio-serial`` endpoints, monitored
+ using ``epoll``
+- Commands include the virtual core to be modified, using ``libvirt`` to get
+ the physical core mapping
+- Uses ``librte_power`` to affect frequency changes using Linux userspace
+ power governor (``acpi_cpufreq`` OR ``intel_pstate`` driver)
+- CLI: For adding VM channels to monitor, inspecting and changing channel
+ state, manually altering CPU frequency. Also allows for the changings
+ of vCPU to pCPU pinning
+
+Sample Application Architecture Overview
+----------------------------------------
+
+The VM power management solution employs ``qemu-kvm`` to provide
+communications channels between the host and VMs in the form of a
+``virtio-serial`` connection that appears as a para-virtualised serial
+device on a VM and can be configured to use various backends on the
+host. For this example, the configuration of each ``virtio-serial`` endpoint
+on the host as an ``AF_UNIX`` file socket, supporting poll/select and
+``epoll`` for event notification. In this example, each channel endpoint on
+the host is monitored for ``EPOLLIN`` events using ``epoll``. Each channel
+is specified as ``qemu-kvm`` arguments or as ``libvirt`` XML for each VM,
+where each VM can have several channels up to a maximum of 64 per VM. In this
+example, each DPDK lcore on a VM has exclusive access to a channel.
+
+To enable frequency changes from within a VM, the VM forwards a
+``librte_power`` request over the ``virtio-serial`` channel to the host. Each
+request contains the vCPU and power command (scale up/down/min/max). The
+API for the host ``librte_power`` and guest ``librte_power`` is consistent
+across environments, with the selection of VM or host implementation
+determined automatically at runtime based on the environment. On
+receiving a request, the host translates the vCPU to a pCPU using the
+libvirt API before forwarding it to the host ``librte_power``.
+
+
+.. _figure_vm_power_mgr_vm_request_seq:
+
+.. figure:: img/vm_power_mgr_vm_request_seq.*
+
+In addition to the ability to send power management requests to the
+host, a VM can send a power management policy to the host. In some
+cases, using a power management policy is a preferred option because it
+can eliminate possible latency issues that can occur when sending power
+management requests. Once the VM sends the policy to the host, the VM no
+longer needs to worry about power management, because the host now
+manages the power for the VM based on the policy. The policy can specify
+power behavior that is based on incoming traffic rates or time-of-day
+power adjustment (busy/quiet hour power adjustment for example). See
+:ref:`sending_policy` for more information.
+
+One method of power management is to sense how busy a core is when
+processing packets and adjusting power accordingly. One technique for
+doing this is to monitor the ratio of the branch miss to branch hits
+counters and scale the core power accordingly. This technique is based
+on the premise that when a core is not processing packets, the ratio of
+branch misses to branch hits is very low, but when the core is
+processing packets, it is measurably higher. The implementation of this
+capability is as a policy of type ``BRANCH_RATIO``.
+See :ref:`sending_policy` for more information on using the
+BRANCH_RATIO policy option.
+
+A JSON interface enables the specification of power management requests
+and policies in JSON format. The JSON interfaces provide a more
+convenient and more easily interpreted interface for the specification
+of requests and policies. See :ref:`power_man_requests` for more information.