From a63504a90f6aa55ae7aea204a8944cd9af9342bd Mon Sep 17 00:00:00 2001 From: David Hunt Date: Wed, 17 Oct 2018 14:05:31 +0100 Subject: [PATCH] examples/power: add JSON string handling Add JSON string handling to vm_power_manager for JSON strings received through the fifo. The format of the JSON strings are detailed in the next patch, the vm_power_manager user guide documentation updates. This patch introduces a new dependency on Jansson, a C library for encoding, decoding and manipulating JSON data. To compile the sample app you now need to have installed libjansson4 and libjansson-dev (these may be named slightly differently depending on your Operating System) Signed-off-by: David Hunt Acked-by: Anatoly Burakov --- doc/guides/rel_notes/release_18_11.rst | 8 + .../sample_app_ug/vm_power_management.rst | 264 +++++++++++++ examples/vm_power_manager/Makefile | 6 + examples/vm_power_manager/channel_monitor.c | 371 ++++++++++++++++-- 4 files changed, 624 insertions(+), 25 deletions(-) diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst index 988cfb3a35..771295df2c 100644 --- a/doc/guides/rel_notes/release_18_11.rst +++ b/doc/guides/rel_notes/release_18_11.rst @@ -200,6 +200,14 @@ New Features See the :doc:`../prog_guide/power_man` section of the DPDK Programmers Guide document for more information. +* **Added JSON power policy interface for containers.** + + Extended the Power Library and vm_power_manager sample app to allow power + policies to be submitted via a FIFO using JSON formatted strings. Previously + limited to Virtual Machines, this feature extends power policy functionality + to containers and host applications that need to have their cores frequency + controlled based on the rules contained in the policy. + * **Added ability to switch queue deferred start flag on testpmd app.** Added a console command to testpmd app, giving ability to switch diff --git a/doc/guides/sample_app_ug/vm_power_management.rst b/doc/guides/sample_app_ug/vm_power_management.rst index 855570d6b5..59439e8f30 100644 --- a/doc/guides/sample_app_ug/vm_power_management.rst +++ b/doc/guides/sample_app_ug/vm_power_management.rst @@ -337,6 +337,270 @@ monitoring of branch ratio on cores doing busy polling via PMDs. and will need to be adjusted for different workloads. + +JSON API +~~~~~~~~ + +In addition to the command line interface for host command and a virtio-serial +interface for VM power policies, there is also a JSON interface through which +power commands and policies can be sent. This functionality adds a dependency +on the Jansson library, and the Jansson development package must be installed +on the system before the JSON parsing functionality is included in the app. +This is achieved by: + + .. code-block:: javascript + + apt-get install libjansson-dev + +The command and package name may be different depending on your operating +system. It's worth noting that the app will successfully build without this +package present, but a warning is shown during compilation, and the JSON +parsing functionality will not be present in the app. + +Sending a command or policy to the power manager application is achieved by +simply opening a fifo file, writing a JSON string to that fifo, and closing +the file. + +The fifo is at /tmp/powermonitor/fifo + +The jason string can be a policy or instruction, and takes the following +format: + + .. code-block:: javascript + + {"packet_type": { + "pair_1": value, + "pair_2": value + }} + +The 'packet_type' header can contain one of two values, depending on +whether a policy or power command is being sent. The two possible values are +"policy" and "instruction", and the expected name-value pairs is different +depending on which type is being sent. + +The pairs are the format of standard JSON name-value pairs. The value type +varies between the different name/value pairs, and may be integers, strings, +arrays, etc. Examples of policies follow later in this document. The allowed +names and value types are as follows: + + +:Pair Name: "name" +:Description: Name of the VM or Host. Allows the parser to associate the + policy with the relevant VM or Host OS. +:Type: string +:Values: any valid string +:Required: yes +:Example: + + .. code-block:: javascript + + "name", "ubuntu2" + + +:Pair Name: "command" +:Description: The type of packet we're sending to the power manager. We can be + creating or destroying a policy, or sending a direct command to adjust + the frequency of a core, similar to the command line interface. +:Type: string +:Values: + + :CREATE: used when creating a new policy, + :DESTROY: used when removing a policy, + :POWER: used when sending an immediate command, max, min, etc. +:Required: yes +:Example: + + .. code-block:: javascript + + "command", "CREATE" + + +:Pair Name: "policy_type" +:Description: Type of policy to apply. Please see vm_power_manager documentation + for more information on the types of policies that may be used. +:Type: string +:Values: + + :TIME: Time-of-day policy. Frequencies of the relevant cores are + scaled up/down depending on busy and quiet hours. + :TRAFFIC: This policy takes statistics from the NIC and scales up + and down accordingly. + :WORKLOAD: This policy looks at how heavily loaded the cores are, + and scales up and down accordingly. + :BRANCH_RATIO: This out-of-band policy can look at the ratio between + branch hits and misses on a core, and is useful for detecting + how much packet processing a core is doing. +:Required: only for CREATE/DESTROY command +:Example: + + .. code-block:: javascript + + "policy_type", "TIME" + +:Pair Name: "busy_hours" +:Description: The hours of the day in which we scale up the cores for busy + times. +:Type: array of integers +:Values: array with list of hour numbers, (0-23) +:Required: only for TIME policy +:Example: + + .. code-block:: javascript + + "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ] + +:Pair Name: "quiet_hours" +:Description: The hours of the day in which we scale down the cores for quiet + times. +:Type: array of integers +:Values: array with list of hour numbers, (0-23) +:Required: only for TIME policy +:Example: + + .. code-block:: javascript + + "quiet_hours":[ 2, 3, 4, 5, 6 ] + +:Pair Name: "avg_packet_thresh" +:Description: Threshold below which the frequency will be set to min for + the TRAFFIC policy. If the traffic rate is above this and below max, the + frequency will be set to medium. +:Type: integer +:Values: The number of packets below which the TRAFFIC policy applies the + minimum frequency, or medium frequency if between avg and max thresholds. +:Required: only for TRAFFIC policy +:Example: + + .. code-block:: javascript + + "avg_packet_thresh": 100000 + +:Pair Name: "max_packet_thresh" +:Description: Threshold above which the frequency will be set to max for + the TRAFFIC policy +:Type: integer +:Values: The number of packets per interval above which the TRAFFIC policy + applies the maximum frequency +:Required: only for TRAFFIC policy +:Example: + + .. code-block:: javascript + + "max_packet_thresh": 500000 + +:Pair Name: "core_list" +:Description: The cores to which to apply the policy. +:Type: array of integers +:Values: array with list of virtual CPUs. +:Required: only policy CREATE/DESTROY +:Example: + + .. code-block:: javascript + + "core_list":[ 10, 11 ] + +:Pair Name: "workload" +:Description: When our policy is of type WORKLOAD, we need to specify how + heavy our workload is. +:Type: string +:Values: + + :HIGH: For cores running workloads that require high frequencies + :MEDIUM: For cores running workloads that require medium frequencies + :LOW: For cores running workloads that require low frequencies +:Required: only for WORKLOAD policy types +:Example: + + .. code-block:: javascript + + "workload", "MEDIUM" + +:Pair Name: "mac_list" +:Description: When our policy is of type TRAFFIC, we need to specify the + MAC addresses that the host needs to monitor +:Type: string +:Values: array with a list of mac address strings. +:Required: only for TRAFFIC policy types +:Example: + + .. code-block:: javascript + + "mac_list":[ "de:ad:be:ef:01:01", "de:ad:be:ef:01:02" ] + +:Pair Name: "unit" +:Description: the type of power operation to apply in the command +:Type: string +:Values: + + :SCALE_MAX: Scale frequency of this core to maximum + :SCALE_MIN: Scale frequency of this core to minimum + :SCALE_UP: Scale up frequency of this core + :SCALE_DOWN: Scale down frequency of this core + :ENABLE_TURBO: Enable Turbo Boost for this core + :DISABLE_TURBO: Disable Turbo Boost for this core +:Required: only for POWER instruction +:Example: + + .. code-block:: javascript + + "unit", "SCALE_MAX" + +:Pair Name: "resource_id" +:Description: The core to which to apply the power command. +:Type: integer +:Values: valid core id for VM or host OS. +:Required: only POWER instruction +:Example: + + .. code-block:: javascript + + "resource_id": 10 + +JSON API Examples +~~~~~~~~~~~~~~~~~ + +Profile create example: + + .. code-block:: javascript + + {"policy": { + "name": "ubuntu", + "command": "create", + "policy_type": "TIME", + "busy_hours":[ 17, 18, 19, 20, 21, 22, 23 ], + "quiet_hours":[ 2, 3, 4, 5, 6 ], + "core_list":[ 11 ] + }} + +Profile destroy example: + + .. code-block:: javascript + + {"profile": { + "name": "ubuntu", + "command": "destroy", + }} + +Power command example: + + .. code-block:: javascript + + {"command": { + "name": "ubuntu", + "unit": "SCALE_MAX", + "resource_id": 10 + }} + +To send a JSON string to the Power Manager application, simply paste the +example JSON string into a text file and cat it into the fifo: + + .. code-block:: console + + cat file.json >/tmp/powermonitor/fifo + +The console of the Power Manager application should indicate the command that +was just received via the fifo. + Compiling and Running the Guest Applications -------------------------------------------- diff --git a/examples/vm_power_manager/Makefile b/examples/vm_power_manager/Makefile index 13a5205ba4..50147c05d3 100644 --- a/examples/vm_power_manager/Makefile +++ b/examples/vm_power_manager/Makefile @@ -31,6 +31,12 @@ CFLAGS += $(WERROR_FLAGS) LDLIBS += -lvirt +JANSSON := $(shell pkg-config --exists jansson; echo $$?) +ifeq ($(JANSSON), 0) +LDLIBS += $(shell pkg-config --libs jansson) +CFLAGS += -DUSE_JANSSON +endif + ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y) ifeq ($(CONFIG_RTE_LIBRTE_IXGBE_PMD),y) diff --git a/examples/vm_power_manager/channel_monitor.c b/examples/vm_power_manager/channel_monitor.c index 53a4efe45a..afb44a069e 100644 --- a/examples/vm_power_manager/channel_monitor.c +++ b/examples/vm_power_manager/channel_monitor.c @@ -9,11 +9,18 @@ #include #include #include +#include #include #include #include #include - +#include +#include +#ifdef USE_JANSSON +#include +#else +#pragma message "Jansson dev libs unavailable, not including JSON parsing" +#endif #include #include #include @@ -35,6 +42,8 @@ uint64_t vsi_pkt_count_prev[384]; uint64_t rdtsc_prev[384]; +#define MAX_JSON_STRING_LEN 1024 +char json_data[MAX_JSON_STRING_LEN]; double time_period_ms = 1; static volatile unsigned run_loop = 1; @@ -43,6 +52,234 @@ static unsigned int policy_is_set; static struct epoll_event *global_events_list; static struct policy policies[MAX_CLIENTS]; +#ifdef USE_JANSSON + +union PFID { + struct ether_addr addr; + uint64_t pfid; +}; + +static int +str_to_ether_addr(const char *a, struct ether_addr *ether_addr) +{ + int i; + char *end; + unsigned long o[ETHER_ADDR_LEN]; + + i = 0; + do { + errno = 0; + o[i] = strtoul(a, &end, 16); + if (errno != 0 || end == a || (end[0] != ':' && end[0] != 0)) + return -1; + a = end + 1; + } while (++i != RTE_DIM(o) / sizeof(o[0]) && end[0] != 0); + + /* Junk at the end of line */ + if (end[0] != 0) + return -1; + + /* Support the format XX:XX:XX:XX:XX:XX */ + if (i == ETHER_ADDR_LEN) { + while (i-- != 0) { + if (o[i] > UINT8_MAX) + return -1; + ether_addr->addr_bytes[i] = (uint8_t)o[i]; + } + /* Support the format XXXX:XXXX:XXXX */ + } else if (i == ETHER_ADDR_LEN / 2) { + while (i-- != 0) { + if (o[i] > UINT16_MAX) + return -1; + ether_addr->addr_bytes[i * 2] = + (uint8_t)(o[i] >> 8); + ether_addr->addr_bytes[i * 2 + 1] = + (uint8_t)(o[i] & 0xff); + } + /* unknown format */ + } else + return -1; + + return 0; +} + +static int +set_policy_mac(struct channel_packet *pkt, int idx, char *mac) +{ + union PFID pfid; + int ret; + + /* Use port MAC address as the vfid */ + ret = str_to_ether_addr(mac, &pfid.addr); + + if (ret != 0) { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Invalid mac address received in JSON\n"); + pkt->vfid[idx] = 0; + return -1; + } + + printf("Received MAC Address: %02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 ":" + "%02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 "\n", + pfid.addr.addr_bytes[0], pfid.addr.addr_bytes[1], + pfid.addr.addr_bytes[2], pfid.addr.addr_bytes[3], + pfid.addr.addr_bytes[4], pfid.addr.addr_bytes[5]); + + pkt->vfid[idx] = pfid.pfid; + return 0; +} + + +static int +parse_json_to_pkt(json_t *element, struct channel_packet *pkt) +{ + const char *key; + json_t *value; + int ret; + + memset(pkt, 0, sizeof(struct channel_packet)); + + pkt->nb_mac_to_monitor = 0; + pkt->t_boost_status.tbEnabled = false; + pkt->workload = LOW; + pkt->policy_to_use = TIME; + pkt->command = PKT_POLICY; + pkt->core_type = CORE_TYPE_PHYSICAL; + + json_object_foreach(element, key, value) { + if (!strcmp(key, "policy")) { + /* Recurse in to get the contents of profile */ + ret = parse_json_to_pkt(value, pkt); + if (ret) + return ret; + } else if (!strcmp(key, "instruction")) { + /* Recurse in to get the contents of instruction */ + ret = parse_json_to_pkt(value, pkt); + if (ret) + return ret; + } else if (!strcmp(key, "name")) { + strcpy(pkt->vm_name, json_string_value(value)); + } else if (!strcmp(key, "command")) { + char command[32]; + snprintf(command, 32, "%s", json_string_value(value)); + if (!strcmp(command, "power")) { + pkt->command = CPU_POWER; + } else if (!strcmp(command, "create")) { + pkt->command = PKT_POLICY; + } else if (!strcmp(command, "destroy")) { + pkt->command = PKT_POLICY_REMOVE; + } else { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Invalid command received in JSON\n"); + return -1; + } + } else if (!strcmp(key, "policy_type")) { + char command[32]; + snprintf(command, 32, "%s", json_string_value(value)); + if (!strcmp(command, "TIME")) { + pkt->policy_to_use = TIME; + } else if (!strcmp(command, "TRAFFIC")) { + pkt->policy_to_use = TRAFFIC; + } else if (!strcmp(command, "WORKLOAD")) { + pkt->policy_to_use = WORKLOAD; + } else if (!strcmp(command, "BRANCH_RATIO")) { + pkt->policy_to_use = BRANCH_RATIO; + } else { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Wrong policy_type received in JSON\n"); + return -1; + } + } else if (!strcmp(key, "workload")) { + char command[32]; + snprintf(command, 32, "%s", json_string_value(value)); + if (!strcmp(command, "HIGH")) { + pkt->workload = HIGH; + } else if (!strcmp(command, "MEDIUM")) { + pkt->workload = MEDIUM; + } else if (!strcmp(command, "LOW")) { + pkt->workload = LOW; + } else { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Wrong workload received in JSON\n"); + return -1; + } + } else if (!strcmp(key, "busy_hours")) { + unsigned int i; + size_t size = json_array_size(value); + + for (i = 0; i < size; i++) { + int hour = (int)json_integer_value( + json_array_get(value, i)); + pkt->timer_policy.busy_hours[i] = hour; + } + } else if (!strcmp(key, "quiet_hours")) { + unsigned int i; + size_t size = json_array_size(value); + + for (i = 0; i < size; i++) { + int hour = (int)json_integer_value( + json_array_get(value, i)); + pkt->timer_policy.quiet_hours[i] = hour; + } + } else if (!strcmp(key, "core_list")) { + unsigned int i; + size_t size = json_array_size(value); + + for (i = 0; i < size; i++) { + int core = (int)json_integer_value( + json_array_get(value, i)); + pkt->vcpu_to_control[i] = core; + } + pkt->num_vcpu = size; + } else if (!strcmp(key, "mac_list")) { + unsigned int i; + size_t size = json_array_size(value); + + for (i = 0; i < size; i++) { + char mac[32]; + snprintf(mac, 32, "%s", json_string_value( + json_array_get(value, i))); + set_policy_mac(pkt, i, mac); + } + pkt->nb_mac_to_monitor = size; + } else if (!strcmp(key, "avg_packet_thresh")) { + pkt->traffic_policy.avg_max_packet_thresh = + (uint32_t)json_integer_value(value); + } else if (!strcmp(key, "max_packet_thresh")) { + pkt->traffic_policy.max_max_packet_thresh = + (uint32_t)json_integer_value(value); + } else if (!strcmp(key, "unit")) { + char unit[32]; + snprintf(unit, 32, "%s", json_string_value(value)); + if (!strcmp(unit, "SCALE_UP")) { + pkt->unit = CPU_POWER_SCALE_UP; + } else if (!strcmp(unit, "SCALE_DOWN")) { + pkt->unit = CPU_POWER_SCALE_DOWN; + } else if (!strcmp(unit, "SCALE_MAX")) { + pkt->unit = CPU_POWER_SCALE_MAX; + } else if (!strcmp(unit, "SCALE_MIN")) { + pkt->unit = CPU_POWER_SCALE_MIN; + } else if (!strcmp(unit, "ENABLE_TURBO")) { + pkt->unit = CPU_POWER_ENABLE_TURBO; + } else if (!strcmp(unit, "DISABLE_TURBO")) { + pkt->unit = CPU_POWER_DISABLE_TURBO; + } else { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Invalid command received in JSON\n"); + return -1; + } + } else if (!strcmp(key, "resource_id")) { + pkt->resource_id = (uint32_t)json_integer_value(value); + } else { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Unknown key received in JSON string: %s\n", + key); + } + } + return 0; +} +#endif + void channel_monitor_exit(void) { run_loop = 0; @@ -555,6 +792,103 @@ channel_monitor_init(void) return 0; } +static void +read_binary_packet(struct channel_info *chan_info) +{ + struct channel_packet pkt; + void *buffer = &pkt; + int buffer_len = sizeof(pkt); + int n_bytes, err = 0; + + while (buffer_len > 0) { + n_bytes = read(chan_info->fd, + buffer, buffer_len); + if (n_bytes == buffer_len) + break; + if (n_bytes == -1) { + err = errno; + RTE_LOG(DEBUG, CHANNEL_MONITOR, + "Received error on " + "channel '%s' read: %s\n", + chan_info->channel_path, + strerror(err)); + remove_channel(&chan_info); + break; + } + buffer = (char *)buffer + n_bytes; + buffer_len -= n_bytes; + } + if (!err) + process_request(&pkt, chan_info); +} + +#ifdef USE_JANSSON +static void +read_json_packet(struct channel_info *chan_info) +{ + struct channel_packet pkt; + int n_bytes, ret; + json_t *root; + json_error_t error; + + /* read opening brace to closing brace */ + do { + int idx = 0; + int indent = 0; + do { + n_bytes = read(chan_info->fd, &json_data[idx], 1); + if (n_bytes == 0) + break; + if (json_data[idx] == '{') + indent++; + if (json_data[idx] == '}') + indent--; + if ((indent > 0) || (idx > 0)) + idx++; + if (indent == 0) + json_data[idx] = 0; + if (idx >= MAX_JSON_STRING_LEN-1) + break; + } while (indent > 0); + + if (indent > 0) + /* + * We've broken out of the read loop without getting + * a closing brace, so throw away the data + */ + json_data[idx] = 0; + + if (strlen(json_data) == 0) + continue; + + printf("got [%s]\n", json_data); + + root = json_loads(json_data, 0, &error); + + if (root) { + /* + * Because our data is now in the json + * object, we can overwrite the pkt + * with a channel_packet struct, using + * parse_json_to_pkt() + */ + ret = parse_json_to_pkt(root, &pkt); + json_decref(root); + if (ret) { + RTE_LOG(ERR, CHANNEL_MONITOR, + "Error validating JSON profile data\n"); + break; + } + process_request(&pkt, chan_info); + } else { + RTE_LOG(ERR, CHANNEL_MONITOR, + "JSON error on line %d: %s\n", + error.line, error.text); + } + } while (n_bytes > 0); +} +#endif + void run_channel_monitor(void) { @@ -578,31 +912,18 @@ run_channel_monitor(void) } if (global_events_list[i].events & EPOLLIN) { - int n_bytes, err = 0; - struct channel_packet pkt; - void *buffer = &pkt; - int buffer_len = sizeof(pkt); - - while (buffer_len > 0) { - n_bytes = read(chan_info->fd, - buffer, buffer_len); - if (n_bytes == buffer_len) - break; - if (n_bytes == -1) { - err = errno; - RTE_LOG(DEBUG, CHANNEL_MONITOR, - "Received error on " - "channel '%s' read: %s\n", - chan_info->channel_path, - strerror(err)); - remove_channel(&chan_info); - break; - } - buffer = (char *)buffer + n_bytes; - buffer_len -= n_bytes; + switch (chan_info->type) { + case CHANNEL_TYPE_BINARY: + read_binary_packet(chan_info); + break; +#ifdef USE_JANSSON + case CHANNEL_TYPE_JSON: + read_json_packet(chan_info); + break; +#endif + default: + break; } - if (!err) - process_request(&pkt, chan_info); } } rte_delay_us(time_period_ms*1000); -- 2.20.1