What Does a CIO Expect from Employees?

What Does a CIO Expect from Employees?

Introduction

Performance metrics are measures of the performance and activities of an organization. They generally support a wide range of needs of the company’s stakeholders, including the employees, shareholders, and the customers. Conventionally, majority of the metrics used for assessment of performance within the organizations are finance based.

However, presently, many companies do employ certain metrics that focus heavily on company’s performance in delivering value and fulfilling customer requirements. The metrics that are used for project management, on the other hand, focus on assessing the progress of the project in six criteria – resources, cost, actions, quality, and scope. In any company, the development of performance metrics involves 3 key steps – (i) Identify customer requirements or the critical processes, (ii) Develop suitable measures, and (iii) Establish the targets against which the performance can be scored. Most often, these metrics are established in alignment with the strategic goals of the company, to assess the performance of process/project/product/activities against the principal CSFs (Critical Success Factors).

What Does a CIO Expect from Employees?

In an IT company, IT can be valued only when the enterprise, as a whole, is viewed as the ‘evolving ecosystem’. It is not possible for the CIOs to escape the management’s avowal on creating and measuring IT business value. But, despite having performance metrics that are quite sound academically, majority of the value equation in an IT company fail as the problem gets attacked backward by the conventional measures. In short, IT itself does not carry any value. It is the information it provides that is valuable. Thus, to make IT a part of value equations, it is essential to first ascertain value of all the information requirements.  Thus, strategic planning in IT can be managed only if there is an agreement and a clear understanding of the values involved. A CIO must, therefore, encourage his/her company to focus more on the value of the information, rather than on value of the IT.

Besides expecting the employees to focus more on value on information, the CIO also expects them to make sure the performance metrics are well aligned with the strategic goals and the overall mission of the company. In almost all the IT organizations, following are the main objectives that are identified to achieve sustainability –

• Ensuring availability of the current processing abilities
• Managing the costs involved in delivering the IT services and optimizing their value
• Making efficient use of the existing staff resources
• Ensuring a timely response is made to every business request made for new services/features.
• Ensuring the system changes are successfully implemented

The CIO must make sure that the employees establish the performance metrics as per the aforementioned objectives. The strategic goal of any IT company is to offer information processing capabilities, which benefit the business in the best possible way. To pursue this goal, a CIO expects the employees must be able to manage costs, prioritize business requests, and optimize value by:

• Operating and supporting infrastructure required to store, process, communicate, and secure information
• Supporting and operating business applications, which process information
• Providing technology planning, training, and consulting services.
• Employing, training, and deploying staff to offer all the required services
• Planning, developing or purchasing, testing, and implementing new software and infrastructure to establish improved capabilities for processing information and fixing problems.

Performance metrics, based on various data elements of different categories within an IT company, are reported as averages or graphs. Employees are expected to report these metrics to the CIO on a daily, weekly, or at least on a monthly basis so that trends across these periods can be assessed.

The IT Performance Metrics – A Brief Overview

The key IT performance metrics offer an immediate insight into the company’s authoritative data of the changing environments of the clients, levels of IT staffing, levels of IT investment, key costs involved, etc. the most commonly used IT performance metrics can be broadly categorized into 5 major areas, based on the 5 key categories of a typical IT portfolio, as described below –

Industry Performance Metrics: These include measures for industry-wise IT staffing and total IT spending at enterprise-level. These measures are important to understand the present-year results and the multi-year trends.

IT Infrastructure Metrics: These include performance, productivity, and unit cost measures for infrastructure environments of multi-year and the present year. The main infrastructure environments, for which metrics are established, include Mainframe, Desktops, Wintel Server, Unix Server, and Peripherals & Client. Help desk, VMware, network (voice and data) & telecommunications, and printers, etc. The metrics data for operations and infrastructure include the IT support and spending measures as well as the productivity & cost ratios for each domain of infrastructure technology, and help to assess the total owner cost for each unit and the total units supported by each environment support.

Applications Metrics: These include a wide range of metrics, such as staffing, cost, and project measures, ERP/SAP measures, and measures for life cycle phases, quality, and productivity. These metrics provide a deep in-sight into ERP/SAP spending, productivity and staffing metrics, distributions, and the total applications used by industry.

Information Security Metrics: These include all the enterprise-level staffing and spending measures related to information security, based on region and industry.

Outsourcing Metrics: These include all the enterprise-level staffing and spending measures related to outsourcing, based on region and industry.

Other important categories into which various metrics are categorized include financial management, change management, customer satisfaction, business alignment, projects & enhancements, and staffing.

Kind of Performance Metrics to Be Considered by the CIO – An In-Depth Analysis

Infrastructure Support & Operations

In any IT organization, it is very important to make sure that the current processing capabilities are easily accessible to the business. That is, the cost and performance information pertaining to the support and operations of the current applications and infrastructure of the organizations must be easily available to its CIO. The infrastructure metrics have been classified broadly into the following 3 categories, depending on the kind of activity evaluated by them –

Infrastructure Performance & Utilization: All the metrics under this category function on the basic principle that a very high utilization percentage and a low time of response are indicative of insufficient capacity, while a low percentage of utilization indicates excess capacity.

Infrastructure Availability: The metrics under this category assess availability of the entire non-desktop infrastructure at peak hours in business.

Infrastructure Incidents: This category includes both, system software and hardware metrics, which are used to measure the number and kind of non-desktop issues, which affect one or more users. These metrics also help evaluate the resolution time of every such problem and demand a stricter definition.

Some of the most important metrics under each of the aforementioned categories are listed in Table 1.

Table 1: Infrastructure Support & Operations Metrics

Following is a brief description of few important performance metrics that a CIO must consider for evaluating the infrastructural performance of desktops, servers, mainframe, help desk, network, telephones, VMware, printers, etc. –

Desktop Metrics:

Desktop performance is usually marked by the total amount of valuable work done by the system, as compared to the amount of resources and the total time utilized during the process. Good performance is typically characterized by high throughput, low response time, less resource utilization, high application availability, highly compact or quick data decompression and compression, less time for data transmission, and high bandwidth.

Some of the most commonly used desktop performance metrics include total ownership cost, LAN availability, total time between ordering and installation of PC, response time, easy availability, latency, service time, channel capacity, completion time, throughput, bandwidth, scalability, relative efficiency, performance/watt, speed up, and compression ratio. Besides, there are many benchmarks available for CPU and software performance assessment as well. The most commonly preferred CPU benchmarks include SPECfp and SPECint. Following are a few most important technical desktop metrics:

FLOPS: It refers to the total count of the floating point operation carried out in every second. It is vital in assessing the performance quality of desktops for carrying out scientific computations.

Instructions/Sec: This metric helps in selecting the right CPU or computer architecture for an existing software. Some users mistakenly evaluate CPU performance based on its operating frequency in MHz

Performance/Watt: In case of Parallel computers, performance of CPUs is best assessed by their speed/watt.

Interrupt Latency: A CPU with a deterministic response and less interrupt latency helps guarantee quick response in worst scenarios in real-time systems.

Besides, performance is also evaluated based on low weight, speed/dollar, and portable size, less power, instruction set, and minimum environmental impact.

Server Metrics

Some of the most commonly preferred Server Metrics in different categories are as follows –

CPU: Run queue length and occupancy, % total, % Wait I/O (WIO), % System (SYS), % User (USR)

Multi-CPU: WIO time/CPU, Sys time/CPU, WIO+SYS+USR, Mutexes Spins/CPU, Interrupts (INTR), number of inter-process cross-callings

Memory: PgFree/Sec, PgScan/Sec, Free Swap, PgIn/Sec, PgOut/Sec, Cache Hit Rates (CHR), and Free memory available

Process (at System Level): Execs/Sec, rate of process creation, Number of running processes, total Number of processes blocked, total number of processes waiting.

Process (single process): Name of the process, GID, UID, PPID, PID, total memory consumed, % utilization of memory (by process), RSS, % utilization of CPU (by process), run time of process, start time of process

Process (Only for Novell NRM): LAN traffic, disk throughput, disk space available, connection usage, ECBs available, memory available, server processes available, Usage of DS thread, Count of Abended Thread,etc.

Workload: Top-10 workloads by User, group, and system processes

Network: Network I/O, collisions, In-errors, Out-errors.

Logical/Physical Disk Performance: % usage, Mean wait time, the mean service time, Mean queue length, total writes/reads. Disk name, throughputs.

A number of similar server metrics exist for assessing performance of Capacity of File Systems, Users, Configuration, Services, Web Logic, Web Sphere, and AIX LPAR servers. There might be some variation in the aforementioned server metrics between Linux, Windows, Novell NRM, and UNIX systems. Also, the availability and use of these metrics depends on whether the system is agent-less (e.g. Net SNMP) or agent-based (e.g. “Up. Time” installed)

VM Performance Metrics

There are around 20 most important performance metrics when it comes to assessing the performance of VMware. These metrics are extremely useful in identifying the existing problems in the current VM environment so that the admin can take preventive actions to avoid and fix similar problems in future. They also help to identify the key CPU resources that are responsible for performance issues and bottlenecks in the system. Following are the some of the most commonly used VM performance metrics –

CPU Metrics: ‘cpu.usagemhz.average’, ‘cpu.ready.summation’, ‘cpu.extra.summation’. The last one in the list helps to assess the mismatch occurring between the speed of CPU and the I/O traffic. The second one assesses if VM has any ready problems, while the first one measures physical CPU usage at VM level.

Disk Metrics: Throughput (mean of disk.write.average & disk.read.average), ‘disk.queuelatency.average’, ‘disk.totallatency.average’, ‘disk.commandsAborted.summation’, ‘Disk.busresets.summation’

Memory Metrics: ‘mem.overhead.average’, ‘mem.consumed.average’, ‘mem.active.average’, memory swapping, and mem.vmmemctl.average

Network Metrics: net.usage.average, net.transmitted.average, net.received.average

Printers Metrics

Owing to rapid changes in print management, printer technology, and ways of printer usage, it is essential for the CIO to consider performance metrics that can help the organization assess the performance of their printers and manage this resource in an efficient manner. The three key benchmarks or metrics that must be included in the print strategies of any IT organization include –

• Number of printers required per user
• Total expenses on printers as % of total IT budget
• Printer expenses allocated/done per user

Besides these three printer metrics, it is also very important to analyze 4-5 years trends in printer expenses and usage. This will help optimizing printer costs in a better manner.

Network & Telecommunication Metrics

These metrics include response time and availability of network, especially during the peak hours. Another important performance metrics is the average time taken to repair the network. The various network activities on which all the network performance metrics are based are as follows:

Local Area Networks (LAN): Network activity in linking all computers within a single site

Wide Area Networks (WAN):Network activity linking different sites (at different locations)

Voice:Performance can be assessed based in telephone systems, voice provided/user, mobiles, and fixed-lines.

Network Perimeter/DC LAN: Metrics under this category evaluate performance of firewalls, protocol converters, internet caches, encryption, network gateways, DMZs, etc. in term of % interference, response time, etc.

Database Metrics

In the present scenario, the role of the Db Administrator (DBA) has become all the more important. To keep pace with the dynamic industry environment, the CIO must consider the following database metrics to evaluate the performance of database and the DBAs –

• Number of DBAs as % of total IT staff
• Storage (terabytes)/DBA

Both these valuable database metrics requires careful evaluation of the existing databases on the following parameters:

• Number of the applications to be always supported
• Number of existing databases & their respective sizes
• Total number of existing Db servers
• Database usage (OLAP, OLTP, ad hoc, data-mining, web-enabled)
• Number of DBMSs (Informix, Oracle, DB2, etc)
• OS platforms supported (AS/400, 0S/390, UNIX, Windows 2000)
• Use of ERP applications
• Number of the concurrent and usual users
• Nature of the SLA(Service Level Agreement)
• Required availability (24×7 or lesser)
• Impact of Db downtime in terms of dollars
• Applications type (mission vs. non-mission)
• Number of change requests and their frequency

Help Desk Metrics

Today, majority of the businesses use helpdesks to ensure regular customer interaction with regard to technical questions, complaints, and order placement. Thus, it is essential to ensure standard performance metrics are in place to evaluate the performance of customer representatives and the facility functions at the company’s call centers. The most commonly preferred helpdesk metrics are as follows –

Queue Wait time– Typically described in minutes, it refers to the average time that a caller experiences in queue/wait-line.

The Mean Talk Time– It refers to the total time spent in conversation on-line between caller and the call-executive.

The Mean Handling Time– It refers to the sum of wrap-up/hold time and the total talk time

Service Level %– It refers to the % of calls attended by the executive within an established time frame.

Calls made / hour– It means the total number of inquiries or calls made every hour.

Work After Call– Also referred to as Wrap-Up/Not-Ready/NR time, it is calculated as the total time taken to process a request without speaking to the customer put on-hold.

Other similar helpdesk metrics include 1^st Call Resolution, Number of Abandoned Calls and Idle time.

Applications Support Metrics

As per majority of experts in this field, applications are much more difficult than infrastructure to evaluate. A wide range of standards are used by today’s organizations to assess the performance of applications. Some of the most commonly used standards and benchmarks include rate of occurrence of defects, implementation of CMM levels, and rate (on-time completion) of the key function points. All these performance metrics have been categorized into 3 broad categories, as follows –

Application Performance & Utilization: These metrics help to assess the existing and past trends (in graphs) that are likely to affect or have affected the performance of applications. The key metrics in this category include mean response time of application and mean number of users per day.

Application Availability: These metrics help to assess the existing and past trends (in graphs) that are likely to affect or have affected the availability of applications. The key metrics in this category include percentage down-time and maximum peak downtime.

Application Incidents: These metrics help to assess the existing and past applications issues. The key metrics in this category include AIR Index (App. Incidents’ Resolution), Max number of users that got affected by 1 app incident, mean number of users affected, maximum and average resolution time/incident, and number of incidents.

Other Metrics to Be Considered by the CIO

Staff Resource Management: Number of open positions, Mean duration of these open positions, total number of contractors, Mean length of a contractor assignment, Mean staff & contractor cost, total number of staff, and staff utilization metrics (% non-working hrs), number of available man-days, etc.

Projects & Enhancements: Percentage of late requests, percentage of projects that exceed budget, number of backlog requests, number of completed requests, ECI (enhancement completion Index), etc.

Change Management: Number of emergency changes in apps made, number of infrastructure changes made, total number of project changes, total number of changes in enhancements, and change quality metrics (total down-time, percentage re-work done, etc.)

Financial Management: CPI (Cost-Performance Index), Percentage Variance, Annual budget, planned value, etc.

Delivering Value: customer satisfaction metrics, business alignment metrics, and metrics to assess continuous improvement

Thus, there are innumerable metrics in different categories that must be considered by the CIO to ensure the sustainability of his/her organizations in the present-day dynamic and competitive scenario. In today’s business world, to become agile, it is necessary to adopt a holistic view of the organization and ensure optimum performance at every level and segment of the business in an integrated manner.