The Value of Fast IT Systems

Most large organisations, especially financial institutions, have systems that are too slow and become the subject or complaints from its users. These issues can be a source of frustration within a business, with the complaints being the source of significant management attention. The focus, however, is usually on simply fixing the direct cause of immediate complaints rather than addressing a strategic solution for slow IT systems within the organisation.

The technical causes of the system being too slow can often be very complex to analyse and investigate. Modern IT systems include layers of client and server technology, with each layer connected by a variety of networks. The result is a complex web that can be difficult to understand why it is not performing acceptably.

On the other hand: I can go to my web search engine of choice and type in a search phrase. It will return back to me in seconds the results of searching millions of web pages for what I am looking for, including targeted advertising. I can start up my home machine and play 3D interactive games with other people across the Internet. These also require complex systems which make the difficulty of making the average IT system perform seem like the proverbial walk in the park. So why are business IT systems slow again?

I would strongly suggest that the key difference between these two sets of experience lies in an understanding of the true value of an IT system that performs well. To make sure that a system will perform well that performance need to be managed right from the start of the system's lifetime. If it isn't managed at the start of a system's lifetime then performance can be improved later on, but it will require significant work. All of this work incurs a cost, and requires management attention.

In the case of a web search engine or a 3D interactive game it is easy to understand that if the system doesn't perform well there will be no market for the product. This means that the original investment in technology will be wasted. It is clear from the start that the performance of the system will be important, and so projects to deliver these systems will be managed with performance high up on the priority list from the start.

For a typical business system the priority of the system performing well will typically be lower. The initial management attention will be on extending the functionality available, and making sure all the various development and delivery activities are synchronised appropriately. It is relatively rare for the initial requirements of a system to include suitable performance requirements, and so system performance doesn't get the necessary management attention. The result of this lack of value attention is often a slow system some time after the initial system implementation.

What is the value of an IT system that performs well?

There are a number of factors that can be considered when looking at how to value the performance of the system. If it is a financial value that is required, however, the first approach will usually be by calculating a value for the amount of time that system users are spending with the system. If a full time equivalent (FTE) of the time people spend waiting for the system to complete can be estimated, then this can be multiplied by the cost of employing people. When looking at this sort of analysis it is worth making sure that the full cost of employing those people, including fixed costs such as office space, is used.

As a simple example, let us assume that each of 5,000 users of a system perform an operation 10 times a day, and take 5 minutes a time. If the performance of the system were improved such that each user only had to spend 4 minutes doing the same thing then the effective saving could be almost £5.2M. Using the same approach, however, if a change only helped 20 users then the saving would account for much less than a single full time employee and the savings would be nearer to £20,000. In this case it could be argued that the business case for such a saving would be marginal if it required a significant amount of engineering work to resolve.

The value of the system performing well, however, might not be simply related to the cost of the time for people directly engaged with it. The 20 people in the second example might be engaged in creating new business. It this case it may be worthwhile considering the value of the business that they might generate in the time that was spent with the system, rather than the cost of their time.

The most important point is that the value of the system performing well needs to be considered, and a way to evaluate this agreed. It is critical that all of the key stakeholders for the system can understand the background of the valuation, since money will need to be invested to maintain this business value. The analysis will usually need to be performed by, or at least on behalf of, the business. It is unlikely that an IT department will be able to provide good quality figures for this sort of analysis unless they are supported by business management.

The management of performance

Having demonstrated that the performance of IT systems can be linked directly to business value, then it is clear that this value needs to be managed. The next issue is what needs to be in place to govern this system performance.

The key management activities are:

1. Defining the performance that is required in a Service Level Agreement, and gaining agreement that it is appropriate.

2. Making sure systems meet the proposed Service Level, and can continue to do so over time.

3. Regular monitoring of on-going performance against the agreed Service Level.

Senior management buy-in is required

Stated like this, the approach sounds relatively simple. There are, however, significant complexities that need to be overcome in achieving this. The hardest issues to are often not technical but involve achieving buy-in. In many businesses the current performance of systems is not monitored well enough to provide a measure of the cost being incurred in using the system, and hence in the value of performance improvements.

When starting, therefore, many of the figures being used will be estimated or even speculative. Thus, the likely performance improvement benefits will be similarly estimated or speculative. Against this background it is necessary to commit to spending in order to start the process of analysing the likely savings, and hence build a more reliable business case for the project. What is often known is the set of user complaints about system performance, but these do not make a business case for spending in themselves. This is likely to be the only basis on which management buy-in for a change in working practice can be gained. Without senior management visibility and backing a project or programme of improvement is likely to flounder before any measurable benefits are delivered.

How to agree the performance required

It is the nature of system performance in complex systems that it varies with time. That is, the same user doing the same thing and timing how long it takes will receive a range of significantly different response times. Thus an approach is necessary that takes account of this fact and allows the system's performance to be managed without fully understanding the detailed mathematics that are used to achieve the results.

The simplest solution here is to define a normal response time, and the response time which is acceptable for an upper band of responses. If it is acceptable for the system to be slower during peak processing times this may be added as a second clause. It is also important to recognise that functions can have very different response times, and so it is generally inappropriate to define all functions to have the same response criteria. It is more reasonable to set a general expectation for all data entry, and then specific values for key business functions.

Thus, the start of a service level agreement definition could read as:

1. The response time for interactive work should be 5.0s or less for 50% of responses, and 10.0s or less for 90%. During peak processing hours this may degrade to 7.5s and 15.0s respectively.

2. The production of a detailed audit report for a day is a complex activity, and may allowed to take 15.0s or less for 50% of responses and 40.0s or less for 90%. This report is generally run outside of peak processing hours and so no separate service level within peak hours is deemed appropriate.

In a real system there would usually be more response times that needed to be defined than this. As mentioned earlier, these response time requirements will need to be backed by a business case that defines the value of achieving these response times. The reason is that the more stringent the criteria the longer it will take and the more it will cost to meet them. When an IT department comes back and provides a range of achievable response times with associated costs it must be possible to make informed decision on which option will provide the best overall value.

This set of definitions also only covers half of the problem, since computer system performance is very dependent on the volumes being processed. Thus, it will be necessary to provide an idea of the level of demand on the system under which these measurements will need to be achieved. Many of the figures discussed above for the estimation of the value of improving performance, however, will be needed to define the processing volumes for the system. If these details are provided as part of the agreement process then the ability to work together with IT to agree the required performance will be much improved.

In agreeing the required performance there are likely to be a number of stages of proposal, discussion between key stakeholders and revision. This process is normal and should be planned for from the start. The whole process will be made much simpler, however, if there is a basic understanding by all parties that improving system performance does cost money but also delivers significant business value.

Making sure the system can perform

The optimisation and rework of systems until they meet an agreed service level is a core IT activity, and will not be covered in detail here. It is important to remember, however, that the changes needed are likely to take a while and employ both detailed testing and paper-based analysis. Initially the IT department may consider this sort of work, called Capacity Management, a specialist area and need to bring in outside help to ensure that the approach used will be sufficient to be able to deliver the requested benefits.

The governance of the project or programme needed to deliver the performance improvements should focus on making sure that the business benefit delivered is the focus. Thus there needs to be evidence that each stage of the project it is the performance in the long term that is being considered. An IT system will tend to degrade in its performance over time, so make sure that it is the performance of the system throughout its perceived lifetime that is considered. There may be a need to provide tactical fixes, but the work is not complete until strategic benefits are delivered along with a mechanism for making sure these benefits are monitored over time.

Monitoring system performance

The most important point here is that there must be a mechanism to allow the user experience of a system's performance to be measured. Again, the technology used to achieve this is a core IT activity and so is inappropriate to cover here. Ideally, the timing of every key user interaction in the system would be captured and analysed. This would be backed up by tool based monitoring of response times, which then improve the ability to analyse the cause of problems.

To provide useful information a format is needed that is closely related to the performance requirements statement. A simple tabular format could be used for the data, providing a count of the number of measurements exceeding the defined thresholds in each of a set of periods. For readability these should be presented as percentages rather than simple counts. It will then be possible to clearly see if the agreed Service Level is being met. An alternative approach is to calculate the response times that the defined percentage of people are experiencing. This format makes it easier to spot trends in the system's response times. It would usually be worth applying a "rolling average" to the raw data when presenting it, so as to flatten out any curve.

Conclusion

This article has demonstrated that there is significant business value in ensuring that IT systems perform well, and has outlined one approach to estimating this business value. The article has then examined the way in which the value of the performance of a system can be managed so as to ensure that the business benefits of fast systems are maintained over time.

The concepts and analysis needed to fully manage the performance of systems can be complex, and may need specialist help to set up and maintain. It can be surprising, however, how often the problem of poor system performance is considered to be purely an IT problem. The level of user complaints about performance can then be monitored, but the underlying value of having fast systems will often be ignored. Only once the business and IT within a company are working effectively together can the true value of fast IT systems be managed effectively.