It seems that evaluating ECM efficiency is a tough task. However, there is one thing that makes this task solvable: evaluating ECM project efficiency is no different from evaluating any other IT project efficiency; while evaluating an IT project is no different from evaluating any other corporate project.
Efficiency of Obvious Things
Sometimes, a company can implement information technologies giving almost no thought to their economic efficiency. This may take place when an IT problem is so obvious that any other reasons are needless. For example, when a company buys a computer for a secretary or deploys an accounting system, it is just a matter of its technological maturity. Some technologies are an absolute “must have” for an efficient enterprise.
In other cases, economic effect is almost impossible to be measured, while the need for a technology is indisputable. Sometimes the measurement alone costs more than the technology itself (take unlimited Internet access for employees as an example).
ECM systems and ECM-based solutions are that sort of systems whose efficiency can hardly be measured. Here at Orienge, we have tried to solve this task and are proud to introduce a unique methodology for evaluating an ECM system’s economic effect.
ROI Calculation for ECM Systems
When we deal with an ECM solution implementation, the project monetary effect is difficult to be measured. The effect of such infrastructure systems is usually described qualitatively, using words rather than figures. “Reduced paper consumption” or “enhanced information transparency" would be the most typical descriptions.
What does economic effect mean? How is it expressed? Why is it needed? The simplest way to measure quantitative effect is Return on Investment (ROI) and its opposite – payback period calculation). ROI is the ratio of money gained on an investment relative to the amount of money invested. Payback period is the period of time required for the money gained to equal the total money invested (i.e. ROI = 1).
ROI helps us understand whether an IT project is going to repay or not. If a company makes a choice between several options (different projects or offers from different suppliers), ROI will show the most profitable one.
In ROI formula economic effect (expected monetary value) is the numerator, and total costs (value of investments) are the denominator. When it comes to IT projects, costs calculation is more or less clear: to get the Total cost of ownership (TCO) value, we sum up the costs of paid licenses, users and dministrators education, implementation/installation services, equipment, system administration and support throughout the project period. Measuring economic effect for such projects is much more problematic. Apart from qualitative (like “enhanced information security”) and obvious (like “reduced paper consumption”) effects, you may want to see in detail how the newly implemented IT solution affects the company and its processes.
ECM Solution Impacts on Company’s Characteristics
We have identified two classes of effects that can be brought about by an ECM solution. The first one is a direct economic impact, including increased proceeds (though such impact is not typical of ECM) or reduced expenses, in particular those related to internal processes.
The second class of effects is an impact on company's strategic indicators such as employee productivity, service quality and other classic Balanced Scorecard (BSC) indicators. Clearly, it is practically impossible to get accurate figures on IT project influence over "global" characteristics. Thus it is impossible to assert that, for example, “the financial stability of a company has improved by 12% as a result of an ECM system implementation.”
There are several reasons for this. First, there are hundreds of minor and major things that influence these characteristics. What you need is a complex multi-factor analysis of the changes, but it may cost you more than the system itself. Second, the changes introduced by the system and their impact on “global” characteristics will be spread over a long period. It means that “global” indicators should be evaluated not only at the end of a project, but on a regular basis, tracking and analyzing the improvements in progress.
However, we can generally state what “global” characteristics and indicators are going to be influenced by an ECM solution.
There are a lot of qualitative characteristics that can be more or less easily converted into quantitative ones. Reduced risks of document loss, facilitated contract drafting, and centralized document storage can be examples.
We have identified four qualitative effects that can be converted into quantitative ones:
- reduction of process cost (due to process reorganization or optimization);
- reduction of individual operation costs (searching and accessing data, or other common operations that can hardly be regarded as processes);
- mitigation of risks (including both their probability and consequences);
- reduction of material resources consumption (paper and other expendables).
A qualitative effect can be converted into several quantitative effects.
The effects can be conveniently grouped by business objectives. After all, that’s what ECM systems are implemented for. For convenience, you can identify some quantitative characteristics relating to a specific group of business processes (e.g. the number of documents, duration of a control operation, the average cost of C-level managers working hours, etc.). Such characteristics will provide a basis for measuring quantitative effect.
Converting Effects into Money
Since we are talking about economic efficiency, in order to convert the business impact into money we should find the difference between the costs before and after an IT solution implementation. Each type of effect requires its own approach to economic impact assessment.
Process cost reduction can be estimated via the process expenses using function value analysis and simulation modeling.
Operational economy can be estimated via reduction of operation complexity and therefore reduced labor input. Direct measurements or expert assessment can be used here.
Material costs reduction is usually easy to calculate. Just remember to take into account any chargeable services such as off-site document storage and retrieval; not just paper and other expendables costs.
Risks mitigation is difficult to assess. A company that has never faced the consequences of regulatory non-compliance or lost important documents, or a company whose long-term approval processes have never resulted in business deal cancellation, can hardly realize how much they can benefit from the system by reducing these risks. On the other hand, even if such incidents did take place, obtaining factual information on their consequences from the employees is quite problematic. Therefore, any estimates could seem exaggerated. But the very fact of estimation is positive, as it allows the company and its top managers to see the additional benefits of ECM project.
Other Issues of ECM Efficiency Calculation
When you define and measure the effects, remember to take into account their mutual influence. Typically, risks mitigation leads to an increase in process cost. An increase in operational cost can be caused by insufficient spending on ECM-related equipment. Or, conversely, an increasing amount of money and time spent on servers, workstations, administrators’ job may result in higher total cost of ownership.
Presently we have several ECM projects whose efficiency has been calculated using this technology. Here you may see how four types of effects make up the “global” financial effect in different companies solving different tasks.
Don’t forget this important matter. Sometimes the results of a preliminary assessment would show that the solution is not going to bring the desired effect or that you should review the project goals and scope. Sometimes tolerating a risk is easier than insuring against it by making the processes more and more expensive.
There is a question that should be answered: Who is supposed to carry out this evaluation? Ideally, the assessment should be done by the customer, as it ensures easy access to all required data and guarantees its confidentiality. To do this, a simple and efficiently designed methodology is needed, including libraries and formulas for converting qualitative effects into quantitative effects.
Using such libraries, we can evaluate IT solutions’ direct effects. Based on this information, we set project goals and tell our customers what “global” indicators are expected to improve. The results can be used to calculate ROI and find out which solution would be more effective.
As you may have noticed, the classic effects pointed out by ECM vendors are not always the most important. For example, material cost savings caused by contractual processes automation are minimal. The most significant effects are those related to risks mitigation and operational cost reduction.
However, we look forward to further developing the practice of evaluating project efficiency and gathering statistics that would help client companies to find a more sensible and balanced approach to project objectives and priorities setting.