A Metrics Generation Model for Measuring the Control Objectives of Information Systems Audit

Mathew Nicho And Brian Cusack

Sujets

• management
• risk management
• managers

Informations

• Publié par: Zawyug
• Nombre de lectures: 21
• Langue: English

Extrait

Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
A Metrics Generation Model for Measuring the Control Objectives of
Information Systems Audit
Mathew Nicho
Auckland University of Technology
[e-mail]
Brian Cusack
[e-mail]
fifth among the top IS issues facing the organisation. In Abstr act
another study conducted in the mid eighties by
Brancheau and Wetherbe [8] on the key issues in IS, Information Technology governance (ITG) which
measuring the effectiveness of information systemswas a relatively new concept in the late 1990s, has
ranked 9th on the list of the information systemsgained importance in the 21st century due to factors
managers and 4th on the list of general managers. At namely the collapse of Enron Inc, and the need for a
the turn of the millennium, Markus and her colleagues better reporting and financial disclosure system as
emphasised that system success is one of the most requested by the US Securities and Exchange
enduring research topics in IS [38]. In point “theCommission chairman in 2001. Subsequent
measurement of information systems success has been legislations namely the Sarbanes Oxley Act (SOX) in
on the research agenda for well over thirty years” [54: the United States and the Turnbull Guidance in the
p. 3]. A recent study conducted by Pricewaterhouse United Kingdom provided further impetus for the need
Coopers, sponsored by the IT Governance Institute for ITG. Other factors that prompt companies to give
(cited in [50: p. 8] on a sample of 7000 respondents, more importance to the management, control and
found that one of the top ten problems cited by these measurement of information systems include the risk
respondents was the “inadequate view of how well IT associated with information, the investments made by
is performing” and furthermore 80 percent are of the companies into the IT resource and the need to be
opinion that IT governance or some sort of governance competitive in the marketplace. All of these factors
mechanism was required to solve the issue.emphasize the requirement to measure the
Overall Information Technology (IT) spending is performance or effectiveness of information systems.
increasing at an alarming rate and it is estimated to be The state of performance of various entities, events and
about US $ 2.5 trillion in 2005 which is 50% of the total process of information systems give a ‘dashboard
corporate capital spend [34]. Taking into account the approach’ vision to management. In this paper a
importance of measurement in one of the largest assets metrics generation model is proposed for generating
(IS) in an organis ation, and the relevance of IS audit in metrics that can measure the key performance
the twenty first century, the authors took two models indicators and goals of the control objectives of CoBIT
namely the CoBIT and the GQM model from the IS field by applying the GQM model..
to generate metrics for information technology audit.
While CoBIT is an IT audit framework that had evolved 1. Introduction
over the last eight to ten years, the GQM is a metrics
generating model used in software engineering for
Measuring the effectiveness of info rmation systems
generating metrics to measure the various goal related
(IS) has been one of the top concerns of IS and
aspects of the software development process. The
corporate managers since the 1980s. A study
objective of this paper is to transpose the GQM model
conducted by Dickson, et al. [15] on the key
into CoBIT by taking the detailed control objectives of
information systems concerns, revealed that measuring
the CoBIT and follow the GQM guidelines for
and improving IS effectiveness/productivity ranked
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007 1530-1605/07 $20.00 2007 IEEE © 1Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
generating metrics that can measure the process and acknowledgement that the board is responsible and
entities that are audited during an IT audit exercise. accountable for the effective governance of its
information and IT assets” [29: p.1].
2. IS Auditing Framework
3. Attributes of IS Measurement
Strous [51: p. 264] defined IT audit as “an
independent and impartial assessment of the reliability, It has been stated by DeLone and McLean [13] that
security, effectiveness and efficiency of automated out of the large number of studies conducted in
information systems, the organisation of the information systems in the nineteen-eighties, half of
automation department and the technical and them relate to identifying the factors that contribute to
organisational infrastructure of the automated information system success. There is overlapping of IT
information processing.” IT auditing is a new auditing and information systems measurement
profession that extends the concept of control in the attributes, such that some of the attributes of IS entities
form of quality assurance, benchmarking and and process are similar. This is evident from the
measurement. It is also used in some organisations to definition of IT auditing proposed by the Dutch
implement IT governance. CoBIT as an IT audit Association of Registered EDP auditors who stated
framework addresses the need for management and that “an IT auditor assesses and advises on the
control of information and information technology [36]. following aspects of information technology:
It focuses on five IT governance areas namely, effectiveness; efficiency; exclusiveness; integrity” [51:
strategic alignment, value delivery, risk management, p. 265]. A review of measurement factors in information
resource management and performance management systems literature shows a similar set of attributes for
[28]. Tools such as the balance scorecard (BSC) for evaluating information systems success namely
IT/business alignment [22], maturity models for efficiency of resources and effectiveness of the users
benchmarking, key goal indicators (KGI) for measuring [26]. Attributes of IS measurement namely: precision,
the outcome, and key performance indicators (KPI) for accuracy and reliability of information quality;
performance measurement which are within the CoBIT completeness, relevancy, timeliness, and up-to-
framework lends a multi perspective approach to IT datedness of information contents; format, clarity and
audit. quality of the IS product [46: p. 110] defines
While the control and governance of information ‘exclusiveness’ and ‘integrity’ from various
systems and related technology were considered a perspectives. Both of the disciplines attempt to
subset of management information system (MIS) during measure attributes of information systems success.
the early stages of MIS development, measurement and DeLone and McLean [13: p. 61] in their study on
tested measurement however, are more generally found information systems success stated that “if information
in the field of software engineering. The late 90s saw systems research is to make a contribution to the world
the emergence of the concept of IT governance that of practice, a well-defined outcome measure (or
emphasised high level control rather than measures) is essential. It does little good to measure
‘management’ with emphasis on compliance, control various independent or input variables, such as the
and measurement rather than generic ‘management’ of extent of user participation or the level of I/S
IS. Business orientation is the main theme of CoBIT investment, if the dependent or output variable —I/S
[36]. Commenting on this, Kordel [35: p.1] states that success or MIS effectiveness—cannot be measured
“to be successful, the business side of an organisation with a similar degree of accuracy”. DeLone and
has to be involved in and committed to what IT does. McLean’s definition of success at the three levels also
To deliver the services the organisation requires IT has reflects the overlap of IT audit measures with IS
to be managed by the business as a business.” Apart measures. At the technical level success is defined as
from the above motives, other reasons for the accuracy and efficiency of the system (Shannon and
widespread adoption of IT audit framework include the Weaver, 1946, cited in [13]). At the semantic level
enactment of SOX in 2002 in the US, the Turnbull success is the information conveying the intended
guidance and the Combined Code in the UK. Australia meaning. At the effectiveness level success is the
has developed its own standard for IT governance, the effect of the information on the receiver. At a functional
AS 8015:2005 standard which “is the first formal level, Chang and King, [12] focused their efforts on
standard for IT governance. It has emerged and creating a functional scorecard for measuring the
recognises that the heart of IT governance is the performance of information systems, based on three
Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07)
0-7695-2755-8/07 $20.00 © 2007 2Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
system output dimensions namely system performance, multiple objectives. The financial aspects of software
information effectiveness and service performance. quality measurement, such as the return on in