Changes to a company’s information technology (IT) environment, both information systems and the underlying platforms, are a source of significant operational risk for every organization. To protect its IT investment and reduce operating risk, robust change management processes are critical. The need for a positive control environment and a very unforgiving attitude regarding unauthorized IT changes by management cannot be overemphasized. Insufficiently tested IT changes is an unacceptable practice.
In fact, a comprehensive study by the IT Process Institute indicates that top-performing IT organizations outperform their counterparts by a significant margin on many different performance indices. Based on the assessment of specific practices that impact key performance measures, two controls stand out: monitoring systems for unauthorized changes and having defined consequences for intentional, unauthorized changes.
Internal audit’s role regarding the implementation of IT initiatives varies substantially but can provide a significant opportunity for internal audit to deliver real value to management and the board. In other words, internal auditors can provide independent and objective feedback that IT investments are being well-managed and that they are having a positive impact on the organization’s operation.
Effective management of an IT project is critical to its success. IT efforts continue to get more challenging and more complicated each year. Associated operational changes are also becoming more complex with each new technology implemented. The system integration complexities also grow each year. The bottom line — having IT initiatives, particularly the more significant ones, being independently and objectively assessed prior to implementation — is an imperative, prudent and recommended quality practice.
An audit of an IT initiative can involve varying levels of effort. At its simplest, the auditor can review the business case and hold a few interviews with key stakeholders. At its most comprehensive, a full-time audit team will participate in almost every part of an IT initiative. This diversity of effort depends on the risks involved and the assurance needs of the organization. If there is potential that the organization would fail should the IT initiative go bad, a comprehensive health “assessment” by internal audit can be extremely valuable.
WHY AUDIT YOUR IT INITIATIVES?
The board and management want to know many things regarding their IT investments, including that the IT initiatives are producing results. They want to know that IT activities are efficient and cost-effective and that the efforts of IT enhance and strengthen the organization’s system of internal control. Every organization wants to know that the next disaster from a failed IT implementation will not happen. Verifying that IT initiatives that make it over the justification hurdle deliver expected benefits within planned costs helps verify that investments are being made in line with the needs of the business. An independent and objective assessment of an IT initiative can provide that type of feedback while reducing future operational problems proactively.
HOW TO AUDIT YOUR IT INITIATIVES
The critical first step: Plan the audit well, that is, define your audit goals, objectives and scope. Internal auditors need to review and assess the overall project plan(s) and project management of the IT initiative. Auditors should perform the following tasks when assessing the accuracy and completeness of the proposed IT solution design and the data requirements for implementation:
- Evaluate the “build vs. buy” decision to ensure that the least expensive yet effective path is chosen.
- Evaluate the system development methodology chosen to make sure it is appropriate for the scope, technical platform and experience level of the proposed project team.
- Evaluate the clear assignment of roles and responsibilities within the project team and the establishment of relationships among internal teams, contractors, the IT department (operations) and functional users.
- Evaluate and monitor management’s project plans for various required system changes.
- Assess the completeness and appropriateness of management’s systems and database design, including security and privacy aspects.
- Verify and test the integrity of any data conversion process.
- Review the user acceptance and test planning and results to demonstrate successful end-to-end system operations and the preparedness for implementation.
- Review the startup of production systems and data to ensure that data integrity is maintained and “backout” plans in the event of a problem will be effective.
Auditors should also perform the following tasks when assessing the comprehensiveness and completeness of the startup of operational responsibilities within the organization for the new IT solution:
- Evaluate and monitor management’s project plans for various operational requirements.
- Assess the completeness and appropriateness of the operational policies and procedures that are developed, communications and management of user expectations, and the training that is planned to support implementation.
An organization’s IT initiatives can cover a broad span of processes, technologies and just as importantly, they can affect business operations in various ways. The audit planning phase needs to ensure the proper focus of the audit efforts. The IIA published a Global Technology Audit Guide (GTAG #12), which provides comprehensive planning guidance for auditing IT projects and is a highly recommended read.
Internal auditors should determine the level of their involvement and the best audit approach to take during the IT initiative’s initiation phase. Base the level of audit involvement on an assessment of the risk in the project and include factors such as the team’s project management experience, size, cost, and complexity of the initiative and effect on the organization if the initiative is delayed or worse, fails. The most appropriate audit approach is defined during the audit project planning phase.
Key issues to investigate during the audit include effective project sponsorship and project management (two absolutely critical factors in every IT project), the accuracy of the business requirements and their approval by sponsors and key stakeholders, representation of all stakeholder groups on the team and existence of an active IT risk management process.
Like most audits, the audit of an IT initiative generally involves three phases: planning, fieldwork and audit reporting. IT initiatives, however, do come in many shapes and sizes, so the audit of an IT initiative must also be flexible and risk-based.
PLANNING PHASE
During the planning phase, the internal audit team should ensure that all key issues are identified and considered, that the audit objectives will meet the organization’s assurance needs and that everyone involved understands the IT initiative that is being audited.
It is important that the audit focuses on evaluating the significant components of the IT initiative and use a risk-based approach to find the project elements most likely to fail or most in need of confirmation. During the audit planning phase, the appropriateness of the audit scope also must be determined. With many organizations now acquiring software package solutions, the entire software selection process and implementation effort have become very important activities for audit to evaluate.
FIELDWORK PHASE
In the fieldwork phase, the auditor analyzes the various components of the IT initiative based on the goals and methodology identified in the planning phase. Among the most important questions to investigate and confirm are:
- Have the business requirements been clearly defined and agreed upon?
- Do the projected costs include the cost to operate the new system/application after project completion?
- Have the requirements and financial cases been approved?
- Will the IT solution meet those requirements?
- Has the IT solution been proven (demonstrated) to work?
- Is the IT solution secure and will the privacy of information be maintained?
- Has the amount of effort involved reflected the risk involved with the solution’s implementation?
- Have results met assumptions about value drivers?
- Were the costs in line with estimates?
Audit tests should include reviewing business-case documentation and various system-related documents; interviewing key participants; looking at the training materials and procedures planned for the solution’s operation; and evaluating test plans, their results and management’s communications to employees regarding implementation.
AUDIT REPORTING PHASE
The audit reporting phase is where the internal auditor ensures that all stakeholders are informed of the audit results and management’s plans to enhance the IT initiative’s efforts. Audit reporting can be straight forward: As soon as possible tell them what you did, what you found and what management plans to do. If your audit reporting can be concurrent with the fieldwork, you may be able to influence the results. Try to be innovative in your style and creative with your communications.
When auditing an IT initiative, the audit feedback (recommendations) needs to begin as early as possible so that changes in project plans and efforts can be considered. Therefore, provide formal, ongoing feedback to the management of the IT initiative. Brief senior management and the audit committee with periodic status reports occasionally.
Formal end-of-audit reporting is still needed, but any “news” from the audit team must be conveyed long before the audit report is formally issued.
ORGANIZATIONS NEED TO ENCOURAGE IT PROCESS IMPROVEMENT
Auditing best practice would indicate that internal auditors are involved throughout an IT initiative’s life cycle not just in post-implementation evaluations where the wounded are shot. Early audit involvement has the additional benefit that management can obtain the views of the audit team at the front end of the project’s overall effort.
Make an internal audit of IT initiatives part of a broader IT audit plan as one audit does not assess the IT function’s overall performance. It is the long-term assessment of IT efforts where true IT process improvement can be encouraged. For example, does the organization have a robust IT risk management process? Is IT implementing comprehensive change management practices? Has the development and implementation process been updated to reflect today’s significant security and privacy requirements? Is there an organizational project management office?
Auditors can bring considerable value to an organization by evaluating both the IT and organizational aspects of an IT initiative. Because conversion to a new IT solution is one of the highest risks that an organization can face, internal auditors’ involvement and independent assessment of the issues and project plans will provide value far in excess of the audit’s costs.
CLOSING THOUGHTS
This blog post focuses on the auditing of an IT initiative throughout the life cycle of the project, focusing particularly on the planning, development and/or software acquisition, and testing and implementation stages. After enough time has passed, consider assessing the actual results to the expected results independently.
The gradual divergence between the actual state of the system and the required state of the system is another IT risk to evaluate periodically, including:
- Changes to regulations making the system noncompliant
- Changes to security threats making the system insecure
- Changes in business volumes, which undermine performance
- Changes in technology, which undermine the long-term viability of IT systems
You can learn more on this topic in our related tools on KnowledgeLeader: