D.2 Perform Post-Implementation Review
After the system is installed and operational and initial problems or
"bugs" have been resolved, conduct a post-implementation review.
The level of effort and depth of this review can vary. For example, the
data collection and analysis activity can be modified in relation to the
size and scope of the maintenance effort.
D.2.1 Determine Time for Evaluation
Do not hold this review immediately after system implementation; postpone
it until adequate and accurate data can be gathered regarding system qualities
and characteristics. Schedule the review after all users are thoroughly
familiar with the automated system and all of the software problems have
been resolved. This review should take place approximately 90 days after
implementation and again in six months.
D.2.2 Conduct Evaluation
Conduct the post-implementation evaluation using interviews of operations
personnel and user representatives. Review project development and operation
documentation. Determine the following:
- Does the system meet the stated needs of the user? Are those needs
still applicable? How much longer will the system meet those needs?
- Were the final project staffing and cost estimates within the original
estimates?
- Have requirements changed, making the hardware, system software, and
application software obsolete?
- Have revisions to the system been made only as necessary, and are they
being accomplished effectively and efficiently?
- Was the project well controlled and documented?
- Is the system responsive to users? Do the system functions meet user
expectations? Are the system interfaces, screens, and documents user friendly?
- Is the data in the system accurate and nonredundant?
- Are the controls (e.g., access, data validation, run-to-run) in the
operational system adequate?
- Can benefits be identified in specific categories (e.g., cost avoidance,
cost reduction, opportunity yield, direct revenue yield, intangibles)?
- Are cost savings being met?
- Have regulations or laws changed? Will these changes impact the volume
of work?
- Because of automation, have staff duties and procedures changed? Will
these changes impact the benefit/cost ratio?
- Does the system receive adequate support from operations personnel?
D.2.3 Formulate Recommendations
The objective of the review is to determine if the system has achieved
or surpassed the system's goals. If it is determined during the review
that the desired return on investment has not been achieved, develop recommendations
for achieving the desired return.
D.2.4 Document Evaluation and Recommendations
Document the review findings to reflect the pre- and post-automation
results. Develop documentation as a standardized means of capturing data
and providing a comparison of anticipated and actual benefits derived from
the automation effort.
D.2.5 Present Evaluation for Management Review
Forward copies of the review findings and recommendations to the management
officials responsible for acquisition, development, and implementation
of the system. While analyzing these results and recommendations, management
should ask the following questions:
- Has work flow improved since the introduction of automation? For example,
have unproductive or repetitive tasks been eliminated?
- Has the paperwork improved in such areas as appearance, format, usefulness,
accuracy, and completeness?
- If processing time has been and continues to be tracked, has this time
decreased?
- Is the new system achieving the estimated goals planned before the
new system was implemented?
- Has feedback from users and clients been evaluated and incorporated
to improve the effectiveness of the system?
*Last Modified: 11-19-01
alt="Go Up">
alt="Go Forward">