EVALUATION OF HEDIS SOFTWARE

Patricia M. Mine

Entisar Dahan

Mariano Petilla

March 2, 1998

The Health Care Business Environment

During the past two decades health care costs have increased rapidly. During the 1990s, the health care industry has been undergoing fundamental change in the way care is delivered and financed. Nationally, health care institutions are coping with growing pressure to reduce cost and improve productivity while maintaining high levels of service. A number of strategies to address these issues have developed including the formation of provider networks, the rise of managed care, the focus on quality improvement principles, and the increased use of technology for both business processes and clinical care.

In western Pennsylvania, health care has traditionally been financed by fee for service insurance. During the 1990's there has been a transition to various forms of managed care. Health insurance companies have shifted from the sole function of financing health care to a new role that adds clinical decision making, medical guideline development, facilitation of preventive care and early detection, responsibility for assuring access to care, and clinical oversight. Managed Care Organizations (MCOs) must not only pay for care, but also ensure that care is appropriate, proactive, available, and of high quality. The MCO is responsible for teaching its members to access care in new ways and to take an active role in the prevention and detection of disease. MCOs must also interact with physicians regarding clinical decisions, and patient service.

The managed care industry is highly competitive. Purchasers of health insurance (employers, the government, and individual consumers) frequently make decisions based on premium cost alone because assessing value is difficult. The "quality" of a health plan or of clinical care is an ambiguous concept that most purchasers do not know how to evaluate and find difficult to quantify. As costs have increased, large purchasers of health care have become concerned that the value of health care has not risen proportionately. Corporations which purchase care on behalf of their employees, and government which purchases care on behalf of seniors (Medicare) or the poor (Medicaid), have sought means to assess the relative value of managed care plans with which they contract. HEDIS was developed to facilitate purchasers in making decisions based on demonstrated value as well as cost.

Health Plan Employer Data and Information Set

Health Plan Employer Data and Information Set (HEDIS) was developed in an effort to compare the performance of health plans nationally and to support market based reform in health care. The original version (HEDIS1.0) was developed by the HMO Group (a coalition of group and staff model HMOs), a consortium of large corporations, and Towers Perrin. HEDIS 2.0, an enhancement of HEDIS 1.0, was the first set of HEDIS measures produced, sponsored, supported and maintained by the National Committee for Quality Assurance (NCQA), an accrediting body for managed care organizations. NCQA incorporates HEDIS measures into its rigorous evaluation of managed care plans' operations and organization.

HEDIS 3.0 is the third set of measures produced by NCQA since 1993. The intention is to promote higher levels of quality without excessive regulation. Comparisons are made in areas of critical importance to purchasers of health care and to consumers. HEDIS is used by large corporations to help guide their managed care purchasing decisions. HEDIS is continually enhancing its measurements and adding new measures in order to give the public increasingly comprehensive and meaningful information on plan performance.

HEDIS consists of a set of standardized performance measures. A mix of process and outcome measures is included with focus on health plan members' ability to function in their daily lives. Of equal importance is demonstration by health plans that they do what they can to conserve health, such as helping members to stop using tobacco. HEDIS measures address a full spectrum of health care issues from prevention and early detection to acute and chronic care. Measurements focus on children, adolescents, adults and seniors, and on conditions of high prevalence such as diabetes, heart disease and breast cancer. Private and public sector managed care organizations use the same measurement standards creating the possibility of comparing across populations as well as across health plans. However, if a MCO has more than one line of business, separate HEDIS studies must be performed for each line of business. For example, if an organization has a Medicaid plan, a Medicare plan, and a commercial plan, the plan must measure the rate of annual pap smears for each line of business.[1]

HEDIS measures are defined precisely and validated and have been successfully used by more than 330 managed care organizations. HEDIS 3.0 provides seventy-one measures in eight domains: Effectiveness of Care, Access/Availability of Care, Satisfaction with the Experience of Care, Health Plan Stability, Use of Services, Cost of Care, Informed Health Care Choices, and Health Plan Descriptive Information. MCOs choose to do measures that are important to a specific population based on utilization and disease prevalence statistics unique to the organization. When documenting a HEDIS study, the MCO must justify why the measurement was meaningful and important for the population. For example, the childhood immunization measure is more meaningful and important to a Medicaid population than a Medicare population because the majority of the membership of a Medicaid HMO is usually under eighteen years of age.

NCQA based its first annual report, The State of Managed Care Quality, on information obtained in Quality Compass. Quality Compass is a database of HEDIS and health plan accreditation information from 329 participating Managed Care Organizations. [2] Quality Compass enables NCQA to generate national and regional averages for various aspects of HMO performance. Health plan participation in Quality Compass is voluntary.[3]

There are many stakeholders in the effective use and reporting of HEDIS measures. Consumers benefit in several ways by the use of the measures. First, they benefit from the overall increase in quality effected by the competition that results from comparing quality measures. Second, consumers benefit because health plans use each measure as a baseline measure for quality improvement. Annually, barriers to improvement are analyzed for each measure and interventions are planned to improve the performance of the health plan. Health plans may ask, "What barriers keep children from being screened for lead poisoning by the age of one year? What can we do to ensure that all of our infant members are screened for lead poisoning by one year of age?" Corporations utilize HEDIS reports to select health plans to offer to their employees. Public purchasers of Medicaid and Medicare also use HEDIS to evaluate health plans.

Managed care organizations must report HEDIS measures in the accreditation process for NCQA. HEDIS reporting affects the viability of managed care organizations since purchasers of health care use HEDIS reports to select plans. Managed care organizations also use HEDIS measures to derive information, which is useful in making strategic decisions.

ABC Health Plan Organizational Background

ABC Health Plan (ABC) is a limited partnership formed in 1992 by General Hospital and Blue Cross Blue Shield in Pittsburgh, and The Health Plan in eastern Pennsylvania. ABC, an independent practice association Medicaid Health Maintenance Organization (HMO), received full three-year accreditation from the National Committee on Quality Assurance (NCQA) in 1996, an accomplishment that is uncommon among plans making their first application for accreditation. ABC is preparing for a second review in the fall of 1999.

ABC's current enrollment totals approximately 102,000 members living in 16 counties in western Pennsylvania. Medicaid recipients in Pennsylvania have the right to choose between Access, which is traditional fee for service medical assistance or one of three Medicaid HMO's in the region--ABC Health Plan, XYZ, or LNM. ABC is currently the only one of the three plans that is accredited by NCQA, therefore, accreditation is a valued marketing tool.

Medicaid MCOs are highly regulated by the State of Pennsylvania's Department of Public Welfare, the Department of Health and the Department of Insurance. Medicaid recipients may enroll or disenroll in a health plan on a monthly basis if they choose. If they disenroll from ABC, they may use Access or choose an HMO competitor. Medicaid recipients also lose their eligibility for Medicaid if their financial situation changes. Due to these competition and eligibility factors, enrollment in a Medicaid HMO is not stable.

Beginning in July of 1999, as a result of Pennsylvania's Health Choices mandate, Medicaid recipients in ten western Pennsylvania counties will be required to choose a Medicaid HMO. ABC is anticipating an addition of at least 75,000 members at that time. In addition, ABC has received permission to begin building a network of providers in seven central Pennsylvania counties and when the network is sufficient will begin to market the plan to Medicaid recipients in that region.

ABC's population is largely young women and children. Approximately 62% are female, 57% are under the age of 18, and 10% are over the age of forty. Ethnicity is 58% Caucasian, 31% Black, and the remaining 11% is Hispanic, American Indian, or other nationalities.

Analysis of the Problem

Use of HEDIS measures is critical to strategically positioning the organization for the future. Among its competitors in the region, ABC is the only Medicaid HMO accredited by NCQA. NCQA uses the measures as a major portion of the accreditation process. ABC is scheduled for a re-accreditation survey in 1999. In addition, the Pennsylvania's Health Choices mandate requires reporting of HEDIS measures.

ABC is presently conducting HEDIS measures of for Childhood Immunizations, Cervical Cancer Screening, Diabetes Eye Exams, Lead Screening, and Ear Infections. The Quality Improvement Consultant begins study by referring to the HEDIS manual for the criteria to be used. A request is made to the Information Services Department to query the managed care information system to produce the needed data. A typical IS request contains eligibility criteria (continuous enrollment for a calendar year, age criteria as of a specific date, gender), Primary Care Physician number, name and address, location by county, and ICD-9 diagnoses codes or CPT-4 procedure codes. Because Medicaid MCOs also use M.A. Fee Codes, and commonly use "home-grown" codes in addition to ICD-9's and CPT-4 codes, the codes specified in the HEDIS manual must be mapped to codes that appear in the database. After the request is reviewed by a Systems Analyst, the Quality Improvement Manager, and the Vice-President of Quality Improvement, it is prioritized and sent to the IS Department.

Upon receipt of the data (known as administrative data), the Quality Improvement Consultant must verify the accuracy of the data, sort and analyze the data, and report the data according to NCQA specification. As a result of capitation, many screening and preventive services are not captured in the managed care information system because physicians are not required to submit claims for routine services. Often it is necessary to pull a random sample of 411 members from the data in order to select medical records to be reviewed by an RN. This is known as "hybrid" methodology. After a nurse reviews the medical records data (medical records must be copied and sent to ABC or an RN must go to physician offices to review the records), the data must be entered in to a spreadsheet to be combined with the administrative data (pulled from claims) to calculate the rate of compliance. Reports of findings must then be formatted according to HEDIS specifications. If a health plan is participating in Quality Compass, all annual HEDIS reports must be sent to Quality Compass by June 1.

The following problems are inherent in the current process for producing HEDIS reports:

• Timeliness

1. The process of developing the IS request could take several days if the measure is one that the plan has not done before. Even if it is a repeat measure, the QI consultant must account for annual changes in the HEDIS criteria. Review of the request may take several more days or a week depending on how quickly responsible parties attend to the task.
2. The data request competes with many other requests of equal or greater importance to the organization and it may take months for IS to actually run the report.
3. It is not possible to contribute to the Quality Compass database because it is not feasible to compile all HEDIS reports in time to submit by the deadline.
4. The ultimate purpose of HEDIS reporting is to develop interventions to increase the rate of compliance for the following year. Since some of the reports cannot be completed until most of the year has passed, it is not possible to implement timely interventions. NCQA evaluates the timeliness of interventions in its survey.

• Communication

• Error

1. The process of mapping codes is subject to error because no one person seems to have expertise in all the possible codes.
2. Entering data from medical record reviews is subject to error

• Efficiency

1. Combining administrative data and record review data must be done using a spreadsheet.
2. Five people are involved in pulling the data.
3. Statistical analysis must be done by the end user.
4. Reports must be manually formatted.

• Raising the bar

ABC would expect to experience the following benefits from investing in HEDIS technology:

• Fewer employees required for pulling the data from the system.
• Decreased time required for producing study data and the report.
• Increased accuracy of data.
• Ability to participate in the Quality Compass database.
• More timely production of information would allow for more timely interventions to improve the quality of care.
• More efficient process for combining administrative and medical record data.
• Ability to conduct a greater number of measures due to increased efficiency and timeliness.

HEDIS Software Selection Process

The Information Systems Manager and a Systems Analyst from ABC were interviewed to determined requirements of a HEDIS information system. An evaluation form was used to rate each of the vendors on the requirements (see Attachment). Ten vendors of HEDIS software were contacted. Nichols Txen [5] and RIMS[6] were eliminated from consideration because the companies only sell HEDIS software as part of a managed care package. Evaluation of HPR was discontinued because the company was unwilling to give information about compatibility without confirming health plan affiliation.[7]

Four criteria were defined as being necessary for a system to pass the initial screening process. The system must be compatible with 1) Hewlett Packard 9000 server, 2) Windows operating system, 3) Pentium workstations, and 4) Oracle database. Any system for which these four requirements can not be met was eliminated.

The AMYSIS product did not interface with Oracle and was therefore eliminated by the initial screening process.[8]

Systems were then rated on database management, functionality, and vendor support. Vendors were rated 1 through a 5, with 1 representing the minimal satisfaction of requirements and 5 representing the extensive satisfaction of requirements. Each category was assigned a value equal to a percent of the total score. Weighted scores were totaled to calculate a total score for each vendor so that vendors could be compared objectively. Cost information was also gathered but not used to calculate the weighted score.

Database management was evaluated in four areas:

• Ability to archive/document the data quarterly to determine if interventions directed toward improving HEDIS measures are having the intended effect.
• Timeliness of compliance with HEDIS updates.
• Ability to map to Medicaid and home grown codes.
• Ability to secure access to the design mode.

• Accommodation of HEDIS hybrid methodology. Can the system select a random sample of eligible members for medical record review and can data be entered using forms to produce a report of aggregated administrative and hybrid data?
• Ability to analyze data to compare significance of change from year to year.
• Reports which adhere to NCQA formatting specifications.
• Ability to import data from disparate sources and to export to Excel for manipulation. For example, can the software import identification numbers of members who have used insulin and compare to continuous enrollment data in order to determine eligibility for the Diabetes Retinal Exam measure?
• Direct export to Quality Compass.
• Ability of users to access software simultaneously.

• Does the vendor provide user training? Is there a limit as to how many individuals they will train and do they charge per individual trained?
• What warranties are included?
• Does the vendor have technical support (on line help, phone access, documentation)?

Costs were not included in the overall scoring of the systems because cost is relative to value. A vendor was not eliminated from consideration because of higher cost.

After information was gathered on each vendor, ratings were totaled for comparison.

VIPS scored highest (92%) on rating of requirements. Marketing and Technical Representatives at VIPS were reluctant to send any literature, and although they answered our inquiries, they would not take the time to elaborate. However, since the product appears to meet requirements, this vendor merits further consideration.[9]

Momentum was rated second (90%). Momentum met all requirements at the highest level except for data analysis. However, the marketing representative stated that this function could easily be programmed into the system. Oleen, the developer of Momentum, has been involved with HEDIS since its inception. The Marketing Representative was readily available and knowledgeable. Momentum was promoted as a system that "walks the MCO through" the mapping process. Forms are used for medical record data entry. The system can be customized for the health plan. According to the representative, Momentum has people working full-time to program updates, unlike competitors. A drawback of Momentum is that it is a new product that has only been implemented by one organization.[10]

QMARK scored 80% in evaluation of requirements. QMARK does not export measures to Quality Compass and does not analyze data for statistical significance of data from consecutive years. User training is not provided because on-line help is available. Representatives from QMARK were helpful in providing the information we requested. Their product is considerably cheaper than the other products.[11]

Focal Point's compliance with requirements was 77%. The rule for compliance with HEDIS updates is April of the reporting year. This does not allow enough time for conducting the hybrid medical record review before the Quality Compass submission date of June 1. The vendor expects that mapping of codes will be done by the MCO. Written documentation was found to be confusing. Focal point does not have on-line or telephonic technical assistance but user training is available. Cost information was not available.[12]

O'PIN System's compliance with requirements was 73%. When contacting the organization we were transferred from Marketing Representatives to Technical Representatives and back to Marketing again. The web site was uninformative. The 800 number is not published in advertising or on the web site. We were not inclined to recommend further consideration of this vendor since customer service is likely to mirror the apparent disorganization we encountered when we called O'PIN.[13]

MEDSTAT Systems requirement score was the lowest at 65%. MEDSTAT supports data collection using hybrid methodology but did not describe how it supports the data collection. Data is updated in the system quarterly. Data analysis can be programmed. MEDSTAT cannot export data. The organization would not share cost information.[14]

Conclusions

Based on the above analysis, it is recommended that representatives from VIPS, QMARK, and Momentum be invited to give sales presentations and demonstrate software to the HEDIS Task Force from ABC. These organizations scored well on the analysis of requirements. Responsiveness to our inquiries and costs varied, but these characteristics can be more adequately assessed in a face to face interview. The final choice of a vendor will be based on consensus after consideration of the following issues:

1. Numeric rating of requirements relative to other vendors.
2. The Task Force's estimation of the vendor's commitment to meeting those requirements.
3. Added value features presented by vendor representatives.
4. Friendliness and functionality as determined by demonstration of the products.
5. Budget constraints and relative value when measured against other characteristics.[15]

Endnotes

[1] HEDIS 3.0, Volume 1, January 1997.

[2] The State of Managed Care Quality, http://www.ncqa.org/news/report.htm.

[3] Quality Compass 1997 Update, http://www.ncqa.org/qc/qcmain.htm.

[4] A Roadmap for Information Systems, Evolving Systems to Support Performance Measurement, HEDIS 3.0/1998.

[5] Michael Taylor, Nichols TXEN, 205-995-9898.

[6] Rims, http://www.rims.com, 1-888-841-3100.

[7] http://www.hpri.com/products/crms.htm, Health Payment Review, 1-800-522-6780.

[8] AMYSIS Managed Care Systems, Inc.,http://www.amcsi.com , 1-301-2251-8600.

[9] VIPS MC Source, http://www.VIPS.com/mcsource.html, 410-832-8300.

[10] Barbara Boykin, Oleen Healthcare Information Management, Momentum Software, 1-800-466-4760.

[11] HEDIS Help by QMARK, 1-800-947-5762.

[12] Focal Point 2.3 Version Offers New Enhancements to Users http://www.csc.com/about/news_stories/csc_focal.html.

[13] Terry Smith, O'PIN Systems, 1-800-888-2788.

[14] MEDSTAT HEDIS Solutions, http://www.medstat.com/products_services/mpmw.html.

[15] Friedman Charles P., Wyatt, Jeremy C., Evaluation Methods in Medical Informatics.

Go back to the Project Page