Repair Sequence - Conclusions and Recommendations

5.Conclusions and Recommendations

5.1. Conclusions

5.1.1 Phase I

Following are the key conclusions drawn from Phase I of the Project:

Both procedures tested resulted in similar emission reductions. The procedure later developed for the downstate program was adjusted to better target the most common problems first (as indicated by the Phase I results).
The results indicate that a wide variety of components must be addressed to properly diagnose and repair vehicles. Identification of all the problems contributing to high emissions is required, not just the most likely problem. In most cases, more than one emission related problem was found to be the cause of high readings. For example, in only 9 of the 42 cases where the catalytic converter was replaced was the catalytic converter found to be the only problem.
Achieving a high rate of compliance (such as greater than 95%) with final standards may be a challenge within a $500 repair cost limit.

5.1.2 Phase II Conclusions

As previously reported, following are the key conclusions drawn from Phase II of the Project:

The downstate diagnostic procedure was effective in identifying emission-related problems. Even though labor costs were included in the repair costs (as opposed to the upstate program that only included parts costs) a greater percentage of vehicles complied with final standards than in the upstate program (63% vs. 57%). Had there been full compliance with the procedure, the pass rate would have been higher. Still, it will be difficult to achieve a high rate of compliance (>90%) with final standards within the $500 cost limit.
Repair success rates by facility varied from 83% meeting final standards to 17%. The wide variation in performance by repair facilities indicates that training needed may vary greatly from facility to facility. Also, the ability and willingness to follow the procedure influenced the results (as evidenced by diagnostic/repair actions described in Appendix H and shop personnel comments summarized in Appendix I). Further, types and mileage of subject vehicles affected the success rate.
The transient emission test provides useful information for the diagnosis of emissions problems.
The catalytic converter should be diagnosed after basic fuel and ignition system problems are corrected. Vehicles that had high HC and NO_x emissions, but low CO emissions (indicating good fuel control), are good candidates for catalyst efficiency testing.
Using traditional criteria which requires replacing an Oxygen sensor can lead to a negative effect on vehicles that fail an emissions test for NOx only.
The $500 cost limit precluded catalyst replacement in some vehicles where such replacement may have result in further emissions reductions.

5.1.3 Status of the Repair Industry

The repair industry in the NYMA has routinely performed HC and CO emissions-related repairs for several years. These repairs were required for vehicles failing the idle mode emissions test. With more stringent cutpoints inevitable and the introduction of the NOx diagnostic and repair challenge, the industry faces the need for a quantum step up in knowledge and competence.

The survey conducted early in this Project indicated that approximately 40% of the shops employed technicians who had not received emissions related training in the past two years, and that 55% of the surveyed shops reported that they did not have a 4-gas analyzer. All of these shops were registered NYS Inspection Stations, indicating that they possessed emissions inspection equipment, but did not consider the equipment to be a diagnostic tool. The survey also indicated that there was a lack of other diagnostic and repair equipment available in many shops.

Considering the survey information and the fact that the enhanced I/M program will also test for NOx, using new and different equipment (the NYTEST system) to perform a transient test, the repair industry will have to move away from some traditional methods of diagnosing and repairing failed vehicles.

To accomplish effective emissions related repairs, it may be necessary to place more emphasis on technician training in areas of 5-gas theory and analysis including how to perform proper diagnostics and repair. It would be reasonable to suggest that those technicians, willing to hone their skills and develop a better understanding of how to use exhaust gases as a tool during the diagnostic process, will realize substantial benefits. These benefits will be in the form of a more accurate diagnostic process, whether attempting to correct a driveability or emissions problem, which often times can be one in the same.

5.1.4 Repairs in Cost-Effective Manner

Repairing vehicles that fail an emissions inspection, for the most part, can only be as effective as the diagnosis performed. Repairs performed on a failed vehicle should be based on information derived from correct and accurate diagnostics. Utilizing a "shotgun" approach rather than pinpoint diagnostics is time consuming and therefore costly. In order to diagnose emissions failures properly, a technician must understand what the measured exhaust gases are and what can cause them to be excessive. Absent this knowledge, the technician may perform diagnostics or repairs on systems or components that are not related to the failed gas. Effective diagnosis of an emissions failure or performance problem may rely heavily on the knowledge and training of the repair technician.

This training needs to include not only the knowledge required to repair vehicles but also how to implement that knowledge. As much as the inspection itself needs a defined set of procedures to enhance accurate test results, so too does the diagnostic and repair process. The technicians must be properly trained to employ the same disciplines in diagnostics (using a set procedure to ensure accuracy) as they do when performing an "official inspection".

5.1.5 Implications of Using NYTEST

The goal of the repair technician, results oriented auto repair, can only be achieved when the diagnostics performed are accurate. Results oriented auto repair requires the technician to take advantage of all the tools available to efficiently diagnosis and repair the vehicle. Some of these tools may be relatively new to the repair industry or may only be new to the technician. Using the information from the transient test and the transient test equipment during diagnostics can very often be as important as using a smoke machine to uncover vacuum leaks or a lab scope to read wave forms from sensors or actuators on a failed vehicle.

The technician, having the ability to observe when the vehicle exhaust gases were excessive, now knows when to perform diagnostics and just as important when not to. The knowledge received from training and experience is a key factor in the diagnostic and repair process. Knowing when to apply this knowledge can often be determined by the transient test information.

5.1.6 Emission Reductions

The Project confirmed that substantial emissions reductions can be achieved through proper diagnosis and repair, and that a majority of vehicles can meet final USEPA cutpoints. However, it is also certain that repair technicians cannot repair all vehicles to the final USEPA cutpoints within the repair cost waiver limit. It must be noted that such vehicles may still experience substantial emissions reductions.

Due to the small sample size, the individual reductions from the vehicles in this program can be considered negligible. However, the potential impact of these project findings, as they relate to the ability of the program to deliver planned reductions, could be substantial.

Appendix K contains estimated emission reduction benefits in tons per year as they relate to the 73 Phase II vehicles that participated in this project.

5.1.7 Repair Cost Limit Waivers

The inability of repair technicians to diagnose and repair all vehicles within the repair cost limit activates the repair cost limit waiver option for motorists who qualify. As the State tightens cutpoints, more vehicles will fail, resulting in increased pressure on the waiver system. It is recognized that USEPA's target maximum for waivers is 3% of the population of vehicles failing to meet standards.

Key variables in the waiver equation include cutpoints, repair industry/technician competence, the age and condition of the fleet and the then-current repair cost waiver limit. Recognizing the 3% "target," it is likely that the State will initiate and/or support proactive and constructive measures that would tend to limit the number of waivers issued. This implies a well-monitored and enforced set of rules pertaining to the issuance of waivers. The NYMA program can use its Data Management system to assess the degree of equity and compliance with its waiver systems and take corrective measures as appropriate.

5.2. Dissemination of the Recommended Sequence

5.2.1 Nature of the Tool

The Project Team, including the Department, evaluated several options for disseminating the diagnostic/repair advisory information, specifically, the diagnostic/repair sequence developed during the Project. During the course of the evaluations, other state's experiences were reviewed.

The Team noted that valuable advisory information has, in some cases, not been readily available to repair technicians, and/or was not considered to be user friendly by technicians. Accordingly, the Project Team sought to find effective, affordable ways to disseminate and promote the recommended sequence.

Understanding the usefulness and limitations of the "tool" and its proper place in the process, including the customer/technician relationship, was important. The Project team recognized that the diagnostic/repair sequence does contain logical recommended steps, that when properly followed, will result in correct and cost effective repairs. But, the sequence does not contain the long-sought-after "silver bullet" in terms of insuring repair effectiveness.

Very detailed, vehicle-specific diagnostic/repair advisory information, based upon the nature of the failure(s), can be and often is obtained from repair advisory Hotlines and certain electronic or printed information sources. The intent of this Project was not to develop detailed, vehicle-specific advisory information, but to create, test, refine and disseminate a mechanism that added discipline and order to the diagnostic and repair process.

5.2.2 Understanding the End User

To be most effective, the recommended sequence must be readily available to repair technicians. Technicians must have confidence that the sequence makes technical sense and is worthy of pursuit. Shop owners and managers must recognize the sequence as a useful tool that can lend order to the process and improve repair effectiveness.

However, the Team noted that, depending upon a shop's and/or a technician's economic incentives and motivations, the recommended sequence could be in conflict with the existing practices of the shop and/or technician.

For example, the Team recognized the difficulty in a technician or service manager educating a customer/motorist as to the need for and value of diagnostics prior to commencing repair, and the cost of those diagnostics. Some practicing technicians believe that simply replacing components minimizes the potential for a dispute with the customer, since most customers will not know the individual significance or effectiveness of component replacements. This culture and practice is potentially in conflict with an ordered, disciplined sequence to diagnostics and repair, and may be more evident when a technician is compensated, in part, on parts sales, versus repair effectiveness.

5.2.3 Options Considered

Four methods of disseminating the diagnostic/repair sequence were evaluated. They are summarized as follows:

A stand-alone software program could be electronically dispatched from the New York Data Manager that would then run on the NYTEST systems capable of running such a program and displaying and/or printing such information.
A stand-alone software program could be prepared for installation on a personal computer located in repair shops.
The sequence could be made available on the I/M Program Web Site (such Site required under the Data Manager contract between TESTCOM and NYSDMV). The Web Site could offer diagnostic links to other repair advisory web sites.
The sequence could become the centerpiece of a stand-alone technician training curriculum and/or be added to, or blended with, existing curricula being offered. A key element of this option would be the interpretation of the second-by-second drive trace information and its usefulness during diagnostics.

5.2.4 Program to Run on NYTEST Systems

The most comprehensive approach would provide a software program which will display the information on the NYTEST system. The program could be loaded into the NYTEST system either from the Data Manager, or from the I/M Program Web Site (maintained by the Data Manager), or from a floppy disk provided to the test and repair shop.

The software would begin by prompting the technician for the vehicle license plate. This information would be used to search for the latest inspection record for the specific vehicle. If the plate is found, the record would be retrieved and the vehicle and test results displayed. If the technician confirmed that this is the vehicle under repair, the software would display the first "generic" diagnostic screen. After the technician has completed the procedures, the software would proceed to the pollutant specific screen as indicated by the failing pollutants (NO_X, HC and CO).

If the inspection record is not found, the technician would be prompted to enter the year, make and model and the measurement results for the three gases. The sequence would then proceed as above.

At the end of the diagnostic sequence, a diagnostic record would be stored in a file. These records could be used for later review by the technician or the Department to assess the effectiveness of the sequence and provide a mechanism for further refining the diagnostic/repair sequence.

5.2.5 Stand-Alone Program for a PC

This approach would provide a software program that will display the information, downloaded from the I/M Program Web Site or loaded from floppy disk. The program would run under Windows 95 or NT.

The software would start by prompting the technician to enter the license plate, year, make and model, and the measurement results for the three gases. The software would display the first "generic" diagnostic screen. After the technician has completed the procedures, the software would proceed to the pollutant specific screen as indicated by the failing pollutants (NO_X, HC and CO).

At the end of the diagnostic sequence, a diagnostic record would be stored in a file. Similar to the above approach, these records could be used for later review by the technician or the Department to assess the effectiveness of the sequence and provide a mechanism for further refining the diagnostic/repair sequence.

5.2.6 Delivering Diagnostics Through the Web Site

Any repair technician with Internet access could connect to the I/M Program Web Site through his/her normal ISP account. Under this approach, the first time the technician accessed the site, (s)he would be asked to register. User activity and profiles could be supplied to the Department.

After logging into the site, the user would have the choice of going straight to repair advisory information suppliers, retrieving inspection results (including second-by-second data) or running the diagnostics/repair sequence.

Having the technician enter the vehicle identification number would activate the process. The results for the last inspection would then be retrieved from the Data Manager database. The vehicle information and readings would be displayed for the technician to confirm the vehicle under repair.

When the first generic page of diagnostics is displayed, it would include links to repair advisory information providers. The technician would be able to retrieve the test information from the third party supplier, return to the diagnostic page, save the place in the diagnostic sequence with a bookmark, and log off to perform the test. After performing the test, the technician could log back in and resume from the same point, or start diagnostics on another vehicle. At any point in the diagnostics, the technician can print out the page for later referral.

5.2.7 Training Curriculum

The diagnostic/repair sequence could become the centerpiece of a stand-alone technician-training curriculum and/or be added to, or blended with, existing curricula being offered in the NYMA. A key element of this option would be the interpretation of the second-by-second drive trace information and how it is used in conjunction with the diagnostic procedure.

When combining the sequence with diagnostic techniques utilizing the second-by-second trace information, it was believed that a stand alone training module would be needed in order to deliver to the technician the proper information required to fully understand 5-gas theory and a disciplined diagnostic sequence.

5.3. Recommendations

To the extent that additional funds become available and/or private sector initiatives materialize, there are natural and logical extensions to the work and results produced during the Project. They include the following.

5.3.1 Additional "Sequence" Testing and Refinements

The refined diagnostic/repair sequence developed during the Project could be further tested. Such testing could lead to further refinement of the sequence and/or better calibration of the method(s) by which the sequence is delivered and "sold" to the test and repair industry (including owners, managers and technicians).

Further testing could take the form of controlled application of the sequence in a real world I/M environment accompanied by careful monitoring of the results. A sample size of several hundred vehicles is recommended. Training of all technicians participating in the testing would be required, concentrating on the need to follow the sequence and to maintain records of the process.

In parallel, or alternatively, additional testing of the sequence could take place in a laboratory and/or academic environment, where the process is totally controlled through constant monitoring and supervision.

5.3.2 Repair Cost Waiver Limit QA System

Whereas other I/M programs have allowed the issuance of waivers for out-of-compliance vehicles after confirming that a "low emissions tune-up" was performed, application of the approved diagnostic/repair sequence would appear to be a more disciplined and potentially effective approach. The very nature of the sequence is to prioritize diagnostic procedures (and eventually repairs) based upon the excess emissions gas(es), noting where the excesses occurred in the drive cycle. There is a high likelihood of achieving substantial reductions early the sequence, even if the entire sequence can not be completed due to reaching the repair cost waiver limit. By their nature, most low emissions tune-up procedures are generic and may actually contribute little or nothing to reducing emissions.

To the extent that the sequence was mandated as necessary in the NYMA waiver application process, the Data Management System, working in concert with the PC-based NYTEST systems and cooperating repair technicians could provide a foundation for electronic participation and/or surveillance of the waiver process, thereby insuring increase quality and consistency.