Skip to main navigation

Enhanced I/M Program Evaluation Report - June 2006 - June 2008 - Report

A. EXECUTIVE SUMMARY

Biennial long-term program evaluations are required for enhanced I/M programs under 40 CFR Part 51.353(c)(1) of the federal Inspection/Maintenance (I/M) rule. Program evaluation provides a mechanism for I/M jurisdictions to evaluate the effectiveness of their programs, and if appropriate, to make enhancements to operating programs. The New York State Department of Environmental Conservation (DEC) has submitted four biennial program evaluations to the United States Environmental Protection Agency (EPA) in January 2001, November 2002, December 2004 and February 2007. This document retains a similar reporting format to these past submissions.

New York State has implemented two enhanced I/M programs, New York Transient Emissions Short Test (NYTEST) and New York Vehicle Inspection Program (NYVIP), to comply with federal I/M requirements. These programs effectively reduce the hydrocarbon, carbon monoxide, and nitrogen oxide emissions from applicable motor vehicles. This report details the overall progression of these enhanced I/M programs. The Department cites the following components that have led to the effectiveness of NYVIP and NYTEST programs:

1. New York implemented mandatory statewide (62 counties) I/M inspection requirements. The 53-county "Upstate I/M Area" complies with the Ozone Transport Region (OTR) low enhanced I/M performance standard noted under §51.351(h). The 9-county New York Metropolitan Area (NYMA) complies with the high enhanced performance standard noted under §51.351(f);
2. New York State requires motorists to perform mandatory I/M inspections on an annual basis and upon change of ownership;
3. Since May 2005, mandatory on-board diagnostic (OBD II) inspections have been required statewide. New York's on-board diagnostics testing was developed in accordance with EPA's final OBD implementation guidance and 40 CFR Parts 51 and 86. New York outlined the components of the OBD II-based NYVIP program in its March 2006 State Implementation Plan revision;
4. EPA approved the NYTEST transient test relative to the model transient I/M test, the "IM240." NYTEST received 95% of the available credit for hydrocarbon emissions, 99% of the available credit for carbon monoxide emissions, and 99% of the available credit for nitrogen oxide emissions.
5. The NYTEST I/M program has operated at "final" cutpoints since April 1, 2003;
6. New York State developed a comprehensive diagnostic and repair procedure to be utilized in conjunction with the NYTEST transient test to insure that failed vehicles receive effective, long term repairs;
7. I/M program data confirms that vehicles failing an I/M inspection receive repairs that last more than two years;
8. Through the combination of sticker-based and registration-based denial enforcement (RBE), New York's motorist compliance rate exceeds the minimum performance standard requirements for enhanced I/M programs. RBE was implemented in the 53-county Upstate I/M area in July 2007 (see Appendix A);
9. Administrative, sticker compliance, and NYTEST equipment audits are completed to ensure that New York's I/M programs maximize emission reductions; and
10. The New York City Taxi and Limousine Commission (TLC) operates an approved centralized, test-only OBD II inspection program. Applicable yellow medallion taxi cabs receive inspections three times a year at the City operated facility located in Queens, NY.

B. BACKGROUND

The Departments of Motor Vehicles (DMV) and DEC jointly administer the State's two enhanced I/M programs. Two programs are currently required within NYMA to reflect revisions to the federal I/M regulations. While onboard diagnostics inspections were contemplated in EPA's initial I/M regulations (1992), they were not required. The NYTEST tailpipe testing program was designed and implemented without OBD II testing capabilities. OBD II inspections were subsequently required for both the OTR low enhanced and high enhanced I/M performance standards and are now completed statewide in New York through the NYVIP equipment.

The NYTEST I/M program is limited to the 9-County NYMA, which geographically includes New York City, Long Island, and Rockland and Westchester Counties. The NYTEST program (1998) represented an enhancement to an existing idle testing program in place since 1981. The NYVIP I/M program features OBD II inspections and was phased into a statewide (62 Counties) program during the period of September 2004-May 2005.

There are presently approximately 3,500 and 6,400 licensed inspection stations operating within NYMA and the Upstate I/M areas, respectively. An overview of each I/M program is included in Table 1 below.

C. NEW YORK TRANSIENT EMISSION SHORT TEST (NYTEST)

Comparison of Registered Vehicles and Vehicles Emission Tested - Appendix B contains the statewide vehicle population numbers derived from the DMV registration database in March 2007.

The Departments are evaluating the past practice of estimating the number of required emissions tests based on registration data, and then completing a comparison to the actual number of completed I/M emissions inspections (as reported through the data manager). Several factors add uncertainty to this comparison:

1. The analysis must screen the registration file to exclude registrations that are exempt from emissions testing by regulation (i.e., exempt registration codes, vehicle age, vehicle weight, vehicle fuel type, or county of registration);
2. The annual inspection frequency differs from the biennial registration renewal cycle. The comparison must include a "look back" period exceeding 12 months to match inspections with a "fixed" registration file date;
3. The registration file represents NY's vehicle population on the day that the database query was completed. Some vehicles may have been retired from service since their last completed I/M inspection; and
4. By design, the NYVIP inspection software does not determine emission test applicability in the same manner as the older NYTEST requirements. The more recent NYVIP software determines a vehicle's model year and weight (actually light-duty determination) based on a detailed VIN validation sequence. The NYVIP decoded information is deemed more appropriate/accurate than similar fields contained within the DMV registration file.

The Departments will continue to review the registration file and I/M data for annual reporting purposes. Quarterly sticker compliance surveys will continue to provide for an independent assessment of motorist I/M compliance based on real world monitoring.

NYMA Sticker Compliance - DMV conducts quarterly sticker compliance surveys statewide. Appendix C contains the results of these surveys for calendar years 2006 and 2007. DMV's sticker compliance surveys show that the compliance rate for the combined NYTEST/NYVIP programs in NYMA exceeds 96%.

NYTEST Quality Assurance/Quality Control Checks:
NYTEST Analyzer Drift - New York State requires a seven day calibration period on its NYTEST analyzers. The adequacy of this time period was confirmed during a DEC NYTEST analyzer drift study completed in 2002.

NYTEST Equipment Audit Results - The DEC commenced limited equipment audits at NYTEST stations on March 21, 2001. This preliminary audit consisted of several analyzer sampling system checks (including the "S-tube," flex probe tip, filters, and sampling hose), an examination of on-board calibration gases (to insure they were BAR97 certified and within the labeled expiration dates), a gas cap pressure tester functionality check, and a check for the current approved version of software. The main purpose of these limited audits was to insure the overall reliability of sampling systems.

DEC subsequently required a NYTEST software update to allow for a more comprehensive equipment audit. This update mandated changes to include a gas injection audit and the transmission of calibration and audit data through the NYTEST equipment. DEC audit staff commenced the expanded equipment audits on August 1, 2001. The upgraded audit consists of:

1. a leak check;
2. a check of the sample system flow;
3. a gas analyzer audit with the ability to inject both audit gases and the on-board gases through the sampling system or though the calibration port;
4. a VMAS flow rate audit using a smooth approach orifice;
5. on-board calibration gas check (to insure they are BAR97 certified and not expired); and
6. a gas cap tester functional check.

Upon completion of a NYTEST equipment audit, the DEC auditor provides the station manager with a copy of the equipment audit form. This form lists the audit results and any supporting information. When the NYTEST equipment fails an equipment audit, the station manager is given a written notice. This notice states that the inspection station can no longer perform inspections until all necessary repairs are completed. Written documentation of such repairs must be submitted to DEC. If the inspection station fails to document repairs, DEC contacts the station. If proper repairs were not made, DEC requests DMV to place an administrative stop on the inspection station. This prevents the station from conducting further inspections.

DEC equipment audit results for 2006-2008 are presented below in the Table 2. Equipment audit criteria and additional data can be found in Appendix D. As a result of the subject NYTEST software update, DEC is capable of completing a desk audit of a station's weekly calibration without a station visit.

Repair Effectiveness

NYTEST Diagnostic and Repair Procedure - An essential component of any I/M program is to complete effective repairs on those vehicles that fail the required emissions test. In 1996, New York obtained funding under the Congestion Mitigation and Air Quality Control (CMAQ) program to conduct a repair effectiveness study for the NYTEST program. The main objective was to develop a comprehensive diagnostic and repair procedure for vehicles that fail the official NYTEST inspection.

The study design required that vehicles failing EPA's model IM240 transient test be repaired by qualified technicians using two diagnostic and repair procedures. The results from both procedures were evaluated and a single NYTEST diagnostic and repair procedure was finalized. The final procedure was printed on a laminated 2-sheet handout and mailed to all NYTEST inspection stations. A copy of this procedure can be found in Appendix E. Additional information related to the CMAQ repair effectiveness study can be found in the 2002-2004 biennial program evaluation report.

NYTEST Emissions Inspection Data

EPA Mobile Model Credit Determination - Appendix F contains the letter from the United States Environmental Protection Agency providing New York State with the following credit for the NYTEST emissions test:

95 percent of IM240 credit for hydrocarbon emissions
99 percent of IM240 credit for carbon monoxide emissions
99 percent of IM240 credit for nitrogen oxide emissions

Software Updates Based Upon Data Analysis - Program issues can be identified through station/inspector complaints, DEC/DMV audit findings, or through data analysis of emission test results. To correct issues or to enhance program components, DEC/DMV periodically requires the NYTEST equipment providers to upgrade the inspection software. Software modifications were outlined to the NYTEST equipment providers in November 1999. With the implementation of this software update, New York was able to conduct and record full equipment audits, capture and transmit detailed calibration records, and incorporate measures to prevent improper testing by placing functional limits on the analyzer and Vmas unit.

An optional NYTEST OBD II software upgrade was offered by two of the NYTEST equipment providers (ESP, SPX) prior to the NYMA roll-out of NYVIP. While NYTEST stations were not required to participate in this program, the OBD II pass/fail results of upgraded equipment were official. Approximately 570 NYTEST stations purchased the OBD II upgrade, and these stations performed approximately 187,000 OBD II inspections. The optional NYTEST OBD II was effective from July 2004 to May 2005.

Cutpoint Analysis and Implementation - Over time, DEC increased the stringency of the NYTEST cutpoints (transient and idle) as technicians became familiar with repair procedures. Final NYTEST cutpoints were implemented on April 1, 2003.

Effectiveness of NYTEST Repairs - To evaluate the duration of repairs, DEC queried the 2005 NYTEST database for vehicles meeting each of the following conditions:

1. Vehicles failed their initial NYTEST inspection;
2. Vehicles were repaired and repair costs were documented within the inspection record; and
3. Vehicles were re-inspected, and subsequently found to pass the NYTEST inspection.

This query yielded 35,938 vehicles. The VINs of these vehicles were then matched to the following year's database (2006) which resulted in 18,526 matches (i.e., inspection records for 18,526 of the initial 35,938 vehicles were found again). Similarly, the 2007 database was queried to identify vehicles that failed in 2005 that were again tested in 2007. The results are presented in the Table 3 which indicate that the majority of NYTEST repairs are still effective after two years.

NYTEST Emission Reductions

Emissions from Initial Test - DEC queried the NYTEST I/M database to calculate program-wide average emissions from the NYTEST transient (dynamometer) inspection for each calendar year using initial inspections as shown in Table 4 below.

Note that mandatory OBD II inspections through NYVIP commenced in NYMA in May 2005. So in calendar year 2005, many vehicles (light-duty, non-diesel/non-electric, model years 1996 to 2002) that were inspected by NYTEST in calendar year 2004 were instead inspected through NYVIP. This subset of vehicles represented the newest, generally cleanest, component of the previous year's (2004) NYTEST population. As such, the declining emissions trend observed for NYTEST transient testing from 1999-2004 was affected by the model year applicability of NYVIP OBD II inspections. Also, with each new calendar year, fewer vehicles will be subject to NYTEST inspection requirements due to the rolling 25 model year exemption and normal vehicle turn-over.

NYTEST Shared Network

As noted above, the demand for NYTEST tailpipe emissions inspections will decrease each year. The consequence of a declining NYTEST population is a corresponding decreased need for NYTEST equipment. Actual and estimated NYTEST volumes are displayed below in Table 5.

During the 2006-2008 reporting period, the Departments, the three NYTEST equipment providers, and several inspection station associations developed a concept to allow for the controlled reduction in the number of NYTEST-equipped stations in NYMA until the official end of the NYTEST program. The principle design feature of this "NYTEST Shared Network" is that participating NYTEST stations would enter into approved agreements for the orderly referral of motorists between nearby stations. The NYTEST shared network approach would allow some existing NYTEST stations to discontinue use of their NYTEST equipment provided that all regulatory requirements were met.

The NYTEST Shared Network is not mandated by the Departments, rather the licensed NYMA inspection stations would have the option of participating or not. The Commissioner of Motor Vehicles controls the size of the shared network to ensure that adequate geographic NYTEST coverage exists to maintain motorist convenience.

Revisions to the New York State motor vehicle regulations under 15 NYCRR Part 79 to provide for the NYTEST Shared Network were effective on September 24, 2008. The Part 79 revisions will be included within a NYMA SIP revision currently under development. The DMV press release and station informational packet associated with the NYTEST Shared Network are included under Appendix G.

D. NEW YORK VEHICLE INSPECTION PROGRAM (NYVIP)

NYVIP OBD II Implementation - New York State initiated statewide on-board diagnostic testing (OBD II) through NYVIP in 2004/2005. New York based its OBD II technical specifications in part on final federal guidance, "Performing Onboard Diagnostic System Checks as Part of a Vehicle Inspection and Maintenance Program" (EPA420-R-01-015, June 2001). The NYVIP inspection software establishes five possible OBD failure criteria:

1. The vehicle's Malfunction Indicator Light (MIL) does not illuminate when the ignition is in the key on/engine off (KO/EO) position;
2. The vehicle's MIL remains illuminated when the ignition is in the key on/engine running (KO/ER) position;
3. Inability to communicate with the vehicle;
4. The vehicle has commanded the MIL On and diagnostic trouble code(s) are reported; and
5. The vehicle fails the monitor readiness evaluation.

Combinations of these failure criteria are possible which requires close scrutiny when evaluating failure rate statistics. For example, a common OBD II failure would include both the "MIL on during KO/ER" visual inspection and the presence of a "diagnostic trouble code (DTC) with the MIL commanded on" criteria.

Upstate Sticker Compliance - DMV conducts quarterly sticker compliance surveys statewide. Appendix C contains the results of these surveys. DMV's sticker compliance surveys show that the compliance rate for the NYVIP program in the Upstate I/M area exceeds 96%. Registration-based enforcement for the 53-County Upstate I/M area was fully implemented during the reporting period (2007). A related press release is included as Appendix A.

NYVIP "Data Trigger" Audits - During the development of NYVIP, the Departments defined the contents (i.e., data fields) and reporting rules of the official electronic inspection record, the INSPREC.DAT file. This record is transmitted electronically from the NYVIP workstations to the NYVIP Program Manager and then made available to DEC and DMV. These records are the basis of New York's annual and biennial reports to EPA, and are utilized by DMV to insure motorist compliance with the annual inspection requirement via registration denial. The Departments also reserved portions of the INSPREC.DAT to capture vehicle and inspection specific information for inspector and station enforcement (auditing) purposes. A series of queries has been developed to identify possible fraudulent testing. These queries are periodically revised, most often to include additional screening criteria for a specific application. These queries were, in part, supported by an independent policy report completed by dKC-de la Torre Klausmeier Consulting, Inc. under contract to SGS TESTCOM in 2005.

The reporting of vehicle-specific OBD II data may vary according to vehicle make, model, model year, transmission type, engine size, etc. This data is reported to the NYVIP inspection workstations in response to standardized OBD II requests. The vehicle specific data is recorded along with inspection data (pass/fail criteria) for each NYVIP OBD II inspection. Under this design, certain data fields can be later used by the Departments to identify potentially fraudulent practices without requiring a station visit. The use of these "clean scan" queries have proven highly effective in targeting NYVIP stations and are frequently used as evidence in DMV enforcement hearings.

For EPA reporting purposes, the "Summary of OBD Inspections" query is completed according to any given time period. The NYVIP failure rates, waiver rates, and other statistics are reported based on the requirements within Section 51.366. The Departments continue to monitor the overall initial failure and waiver rate on a monthly basis.

NYVIP statewide trends related to the initial OBD II failure rate and waiver rate were completed using this query, and are presented in Charts 1 and 2, respectively.

Chart 1 tracks the NYVIP OBD II failure rate (for initial inspections) during the reporting period. The average OBD II failure rate has been fairly stable at approximately 6.4 % during the 2006-2008 reporting period.

Chart 2 shows an increasing statewide waiver rate beginning in December 2006. The Departments observed this trend. Through inspector education and increased enforcement, the waiver rate began to decline in the fall of 2007.

The waiver rate for NYMA and Upstate were 1.74% and 2.06%, respectively, in calendar year 2008. The NYMA waiver rate reflects waivers from both the NYTEST and NYVIP programs.

CHART 1 : NYVIP Statewide OBD II Failure Rate (Initial Inspections)

CHART 2 : NYVIP Statewide OBD II Waiver Rate

NYVIP Data Entry Sequence - VIN Validation and GVWR Determination - The NYVIP inspection begins with requests to obtain vehicle information. Required vehicle information includes the Vehicle Identification Number (VIN), make, model, model year, fuel type, weight, and area of registration. Based on this information, the NYVIP inspection software determines the appropriate inspection type. This information is also used in registration-based enforcement (RBE).

There are four possible methods of entering vehicle information: DMV's 2D registration bar code, DMV's 1D registration bar code, the vehicle's VIN plate located on the door jamb or window plate, and (least preferred) inspector manual entry using the NYVIP workstation's keyboard.

The NYVIP data entry sequence was revised during the reporting period to improve the quality of vehicle information used in the determination of inspection test type. The revised test sequence includes VIN validation and the relative determination of "light-duty" based on Gross Vehicle Weight Rating (GVWR). Upon entry of the VIN, the NYVIP software completes a validation based on 4 criteria. The VIN must be 17 digits in length, the last four digits must be numeric, "I", "O", or "Q" cannot be contained with the VIN, and the 9th character must pass a standardized VIN digit check. Should a given VIN pass validation, the model year is decoded from the VIN, and the inspector will not be allowed to make a model year change.

If a given VIN "fails" the validation, the inspection must be conducted online, and the subject (invalid) VIN is matched against the DMV registration database. If there is not a match, the inspection will be stopped, and the inspector will be required to enter a valid VIN.

Validated VINs are also compared to look-up table to potentially determine a vehicle's status as "light duty" or less than 8,500 lbs. GVWR. The NYVIP determination avoids the use of registered weight, which may or not be actual GVWR. The look-up table was developed by the Departments and consists of numerous VIN strings identifying vehicles with a GVWR of less than 8,500 lbs. Similarly, an inspector cannot make a weight change if a match is made to the look-up table. If a VIN validates, but cannot be matched to the GVWR look-up table, the NYVIP software instructs the inspector to enter the vehicle weight (GVWR) listed on the vehicle's door jamb.

The VIN Validation Flow Chart and a related NYVIP station message can be found in Appendix H. A sample of the GVWR table can be found in Appendix I. The Departments continue to evaluate the potential of additional VIN decoding checks to NYVIP inspection software to further improve the quality of vehicle information.

The structure of the 17-digit VIN is regulated by National Highway Traffic Safety Administration and the Unites States Department of Transportation under 49 CFR Part 565. The Departments provided comments on a Notice of Proposed Rulemaking to these regulations in November 2007. These comments can be found in Appendix J.

OBD II Communication - The Departments and the NYVIP Program Manager monitor the communication success rate for vehicles types (make/model/model year) subject to OBD II inspections. The NYVIP Program Manager is contractually required to report non-communication rates on a model name basis on a monthly frequency. Those vehicles identified as having high non-communication rates are flagged, and are subsequently investigated by the NYVIP Program Manager. These investigations are crafted to determine if non-communication is related to the functionality of the NYVIP scan tool or the design/operation of the vehicle's onboard diagnostics.

Most "non-comms" are identified during the first few months of mandatory testing. For example, MY 2006 vehicles were first OBD II inspected beginning on January 1, 2008. Most of the communication issues related to MY 2006 vehicles were identified by March 2008. Through this process, NYVIP identified communication problems with certain BMW, Saturn, Morgan, and Chevrolet vehicles. In several cases, the Departments have assisted vehicle manufacturers in their efforts to resolve vehicle specific issues.

Readiness Evaluation - Appendix D to EPA's final OBD II Inspection Guidance maintains a list of "Manufacturers Known to Have Readiness Issues." This guidance provides I/M jurisdictions with recommendations for vehicles with confirmed readiness related issues. New York was one of six I/M states that provided OBD inspection data to EPA to support continuing efforts to update Appendix D. As part of this workgroup, NYVIP inspection data supported several recommendations that additional vehicles be included within a revised Appendix D, while other vehicles require further evaluation. The Departments also provided technical assistance to several vehicle manufacturers in their evaluation of vehicles identified as having "high" initial readiness failure/rejection rates.

EPA's "I/M Transitioning" FACA WorkGroup - DEC participated in the FACA WorkGroup tasked to identify options available to states as tailpipe testing ends, and many I/M programs transition to OBD-only testing. The Departments collaborated in preparing final comments to the Work Group which are included as Appendix K.

E. NEW YORK CITY TAXI AND LIMOUSINE COMMISSION (OBD II)

Since 1977, yellow medallion taxi cabs operating within New York City have been subject to emissions testing at a frequency of three times per year. The New York City Taxi and Limousine Commission (T&LC) upgraded their Woodside (Queens) testing facility, and required OBD II inspections for applicable medallion cabs beginning on December 8, 2003. DEC and DMV completed acceptance testing of the test equipment and software at the TLC facility from August 2004 to June 2006. The Departments approved the TLC inspection program, which includes OBD II, emission control device (ECD), and safety component checks, as being equivalent to the New York State requirements. The joint DEC/DMV approval of the T&LC inspection dated October 4, 2006 is contained within Appendix L. The T&LC provides emission test results to DEC through the NYVIP Program Manager on a monthly basis.

The TLC OBD Summary Table for calendar years 2006/2007 is included under Table 6. The T&LC OBD II failure rate (initial inspections) for calendar years 2006/2007 is included as Chart 3. Due to mandatory TLC retirement requirements, some model years have very few vehicles in service. Note that some monthly "by model year" values are accurately reported as "0" counts.

CHART 3 : New York City Taxi and Limousine Commission (TLC) OBD II Failure Rate (2006, 2007)