Data Quality
Introduction
Validation
Data completeness
Latest Revisions
New Ways Refresh
New Publication Tables
Tables not included in this publication
The Data Quality Assessment project
Early 'New Ways' data
Acute Activity Comparison
Introduction
ISD, NHS Boards and the Scottish Government have put in place measures to help ensure the quality of the data held within the Waiting Times Data Warehouse. However, it is inevitable that any new data collection system, and especially one that represents as big a change as New Ways, will require time to settle down before NHS Boards' data is recorded nationally with full consistency.
The vast majority of the data have been judged fit for publication. Where the statistics are not yet finalised and may be subject to some change, they are considered to give a reasonable reflection of the current position and therefore to be useable by the public. In line with National Statistics requirements, as much as possible of the planned information is presented in this publication. Some data are not presented at this stage as they require further work to improve their reliability. Data previously published are updated in this publication to reflect on-going work to improve data quality. A description of known data quality issues is available here 
[88kb].
The process to assure the quality of the data began with implementation. The items held within the data warehouse have nationally consistent definitions 
[213kb]. Training was provided for the staff in NHS Boards involved. A New Ways website was created, making guidance readily available. Data submitted to the warehouse is subject to central validation. Within NHS Boards, great efforts were made to ensure that new systems and processes were working effectively. Finally, NHS Board Chief Executives were asked to sign off key statistics from the planned publication to confirm their accuracy.
The sections below describe in more detail aspects of the quality of the data held within the Waiting Times Data Warehouse.
Validation
Data provided by NHS Boards must pass central validation before they are accepted into the Warehouse and therefore available for analysis. The validation rules and processes are maintained centrally so that all NHS Boards are working to the same standards. Any records which fail validation are reported back to data providers with an indication of the error. The validation rules are under regular review.
Data completeness
A number of NHS Boards report that, due to issues with local systems, there may be some incompleteness in the records returned to the data warehouse at ISD. This is being addressed by ISD and the NHS Boards. Further work is required to understand and assure these data.
Latest Revisions
Waiting times figures that were published previously have been revised to reflect the latest available update of records held in the national (New Ways) data warehouse (as at 28 January 2011).
The national data warehouse is an 'operational', dynamic database that gathers records very frequently from hospital patient administration systems for those people waiting for care. It was introduced in January 2008 and reflects more rigorous and complex but fairer national rules about measuring patient waits, in particular dealing with patient unavailability without excluding such from published statistics. It is inevitable that such a big change affecting the large number of services and the staff managing them requires a settling in period in terms of the processing of information and before fully comparable Scotland-wide data are available. The revised statistics reflect improvements to the quality of published data.
Any changes in the total volumes of patients seen or waiting equate to a relatively small proportion of total numbers of patients seen/waiting. There is not yet a full understanding of the reasons for these changes, however the revised figures are considered to be largely a result of improved data quality. These improvements include: increased completeness partly arising from improved data validity; and impact of the submission of updated records (for example changing records that were previously classed as 'waiting' at the end of the quarter to being classed as 'being seen' during the quarter). In terms of numbers of patients shown to have waited longer than the national standard the revisions are extremely small in relation to total number of patients seen/waiting.
New Ways Refresh
The New Ways Refresh project aimed to help reduce the administrative effort on staff involved in collecting and monitoring waiting times data while still retaining the transparency of waiting times reporting, as intended by the original New Ways policy. It is hoped that the refresh project helps NHS Boards in monitoring the 18 Weeks Referral to Treatment standard.
The changes implemented through New Ways Refresh ensured that the majority of records enter the data warehouse, irrespective of data quality and addressing what has been a long standing issue for Health Boards. The refresh also introduces a new four 'flag' scale for records so users can quickly identify which records are free of errors right through to records that are not valid for waiting time calculations.
More details are also available here.
New Publication Tables
There have been some key changes to the Waiting Times publication since quarter ending 31 December 2009.
New Outpatient Tables:
The new outpatient tables have been updated to reflect the change in the national standard to include all referral sources. ISD have only published data for patients 'added to the waiting list' from 1 April 2009, one year prior to the introduction of the new standard. This new methodology prevents NHS Boards from having to quality assure historical data within the New Ways warehouse that applies to patients from over 12 months prior to the introduction of the new standard for outpatients; which was felt would be of 'disproportionate burden' for NHS Boards.
Figures for any patients added before 1 April 2009 have been provided by NHS Boards from their local records and are published in a separate table [26kb]. Data previously published for only GP/GDP referrals is available in the archive.
Table I3 - 2nd eye/one-stop cataract clinics:
A new cataract table for second eye and one stop cataract clinics has been added to the publication suite.
Additional changes:
All cardiac and cataract procedures have been removed from the Summary of Waiting Times tables (A tables) and all related tables and is published separately in the Cardiac and Cataract publication tables (Tables H and I).
Tables not included in this publication
Prior to the implementation of New Ways, ISD and Scottish Government agreed a list of proposed tables for publication. Some of these analyses have not been included in this publication because the relevant data items need more work to improve their reliability. The analyses not presented at this time are:
- Waiting times for patients who had a procedure at a return outpatient clinic. Quality assurance showed there to be significant under-recording, at this stage, of procedures undertaken at return outpatient appointment. This statistic has not been previously published;
- Time from GP referral to receipt of referral at hospital. This is a new statistic and further work is required to understand and quality assure the statistics;
- Waiting times for patients seen at a Rapid Access Chest Pain Clinic. This table depends on the accurate recording of a field indicating which national standard the patient was covered by. Analyses showed there was significant under-recording at this stage and some inaccurate recording of this field.
A full list of the data items collected [213kb] is published.
The Data Quality Assessment project
In 2008 ISD carried out a Data Quality Assessment of New Ways waiting times data on behalf of the Scottish Government, to assess the quality of New Ways data and to support NHS Boards with their own quality assessment processes. There were four strands to this project: a casenote review, an analysis of data from the national New Ways data warehouse, employee surveys and a patient survey.
The results from the project and recommendations to support the further improvement in the quality of data are reported in the project's Executive Summary [57kb]. Supplementary reports have also been produced for each of the four project strands and distributed to NHS Boards.
Early 'New Ways' data
Information derived from New Ways data for early reporting periods are considered unlikely to be fully consistent with data for later periods. This is due to two main reasons, arising from the transition to New Ways:
- New Ways was introduced on 1 January 2008 and it is inevitable that such a big change will require a settling down period before fully comparable Scotland-wide data is available;
- Patients placed on waiting lists prior to 2008, and seen or waiting in 2008, may not have full New Ways information recorded against them (e.g. CNA, DNA, unavailability and offer information). In addition, patients who had Availability Status Codes (ASCs) prior to 2008 were allowed a period of 'retrospective unavailability' to be applied, to cover periods prior to 2008 (it is not known how many sites applied such 'retrospective unavailability').
New Ways records can therefore be categorised into the following:
- 'complete' New Ways data (i.e. for all patients added to lists in 2008 and for some added to lists in 2007);
- incomplete or compensatory New Ways data (i.e. for many patients added to lists prior to 2008).
Acute Activity Comparison
Please note: there are apparent anomalies between figures published from 'New Ways' and Acute Hospital Care data sources. Some of the figures for Elective Admissions and New Outpatients in 'New Ways' appear to be lower than the equivalent information published on the Acute Hospital Care web pages . This is largely due to the use of different definitions between the two sets of figures. For example, Acute Hospital Care information includes non-waiting list cases which do not form part of many of the published Scotland figures in 'New Ways'. ISD are carrying out further detailed investigations into these differences.
