Feb. 18, 2009
National Academy of Sciences
National Academy of Engineering
Institute of Medicine
Office of News and Public Information
'Badly Fragmented' Forensic Science System Needs Overhaul;
Evidence to Support Reliability of Many Techniques Is Lacking
WASHINGTON -- A congressionally mandated report from the National Research Council finds serious deficiencies in the nation's forensic science system and calls for major reforms and new research. Rigorous and mandatory certification programs for forensic scientists are currently lacking, the report says, as are strong standards and protocols for analyzing and reporting on evidence. And there is a dearth of peer-reviewed, published studies establishing the scientific bases and reliability of many forensic methods. Moreover, many forensic science labs are underfunded, understaffed, and have no effective oversight.
...with the exception of nuclear DNA analysis*, the report says, no forensic method has been rigorously shown able to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source. Non-DNA forensic disciplines need substantial research to validate basic premises and techniques, assess limitations, and discern the sources and magnitude of error; there has been little rigorous research to investigate how accurately and reliably many forensic science disciplines can do what they purport to be able to do.
For example, there is some evidence that fingerprints are unique to each person, and it is plausible that careful analysis could accurately discern whether two prints have a common source, the report says. However, claims that these analyses have zero-error rates are not plausible; uniqueness does not guarantee that two individuals' prints are always sufficiently different that they could not be confused, for example. Studies should accumulate data on how much a person's fingerprints vary from impression to impression, as well as the degree to which fingerprints vary across a population. With this kind of research, examiners could begin to attach confidence limits to conclusions about whether a print is linked to a particular person. In addition, the committee found no evidence that microscopic hair analysis can reliably associate a hair with a specific individual, but noted that the technique may provide information that either includes or excludes a sub-population.
In addition to investigating the limits of the techniques themselves, studies should also examine sources and rates of human error, the report says. As part of this effort, more research should be done on "contextual bias," which occurs when the results of forensic analysis are influenced by an examiner's knowledge about the suspect's background or an investigator's knowledge of a case. One study found that fingerprint examiners did not always agree even with their own past conclusions when the same evidence was presented in a different context.
There are great disparities among existing forensic science operations in federal, state, and local law enforcement agencies. The disparities appear in funding, access to analytical instruments, and availability of skilled and well-trained personnel; and in certification, accreditation, and oversight. This has left the forensic science system fragmented and the quality of practice uneven. Except in a few states, forensic laboratories are not required to meet high standards for quality assurance, nor are practitioners required to be certified. These shortcomings pose a threat to the quality and credibility of forensic science practice and its service to the justice system, concluded the committee.
Existing data suggest that forensic laboratories are underfunded and understaffed, which contributes to case backlogs and makes it hard for laboratories to do as much as they could to inform investigations and avoid errors, the report says.
The committee was not asked to determine whether analysis from particular forensic science methods should be admissible in court, and did not do so. However, it concluded that the courts cannot cure the ills of the forensic science community. "The partisan adversarial system used in the courts to determine the admissibility of forensic science evidence is often inadequate to the task... And because the judicial system embodies a case-by-case adjudicatory approach, the courts are not well-suited to address the systemic problems in many of the forensic science disciplines."
The committee also concluded that two criteria should guide the law's admission of and reliance upon forensic evidence in criminal trials: the extent to which the forensic science discipline is founded on a reliable scientific methodology that lets it accurately analyze evidence and report findings; and the extent to which the discipline relies on human interpretation that could be tainted by error, bias, or the absence of sound procedures and performance standards.
The report points out the critical need to standardize and clarify the terms used by forensic science experts who testify in court about the results of investigations. The words commonly used -- such as "match," "consistent with," and "cannot be excluded as the source of" -- are not well-defined or used consistently, despite the great impact they have on how juries and judges perceive evidence. Currently, failure to acknowledge uncertainty in findings is common. The simple reality is that interpretation of forensic evidence is not infallible -- quite the contrary, said the committee. Exonerations from DNA testing have shown the potential danger of giving undue weight to evidence and testimony derived from imperfect testing and analysis.
Certification should be mandatory for forensic science professionals, the report says. Among the steps required for certification should be written examinations, supervised practice, proficiency testing, and adherence to a code of ethics. Accreditation for laboratories should be required as well. Labs should establish quality-control procedures designed to ensure that best practices are followed, confirm the continued validity and reliability of procedures, and identify mistakes, fraud, and bias, the report says.
Strong leadership is needed to adopt and promote an aggressive, long-term agenda to strengthen forensic science, the report says. To achieve this end, the report strongly urges Congress to establish a new, independent National Institute of Forensic Science to lead research efforts, establish and enforce standards for forensic science professionals and laboratories, and oversee education standards. To ensure the efficacy of the work done by forensic scientists and other practitioners in the field, public forensic science laboratories should be made independent from or autonomous within police departments and prosecutors' offices, the report says. This would allow labs to set their own budget priorities and resolve any cultural pressures caused by the differing missions of forensic science labs and law enforcement agencies.
The recommended new National Institute of Forensic Science could take on its tasks in a manner that is as objective and free of bias as possible -- one with the authority and resources to implement a fresh agenda designed to address the problems found by the committee. The institute should have a full-time administrator and an advisory board with expertise in research and education, the forensic science disciplines, physical and life sciences, and measurements and standards, among other fields. The committee carefully considered whether such a governing body could be established within an existing agency, and determined that it could not. There is little doubt that some existing federal entities are too wedded to the current forensic science community, which is deficient in too many respects. And existing agencies have failed to pursue a strong research agenda to confirm the evidentiary reliability of methodologies used in a number of forensic science disciplines.
The report was sponsored by the National Institute of Justice at the request of Congress. The National Academy of Sciences, National Academy of Engineering, Institute of Medicine, and National Research Council make up the National Academies. They are private, nonprofit institutions that provide science, technology, and health policy advice under a congressional charter. The Research Council is the principal operating agency of the National Academy of Sciences and the National Academy of Engineering.
http://www8.nationalacademies.org/onpinews/newsitem.aspx?RecordID=12589
* -In "It Ain't Necessarily So; The Dream of The Human Genome and Other Illusions" ; New York Review Book; 2001(Second Edition) Richard Lewontin cited the 1992 "DNA Technology in Forensic Science" report of National Academy of Science:
ReplyDelete"..laboratory settings; stated succinctly, forensic scientists have little control over the nature, condition, form or amount of sample with which they must work."
Each forensic-science laboratory engaged in DNA typing must have a formal, detailed quality assurance and quality control program to monitor work...quality-assurance programs in individual laboratories alone are insufficient to ensure high standards. External mechanisms are needed. Courts should require that laboratories providing DNA typing evidence have proper accreditation for each DNA typing method used."
Since no laboratory currently meets those requirements and no accreditation agency now exists, it is hard to see how the committee's report can be read as an endorsement of the current practice of presenting evidence.
Because it is impossible or impractical to draw a large enough population to test directly calculated frequencies of any particular DNA profile much below 1 in 1,000, there is not a sufficient body of empirical data on which to base a claim that such frequency calculations are reliable or valid."
"The National Academy of Science's 2009 report indicated that scientific objectivity is routinely sacrificed to the perceived needs of law enforcement, and provided several hundred examples- and in the case of Phil Spector more than a dozen- of crime lab scientists tailoring their findings to fit the theories of police officers and prosecutors. It specifically criticized government witnesses for bias and "junk science" testimony."
ReplyDelete"The reader may want to take another look at the scientific evidence and ask whether guilt, in this case, was proven beyond a reasonable doubt", Dr. Lee comments.
" My feeling about the trial is that it went counter to what I have always believed is at the core of the American system of justice: a sense of fairness and truth. It appears to me that a full 162 days of trial consisted of little more than attacks on the victim, attacks on the suspect and attacks on the defense witnesses. Especially galling were references that expert defense witnesses (including Dr. Lee) were all "hired" guns. The amount of time and money the prosecution expended in trying to dig up "some dirt" by checking educational backgrounds, income tax reports, and other personal affairs far exceeded what was necessary to unearth the facts- to seek the truth."