Comprehensive reform to the federal criminal justice system has been a challenge—indeed, elusive. There have been numerous attempts to do so, dating back decades. But in 2018, under the leadership of President Donald Trump, a diverse group of stakeholders, and a broad bipartisan congressional coalition, the First Step Act (FSA) became law.REF

One of the main goals of the FSA was to improve criminal justice outcomes, “as well as reduce the size of the federal prison population while also creating mechanisms to maintain public safety.”REF Given the size of the federal criminal justice system and the number of stakeholders involved in changing such a complex system, implementing the FSA is a time-consuming and complicated process.

A key component of achieving those goals was the creation and implementation of a system to evaluate the risks and needs of each federal inmate. The FSA requires the U.S. Attorney General to “develop a risk needs assessment system to be used by the federal Bureau of Prisons (BOP) to assess the recidivism risk and criminogenic needs of all federal prisoners and to place prisoners in recidivism reducing programs and productive activities to address their needs and reduce the risk.”REF The tool is supposed to help correctional officials identify the risk and needs of each federal prisoner, make treatment referrals to the right program(s) for each person, and determine which prisoners might be suitable for early release from prison to serve the remainder of their sentences in home confinement or a halfway house.

Under a compressed time schedule, the Department of Justice (DOJ) announced its new risk and needs assessment system (RNAS), called the Prisoner Assessment Tool Targeting Estimated Risk and Need (PATTERN) in July 2019. The DOJ sought input from stakeholders and the public on PATTERN after it was released, and have since responded to that input (as discussed infra).

Risk and Assessment Methodology

Assessing the risk and needs for persons involved in the criminal justice system is not new. Every state and the federal government use risk and/or needs assessments, and have done so in varying degrees for decades. Risk and needs assessment tools, of which there are many, are not infallible. How they are constructed and what they measure determine, in large part, how efficacious they are. The best risk and needs assessments tools include validated, accurate, race-neutral actuarial predictive methods across a variety of criminal justice risk outcomes. Different tools are necessary for different populations, as the tools themselves are not ubiquitous.

PATTERN was developed in a compressed time frame and informed in large part by the DOJ’s already existing and well-performing assessment system, the Bureau of Prisons (BOP) program called Bureau Risk and Verification Observation—Recidivism (BRAVO-R), which is used to assess the risk of prisoners being re-arrested or returned to the BOP within three years of being released from prison.

PATTERN, like any RNAS, is a work in progress. The DOJ has vowed to, and is required to, continue refining PATTERN. It will take years, if not a decade or more, to evaluate whether it is effective. To be effective, PATTERN should be calibrated to fulfill the dual purpose of the FSA: reduce the federal prison population and maintain public safety.

Fortunately, there is a massive body of literature on risk and needs assessments and a plethora of experts in this field who have published widely. Several of the most utilized tools have been around for decades, subjected to peer review and scrutiny, and have evolved with the times. Research is ongoing, and the DOJ should be guided by objective findings from experts in the field, as it has been so far.

To date, the DOJ has been prudent in the creation and implementation of PATTERN. The DOJ held three public listening sessions and invited written criticism about PATTERN. Some of the criticism has merit, and the DOJ would be wise to incorporate those suggestions into PATTERN over time. Much of the criticism, however, lacks merit, and should be rejected, for the reasons discussed below.

Furthermore, although the DOJ has developed and published the risk assessment side of PATTERN, it has yet to publish the new needs assessment suite of tools. Instead, the DOJ has merely identified areas to expand upon in the current system and issued projected timelines for the completion of the new and improved needs assessment system. As such, we are not able to evaluate the pros and cons of the needs assessment system.

As time passes, and PATTERN evolves, the biggest challenge for the DOJ will be to resist calls to change the program under PATTERN into a tool that serves the political goal of reducing the federal prison population in and of itself—at the expense of public safety—while, at the same time, refining PATTERN into a 21st-century risk and needs assessment tool befitting the lofty goals of the First Step Act.

Understanding Risk and Needs Assessments

The year is 2054. In Washington, DC, the murder and violent crime rate is zero because advanced technology allows the police to know which people will commit crimes in the future. Armed with this information, the pre-crime unit arrests and convicts people before they actually commit a crime.

This is the premise of science fiction writer Philip K. Dick’s book, Minority Report, which was later made into a popular movie starring Tom Cruise. We are a long way—if ever—from being able to predict whether a person will commit a crime in the future and arresting him before he does so. For now, stakeholders in the criminal justice system have a much less predictive, but important, set of tools that are applied to persons already involved in the criminal justice system; these tools are generically called risk and needs assessments.

Risk and needs assessments have a more than 40-year history in probation and parole supervision.REF The roots of using risk-assessment tools to predict offender behavior go back even further, almost a century.REF The first efforts to use actuarial methods to predict parolee success as part of parole release decision making originated in Illinois in the 1920s.REF

Most advancements in managing individuals in the criminal justice system begin with a discussion of the need to adopt a standardized risk and need assessment (RNA) tool.REF The use of RNA tools are generally recommended at all decision points along the criminal justice system—from booking to pretrial release to sentencing to release from prison or jail to services.REF Standardized RNA tools, in theory, offer a science-based approach to regulate decision making to avoid or minimize biases, decrease unnecessary discretion, improve proper use of resources, and/or increase fairness.REF

There are a variety of risk and needs assessment tools in use across the United States and around the world. Each has its own strengths and weaknesses. Risk assessments are made for the extended supervision and continuing detention of high-risk sex, terrorism, and violent offenders to ensure the safety and protection of the community and to promote their rehabilitation.REF

The history of risk assessments shows a trend toward increasing complexity reflecting trends in both research on the causes of criminal behavior and the development of statistical methods, particularly as influenced by the availability of inexpensive computing power.REF Risk-assessment methods were developed largely piecemeal; efforts to clarify the history have applied the post hoc categorization into four generations.REF

First-Generation RNAS Tools

The first generation was typified by informal procedures, such as a clinical assessment based on expert judgment.REF They involved justice actors, including probation officers, judges, and parole boards, making decisions using information available about an individual.REF In this first generation, the approach was to rely on clinical and professional judgments that did not have explicit or objective scoring rules.REF

Typically, a probation officer would interview the offender and review available reports and files. Based on that information, the officer applied expertise gained through experience to make a judgment about the level of risk that offender posed.REF The assessment was idiosyncratic, opaque, and largely unreviewable.REF

First-generation RNAS tools dominated institutional and community corrections for several decades and are still preferred by many decision makers.REF The interviews of subjects were unstructured or semi-structured, and thus were dependent on the quality of the staff and their ability to use the information gained in an appropriate manner.REF

Naturally, the concerns about this approach were that it was subjective and depended in large part on the acumen of the interviewer, the thoroughness of his questions, and any biases he may have had.REF These concerns called into question the reliability of the assessments themselves.REF

Second-Generation Risk Assessments

The second generation of risk assessments introduced formalized actuarial risk-assessment methodologies.REF The introduction of statistical methods was considered by many to be a significant improvement over first-generation RNAS tools.REF These instruments were based on actuarial analyses of the previous experiences of different types of offenders.REF The analysis identified those variables that were statistically related to the outcome that researchers were most interested in measuring, such as reoffending.REF

The information was mainly derived from the criminal history and case file of each offender. The selected information drew from research on the factors that were linked to the outcome, usually recidivism, which could be measured according to the risk of the offender being rearrested, reconvicted, reincarcerated or the like.REF

The variables that had proved to be trustworthy indicators were incorporated into an instrument. Second-generation instruments, at least compared to first-generation tools, were structured, consistent, transparent, and reviewable. The process of tallying up past behavior resulted in the creation of the second generation of risk-assessment tools, which provided better guidance to the parole board on an individual’s likelihood of having further involvement with the criminal justice system through the systematic identification of factors related to “failure.”REF This process of using statistics to identify the key set of factors related to success and failure was considered an advancement in the field,REF and many researchers concluded that second-generation instruments were more accurate than the first-generation tools across a variety of fields.REF

Third-Generation Risk Assessments

The third generation supplemented static-risk characteristics, such as criminal history with dynamic factors including attitudes, employment, and relationships, considered to be indicators of criminogenic needs and destabilizing or protective factors that can be addressed by rehabilitative interventions.REF Dynamic factors refer to the psychosocial functionality of an individual that is directly or indirectly related to further involvement in the criminal justice system.REF

These dynamic risk factors were meant to improve the predictive validity of tools by identifying contemporary factors that influence behavior and are capable of change, and which can be used to inform decisions about supervision level, case planning, and treatment placements.REF Research has shown that assessing dynamic risk factors can be useful in identifying the treatment programs that would most benefit an individual.REF

Fourth-Generation Risk Assessments

The fourth-generation instruments go beyond assessing dynamic risk and need factors by linking the assessment to a case management plan.REF This plan helps to ensure that risk factors identified in the assessment are addressed in the supervision. It also provides a more systematic approach, and incorporates a broader range of factors important to correctional treatment, such as offender strengths and responsivity factors.REF

As the risk paradigm has expanded, the trend is for instruments to expand to include more measures across more factors, increasing instrument complexity with, it is hoped, improvements in predictive ability.REF Fourth-generation instruments can include more than a dozen subcategories and as many as 100 items.REF Besides criminal history, two experts in the fieldREF have identified eight main dynamic risk factors in the third- and fourth-generation tools:

1. Anti-social attitudes,
2. Peers,
3. Personality,
4. Substance abuse,
5. Marital/family issues,
6. Employment,
7. Education, and
8. Use of leisure time.

While there is consensus among scholars that some or all of the factors are important, there is a thriving debate among researchers on the weight and significance of each factor and whether there are other factors that should be used.

As discussed below, PATTERN has all the hallmarks of a sophisticated fourth-generation risk and needs assessment tool. It draws not only from the legacy DOJ tool, but has been enhanced by input from some of the nations’ leading experts in the field—and it continues to evolve.

The First Step Act

The First Step ActREF places responsibility for the development of a risk and needs assessment system on the Attorney General and the BOP. Both are responsible for working together to create guidelines for, and gathering data on, the system.

The Attorney General and an Independent Review Committee (IRC) are responsible for “develop[ing] and releas[ing] publicly…a risk and needs assessment system.”REF This system should consider various factors in ensuring that the recidivism rate of prisoners released back into society is effectively reduced by considering the risk of recidivism and misconduct of particular prisoners.REF The system’s effectiveness is based on the idea that prisoners can earn time credits for their successful participation in programs for which they are eligible.REF

The program incentivizes prisoners to successfully participate in these programs by offering extra phone, e-mail, or visitation privileges; “transfer to institution closer to release residence”; “[i]ncreased commissary spending limits and product offerings”; preference for housing placement; or other incentives, which vary depending on the risk of recidivism that the particular inmate poses.REF Not all prisoners are eligible to receive time credits for participation in recidivism-reduction programs, and the FSA lays out a detailed list of crimes that make prisoners ineligible to receive time credits through the programs.REF

As part of the risk and needs assessment system, the recidivism-reduction programs must be continuously evaluated to ensure that the programs’ goals are met. The FSA provides that the Attorney General is responsible for collecting data on the recidivism-reduction programs using a risk and needs assessment tool.REF Under the FSA, a risk and needs assessment tool is defined as an

objective and statistically validated method through which information is collected and evaluated to determine—

(A) as part of the intake process, the risk that a prisoner will recidivate upon release from prison;

(B) the recidivism reduction programs that will best minimize the risk that the prisoner will recidivate upon release from prison; and

(C) the periodic reassessment of risk that a prisoner will recidivate upon release from prison, based on factors including indicators of progress and of regression, that are dynamic and that can reasonably be expected to change while in prison.REF

The FSA assigns the duty of ensuring implementation of the programs to both the Attorney General and the BOP. The Attorney General must also ensure that the provisions of the FSA go into effect within 180 days of the passage of the statute.REF

The BOP is required to expand the availability of recidivism-reduction programming and other qualifying productive activities so that eligible prisoners have an opportunity to participate in them within two years of the BOP completing the initial risk and needs assessments for all prisoners. During the two-year period when the BOP is expanding recidivism-reduction programs and productive activities, prisoners who are nearing their release date are given priority for placement in such programs.REF

The BOP is required to provide all eligible prisoners with the opportunity to participate in recidivism-reduction programs or productive activities that address their criminogenic needs throughout their term of incarceration. High- and medium-risk prisoners are given priority for placement in recidivism-reduction programs, while the focus for low-risk prisoners is on participation in productive activities.REF

Prisoners who successfully participate in recidivism-reduction programming or productive activities must be reassessed not less than annually, and high- and medium-risk prisoners with less than five years remaining until their projected release date are required to have more frequent reassessments. If the reassessment shows that a prisoner’s risk of recidivism or specific needs have changed, the BOP must reassign that prisoner to recidivism-reduction programs or productive activities consistent with those changes.REF

In addition, the BOP is responsible for implementing the risk and needs assessment systems and developing tools to evaluate their success.REF The BOP can also partner with private or nonprofit organizations, as approved by the Attorney General, to implement programs.REF

By participating in recidivism-reduction programs, inmates may be released from BOP custody earlier than their sentences initially permitted. To accomplish this, the BOP is responsible for creating guidelines for inmates that incorporate time credits earned through program completion,REF risk assessment, amount of time left to serve, and a warden’s assessment of whether an inmate has made a “good faith effort to lower recidivism risk” and “would not be a danger to society.”REF

Inmates who are granted pre-release custody must serve the balance of their sentences in the form of home confinement, placement in a halfway house, or supervised release. As part of home confinement, inmates must remain in their residence except for BOP-approved work, community service, or religious or family activities.REF Generally, a “prisoner who is placed in home confinement shall remain in home confinement until the prisoner has served not less than 85 percent of the prisoner’s imposed term of imprisonment.”REF

Inmates are subject to “24-hour electronic monitoring”; however, an alternative form of monitoring may be used, if necessary.REF The director of the BOP may use his discretion to change the conditions of home confinement based on the prisoner’s performance.REF Inmates may be placed on supervised release “[i]f the sentencing court included as a part of the prisoner’s sentence a requirement that the prisoner be placed on a term of supervised release after imprisonment.”REF The director of the BOP has discretion to “transfer the prisoner to begin any such term of supervised release at an earlier date, not to exceed 12 months.”REF Inmates may be given more or less freedom based on their compliance with the conditions of their pre-release custody.REF

Much of the focus and criticism of the FSA to date has been on what PATTERN should and should not include, the need for more transparency and definitions, greater involvement by victims, and the like, as discussed herein.

But there is another aspect to this process that is obvious, but rarely mentioned: The success or failure for any federal prisoner, once PATTERN is fully operational and funded, falls on the shoulders of the prisoner. It is the convicted prisoner who must decide whether to fully participate in custom-designed programs in prison created to help him re-enter society with the skills to succeed, and then stay out of trouble once free from custody. While some will succeed, it is inevitable that some will fail and choose to commit additional criminal acts.

The First Step Act Risk and Needs Assessment System: PATTERN

The FSA is an onerous statute, as it places on the DOJ and its components a herculean task: Evaluate the risk and needs of every federal prisoner, design a program for each prisoner to enhance his chances of success, and re-evaluate the prisoner each year to see whether he has demonstrated improvement and is now eligible for early release based on his risk factors.

This is a huge undertaking, especially given the sheer number of men and women in federal prison. As of February 6, 2020, there were a total of 174,728 federal inmates, including 146,453 inmates in BOP custody; 17,150 inmates in privately managed facilities; and another 11,125 inmates in other types of correctional facilities.REF

Developing, testing, and refining a validated and accurate risk and needs assessment suite of tools will be critical to the ultimate success, or failure, of the First Step Act. The FSA set aggressive and specific timelines for action by the Attorney General. In July 2019, the DOJ rose to the challenge and announced its new risk and needs assessment system called the Prisoner Assessment Tool Targeting Estimated Risk and Need, or PATTERN.REF

On that date, the Attorney General announced that the BOP would release approximately 3,100 prisoners from custody because of the good conduct credits accrued under the FSA. He noted that there were an additional 1,691 prisoners who would also be released because their sentences had been reduced due to the retroactive application of the Fair Sentencing Act.REF

Under the First Step Act, the Attorney General was required to develop, validate, and release for public review a RNAS by July 2019. The act also provided that, by January 2020, the Attorney General must implement the RNAS through the BOP; identify effective evidence-based, recidivism-reduction programs and productive activities for BOP inmates; and ensure that all prisoners in BOP custody have access to such programs and activities.REF

Section 107 of the act provided for the establishment of an IRC, the commissioners to be selected by the National Institute of Justice (NIJ). The NIJ must appoint no fewer than six members, each of whom must be experts in risk and needs assessment systems, among other qualifications. The IRC members were selected and have participated in this process in a meaningful way since the IRC’s creation.REF

After PATTERN was released, the DOJ held a 45-day study period in which the public was able to review the system and consider ways in which it could be improved. After the study period, the DOJ proactively engaged stakeholders to receive input from interested parties.

The DOJ is to be commended for these efforts, which included two listening sessions in September 2019 hosted by the NIJ. Eight stakeholder organizations provided public comments, and other comments were submitted in writing to the NIJ. Additionally, on October 7, 2019, the IRC met with senior staff at the DOJ to discuss its members’ views and recommendations regarding PATTERN, including potential improvements for a revised version of the tool. Many of the IRC’s recommendations were accepted by the DOJ. On November 19, 2019, Attorney General William Barr met with the IRC to discuss proposed changes to PATTERN, including several refinements suggested by the IRC.

The DOJ also created a specific e-mail address ([email protected]) for interested members of the public to provide comments and feedback on the risk tool. So far, the DOJ has received approximately 175 comments by stakeholders and other interested parties.REF Since the rollout of PATTERN, the DOJ has continued to work with the IRC and two additional experts, Drs. Grant DuweREF and Zachary Hamilton,REF to improve PATTERN.

The FSA is quite specific with respect to how the risk and needs assessment system is supposed to be used. The act provides that the system must be used to:

(a) All prisoners at each risk level have a meaningful opportunity to reduce their classification during the period of incarceration;

(b) The specific criminogenic needs of the prisoner are being addressed; and

(c) All prisoners are able to successfully participate in such programs.

1. Determine the recidivism risk of each prisoner as part of the intake process, and classify each as a minimum, low, medium, or high risk for recidivism;

2. Assess and determine, to the extent practicable, the risk of violent or serious misconduct of each prisoner;

3. Determine the type and amount of evidence-based recidivism-reduction programming appropriate for each prisoner and assign each to such programming based on his specific criminogenic needs;

4. Re-assess the recidivism risk of each prisoner periodically, based on factors, including indicators of progress and of regression, that are dynamic and that can reasonably be expected to change while in prison;

5. Reassign the prisoner to appropriate evidence-based recidivism-reduction programs or productive activities based on the revised determination to ensure that:

6. Determine when to provide incentives and rewards for successful participation in evidence-based recidivism-reduction programs or productive activities;

7. Determine when a prisoner is ready to transfer into pre-release custody or supervised release; and

8. Determine the appropriate use of audio technology for program course materials for inmates with dyslexia.

This broad mandate presents a significant challenge to the DOJ, especially given the number of federal inmates.

It is easy to reduce a prison population if public safety is not taken into consideration. Simply open the cell doors and inmates will gladly walk out. It is much more difficult, however, to reduce the prison population while also “creating mechanisms to maintain public safety.” How do you identify the right inmates to let go, and what criteria should you use to do so? What tools do you use to assist in the process, and how can you be sure these tools have accurate predictive qualities, so that when they are applied to a particular convict, they can, with some degree of accuracy, assist in evaluating whether that person is a good candidate for early release?

If the tools are not accurate and are too lenient, the consequences can be deadly, as we demonstrate later in this paper. If the tools are too stringent, then few convicts are released under the revised rules, resulting in good candidates for early release languishing in prison instead of getting on with their lives as law-abiding citizens outside of confinement.

PATTERN: What It Measures and Its Predictive Value

The DOJ, through the issuance of PATTERN, has developed a useful risk-assessment tool, which, over time, will evolve into a cutting-edge risk assessment tool. As previously stated, the DOJ has not yet developed a needs-assessment tool, but has vowed to do so in the coming months.REF

PATTERN is based in large part on the existing and well-performing BOP BRAVO-R assessment system. BRAVO-R was a risk-assessment instrument developed by the BOP’s Office of Research and Evaluation and designed to predict recidivism, which is defined as a new arrest or return to federal prison within three years after release. BRAVO-R is a modern and modified version of a legacy classification assessment system used by the BOP in the 1970s.

The DOJ identified four common elements of effective risk and needs assessment tools when it published PATTERN in July 2019: dynamic, individualized assessments; periodic re-validation and updating; racial and ethnic neutrality; and assessment of criminogenic needs.REF Not surprisingly, those elements are consistent with the most commonly used risk and needs assessment tools, and are typical of fourth-generation risk-assessment tools in the United States and in developed countries.

Given the time constraints placed on it by the statute, the DOJ took a novel approach to develop PATTERN. In early May 2019, the NIJ hired two consultants to develop PATTERN and gave them access to non-identifying BOP data, including three-year re-arrest data. It contained data for 278,940 BOP inmates who were released between 2009 and 2015.REF The consultants then included static risk factors as well as dynamic items that are associated with either an increase or reduction in the risk of recidivism.

Using that data, building on BRAVO-R, and applying its expertise in the field, the DOJ developed PATTERN, which measures 17 different items, eight of which are measurements of conduct while incarcerated. The 17 items are:

1. Age of first arrest/convictions;
2. Age at time of assessment;
3. The number of infraction convictions (any);
4. The number of infraction convictions (serious and violent);
5. The number of programs completed (any);
6. The number of technical or vocational courses;
7. Employment in federal custody (UNICOR);
8. Drug treatment while incarcerated;
9. Drug education while incarcerated;
10. Non-compliance with financial responsibility;
11. Current offense (violent);
12. Sex offender;
13. BRAVO-R Criminal History Score;
14. BRAVO-R History of Violence;
15. BRAVO-R History of Escapes;
16. BRAVO-R Voluntary Surrender; and
17. BRAVO-R Education Score.

The DOJ then measured the predictive validity of PATTERN compared to BRAVO-R and the most prominent risk and needs assessment tools in use, and found that PATTERN was 15 percent more predictive regarding someone’s risk of recidivating than other tools on the market.REF Those commercial tools include the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), the Level of Service Inventory-Revised (LSI-R), Static Risk and Offender Needs Guide for Recidivism (STRONG-R), and the Wisconsin Risk-Need (WRN), among others.

COMPAS is a widely used risk-assessment tool.

The PATTERN assessment includes different scales for women, and the instrument contains nine domains: criminal history, disciplinary history, classifications history, family/social support, substance use, education, work and financial history, self-efficacy, and anger. Many of the scales have subscales.REF The predictive value of COMPAS was 67 percent, compared to the PATTERN instrument, which has an 80 percent predictive value for men and 79 percent for women.

LSI-R is one of the most often studied assessments in terms of predictive value.REF It consists of 10 categories: criminal history, education/employment, financial, family/marital, accommodations, leisure/recreation, companions, alcohol/drug problems, emotional/personal, and attitudes/orientation. The SLI-R contains different scoring ranges for females and males, and the category of risk (e.g., high, medium, low) for each gender by using national norms. The predictive value of LSI-R was 64 percent.

STRONG-R was developed with offense-specific and gender-related scoring for each of the individual need-related items: education, community, employment, friends, residential, family, substance use/abuse, mental health, aggression, and attitudes and behaviors.REF The predictive value of STRONG-R was 74 percent.

The WRN tool was developed in the 1970s, modified in the 1980s, and is one of the most widely adopted risk and needs assessments in the United States.REF It consists of 12 domains: academic/vocational, employment, financial management, marital/family, companions, emotional stability, alcohol usage problems, other drug usage problems, mental ability, health, sexual behavior, and parole officer’s impression. The WRN has no separate scoring for females. The predictive value of the WRN tool was 67 percent.

Chart 1, reproduced from the DOJ’s July 2019 report on PATTERN, shows the predictive value of PATTERN for men and women, compared to 10 of the most widely respected and used risk and needs assessment tools in the country.

In reality, there is no way of knowing how precise PATTERN will be in accurately predicting which inmates are appropriate candidates for early release, and which will recidivate. In time, as PATTERN is refined, programs are made available to all eligible federal inmates, the risk and needs of each inmate is re-evaluated, and inmates are released early, there will be studies on the accuracy of PATTERN itself, and it will, no doubt, be compared to the results from other widely used instruments.

But at this point it is clear that the 17 factors included in PATTERN are typical for the most widely respected and used risk-assessment tools across the country and relied upon by experts in the field.

Stakeholder Feedback on PATTERN

As mentioned above, the DOJ sought and received substantial feedback during the development of PATTERN and after it was released.REF As the Attorney General said at the time of the release, the “experience of these communities will aid the Department as it works to improve the System.”REF The DOJ made clear that PATTERN is “only the first step in an ongoing, collaborative, and dynamic process to enhance prison programming, improve inmate outcomes, and ultimately reduce recidivism and make every community safer.”REF

After receiving comments from stakeholders and the public, the DOJ worked with the IRC and Drs. Duwe and Hamilton to identify ways to improve PATTERN while maintaining its high level of predictability.REF The DOJ, to its credit, updated PATTERN as a result of the feedback it received. DOJ officials also pushed back on some assertions made by stakeholders which, in their opinion, lacked merit.

Adding Dynamic Factors to PATTERN

The IRC and othersREF suggested that the DOJ identify other dynamic factors that would be appropriate measures to add to PATTERN. In its January 2020 update, the DOJ emphasized that any risk and needs assessment program must accurately measure an inmate’s change in risk level during incarceration and provide opportunities for inmates to reduce their post-intake risk scores as measured through periodic reassessments.REF

The DOJ studied the issues and announced that it will augment PATTERN by:

• Including an additional dynamic measure of an offender’s “infraction free” period during his or her current term of incarceration; and
• Modifying programming measures by adding psychology treatment programs such as sex-offender treatment, faith-based Life Connections Program, and other technical and vocational programs.REF

Such measures will likely strengthen PATTERN for the future.

Assertion That PATTERN Discriminates Against Minorities

At least two stakeholders asserted that PATTERN discriminates against minorities. If true, this would be devastating for obvious reasons.

One stakeholder, the Sentencing Project, asserted that African American males were significantly more likely to be scored as high risk (53 percent compared to 29 percent for white men and 33 percent for Hispanic men) under PATTERN. They claimed that this higher rate is “likely due to their elevated interactions with the criminal justice system which research partially associates with structural racism, policing practices in communities of color and socio-economic status.”REF

Another stakeholder, the Leadership Conference on Civil and Human Rights, claimed that the DOJ’s claim that PATTERN is unbiased across racial and ethnic classifications is “dependent on a constrained definition of racial bias as a statistical matter.”REF They called for the DOJ to “suspend the use of PATTERN” until it “adequately addresses” their concerns. They urged the DOJ to “consult with computer scientists and data scientists working on fairness, accountability, and transparency and to adopt their recommendations for equitable outcomes.”REF

In response, the DOJ agreed that “inmates and the public are not served by a risk assessment tool that is racially biased or does not accurately reflect inmate risk.”REF To test whether PATTERN, as constructed, was a race-neutral tool and to ensure it was predictive across all races and genders, the DOJ subjected PATTERN to an Area Under the Curve (AUC) analysis.REF The DOJ concluded that “PATTERN is a neutral assessment tool, as evidenced by the nearly equal scores on the Area Under the Curve Analysis,” reproduced in the chart below.

Despite the fact that the AUC analysis demonstrated that PATTERN is a race-neutral tool, the DOJ, sensitive to the claims of stakeholders, removed two factors from PATTERN, as they “might be associated with bias, especially racial bias.”REF The two factors removed were: (1) age of first arrest/conviction, and (2) voluntary surrender. Those changes, according to the DOJ, reduced PATTERN’s predictive accuracy by “approximately one percent.”REF

A related but separate criticism based on race was that considering “supervised release violation” as a factor in the risk score would disproportionately affect blacks and Hispanics, as “they are more likely to have their supervised release revoked due to biases in the criminal justice system.”REF After researching that issue, the DOJ found the opposite to be true; if supervised release violations were removed from the data, it would “actually increase the potential racial disparity and have a negative impact on the predictability of the tool.”REF

The DOJ updated PATTERN into a “more streamlined version of the risk tool previously released…[and it] contains fifteen factors, eleven dynamic and four static.”REF

Dynamic factors now include:

1. Infraction convictions (any) current incarceration;

2. Infraction convictions (serious and violent) current incarceration;

3. Infraction-free (any) current incarceration;

4. Infraction-free (serious and violent) current incarceration;

5. Number of programs completed (any);

6. Work programming;

7. Drug treatment while incarcerated;

8. Non-compliance with financial responsibility;

9. History of violence;

10 History of escapes; and

11. Education score.

Static factors now include:

12. Age at time of assessment;

13. Instant violent offense;

14. Sex offender (Adam Walsh Act);REF and

15. Criminal history score.

After updating PATTERN with those dynamic and static categories, the DOJ again tested it to see if it retained the high level of predictability across races and sexes—and found that it did.REF

Are Separate Risk Assessments for Men and Women Unconstitutional?

The Leadership Conference on Civil and Human Rights claimed that there are “serious constitutional questions around the creation of separate risk-assessment algorithms for men and women,” citing the Supreme Court case of Craig v. Boren.REF They urged the DOJ to “abandon gendered risk assessment models altogether.”REF

Arguments that applying PATTERN differently for male and female inmates would be unconstitutional based on the Supreme Court’s decision in Craig v. BorenREF are completely without merit.

In Craig, the Court considered the constitutionality of an Oklahoma statute that prohibited the sale of “nonintoxicating” 3.2 percent beer to women under the age of 18 and to men under the age of 21.REF Applying an intermediate level of scrutiny,REF the Court held that this facial discrimination based solely on sex violated the Equal Protection Clause of the Fourteenth Amendment because there was insufficient statistical evidence about the drunk driving habits of the two sexes to justify disparate treatment.REF

For one thing, there was only a 2 percent correlation between male gender and an arrest for drunken driving.REF That correlation was “an unduly tenuous fit.”REF For another, the statistics offered in Craig suffered from “obvious methodological problems.”REF And last, the statistics did not “justify the salient features” of the law—viz., “the use and dangerousness of 3.2 percent beer as opposed to alcohol generally.”REF In fact, although the law prevented the sale of 3.2 percent beer to men under 21, it did not prevent them from drinking it once acquired.REF In sum, the statistics were of poor quality, loose fit, and relied on generalities.

That is not the case here.

PATTERN uses separate modeling for men and women because evidence-based research shows that men and women have different pathways to crime and different risk factors.REF For example, there are particular issues, such as unhealthy intimate relationships, economic marginality, unsafe housing, past and current victimization and trauma, addiction, low self-efficacy, parental stress, anger/hostility, and depression/anxiety, that have been shown to be particularly salient among women.REF

Existing risk and needs assessment tools, particularly the LSI-R, have been criticized for primarily assessing men and not adequately accounting for risk and needs that are more appropriate for women.REF There is a general concern among experts that these tools do not integrate specific measures that would be useful in assessing women’s pathways to criminal behavior, the gender-responsive pathways that affect dynamic needs, or the unique factors that are important for women’s specific responsivity needs.REF The DOJ noted that the “connection to crime varies for men and women,” and that it was “imperative that PATTERN model the risk of recidivism…separately.” And as a practical matter, in PATTERN’s dataset, men outnumber women by six to one.

Moreover, applicable Supreme Court precedent clearly establishes that sex-based differential treatment is permissible provided the differentiation is based on real differences between men and women.REF What the Court is concerned with, and has prohibited in the past, is differential treatment that uses gender as a proxy for other relevant characteristics—in other words, gender stereotypes.REF PATTERN is not based on gender stereotypes but on real differences between men and women—and their involvement in the criminal justice system.

Using different risk-assessment tools on men and women is clearly based on valid differences between men and women, is called for by leading academic researchers, and is the best way to identify the risks and assess the needs of two very different justice-involved genders. As such, it is clearly constitutional, and indeed prudent, and the DOJ is wise to use distinct tools for each gender.REF

Should the DOJ Narrow the Definition of Recidivism?

A diverse set of groups urged the DOJ to narrow the definition of recidivism by not taking into account arrests made within three years of release from the BOP, contending that only convictions should count. The American Conservative Union (ACU) Nolan Center for Justice found the use of arrest records “problematic” because those arrests “presumably includes instances where charges against a defendant have been dropped, where the defendant has been found guilty, or he/she was ultimately exonerated due to post-conviction relief.”REF The ACU noted that the number of people each year who have been arrested, but never convicted of a charge, is “not insubstantial.”REF The ACU urged the DOJ to take the time to get the data regarding convictions, and use that, instead of arrests, as proof that someone is a recidivist.

Similarly, The Leadership Conference on Civil and Human Rights criticizes PATTERN as overly reliant “on mere arrest data, and does not consider whether or not the arrest led to a negative disposition for the person arrested.”REF To them, PATTERN is deficient because it “assumes that an arrest itself is proof of a crime without any Due Process.”REF

In response to these criticisms, the DOJ noted several facts and issues that led them to keep the current definition of recidivism. First, there is no uniform definition of recidivism in the criminal justice system across the United States. The Bureau of Justice Statistics, which keeps criminal statistics, uses the same definition—a return to BOP custody or a re-arrest within three years of release from BOP custody, including driving under the influence or driving-while-intoxicated offenses.

The challenge is that the BOP does not have access to accurate and complete data for case dispositions, including convictions or acquittals.REF The states push data to the BOP of their own choosing, and the BOP lacks the authority to compel states to enhance their reporting and to dedicate the resources needed to include the ultimate disposition of a case.REF

Bureau of Prisons Programs Need Efficacy Testing

A major underlying assumption of the FSA is that the recidivism-reduction programs offered by the BOP will be effective at reducing recidivism. Determining what works in helping former federal inmates lead law-abiding and successful lives, thereby enhancing public safety, is critical to the long-term success of the FSA. The DOJ must know whether or not participation in recidivism-reduction programming actually reduces recidivism. To accomplish this task, the DOJ needs to rigorously evaluate these programs.

Unfortunately, the BOP has done an inadequate job of determining the effectiveness of its current recidivism-reduction programs.REF A review of the scientific literature by the IRC concluded, “Serious, formal evaluations of current BOP programming are too scarce to tell us much about the effectiveness of that programming.”REF The BOP’s evaluations of its existing programs have not been scientifically rigorous and are also seriously outdated. For example, the last time the BOP released an evaluation of the Federal Prison Industries work program was in 1996—over 20 years ago.REF

As the research, development, and evaluation agency within the DOJ, the NIJ is uniquely positioned to oversee rigorous and independent evaluations of programs offered to inmates in BOP custody. The NIJ should, therefore, be entrusted with carrying out the requirements set forth in Section 3633 of the FSA, which provides that the Attorney General, in consultation with the IRC, shall:

1. Review the effectiveness of evidence-based recidivism-reduction programs that exist…in prisons operated by the Bureau of Prisons;
2. Review available information regarding the effectiveness of evidence-based recidivism-reduction programs and productive activities that exist in state-operated prisons throughout the United States; and
3. Identify the most effective evidence-based recidivism-reduction programs.

Program evaluation is part of the NIJ’s legislated duties, and its independence from the BOP will lend additional credibility to evaluation results. Because the NIJ engages in independent and rigorous research, the NIJ, rather than the internal BOP research office, is the appropriate agency to carry out this work. To accomplish this important mission, the NIJ should conduct ongoing research and data analysis on:

• The most effective and efficient uses of recidivism-reduction programs; and
• Which evidence-based recidivism-reduction programs are the most effective at reducing recidivism, and the type, amount, and intensity of programming that most effectively reduces the risk of recidivism.

Ensuring Appropriate Funding

In order to begin this important work, Congress should allocate money in fiscal year 2020 to the NIJ to fund independent evaluations of programs offered by the BOP aimed at reducing recidivism. This money should be viewed as a down payment on a long-term commitment by Congress to fund a robust evaluation agenda of BOP programs. A sustained commitment of $5 million per year over five fiscal years—for a total of $25 million—would yield extremely useful information on which BOP programs successfully reduce recidivism. The information provided by these NIJ-funded evaluations will help the BOP effectively allocate resources to lower recidivism rates and protect public safety.

To appropriately meet the mandates in Section 3633 of the FSA and to provide proper advice to the Attorney General about the most effective programs, a thorough and expert knowledge of evidence-based reduction programming is in order. Moreover, it is critical that this knowledge be combined with expertise in implementation science and program evaluation. The NIJ is the logical choice to lead this effort since its science staff has the requisite expertise and relevant experience to facilitate evaluations of evidence-based recidivism-reduction programs. REF

Conclusion

The First Step Act is a significant achievement. It was a rare moment in time when a bipartisan congressional delegation and an Administration supported meaningful and comprehensive criminal justice reform. Stakeholders from across the ideological spectrum came together to get behind much-needed legislation.

A key pillar to that reform ultimately succeeding is the creation and implementation of a 21st-century risk and needs assessment system. To date, the DOJ has risen to part of the challenge by publishing PATTERN, its risk-assessment tool. In short order, it refined PATTERN after taking into consideration a wide variety of viewpoints. No doubt, PATTERN will continue to be refined, as any modern risk-assessment program is only as good as the latest science and research.

With respect to developing a new and improved needs-assessment program under PATTERN, the DOJ has so far fallen short, but has acknowledged an ambitious time frame in which to publish that program.

As PATTERN matures, and more data becomes available, we will be able to ascertain how accurate PATTERN is in predicting recidivism and whether, in its application, it proves to be both race and gender neutral and an effective tool. The DOJ should continue to be prudent in studying the data as it accrues and considering a wide variety of feedback on PATTERN, and should base future decisions based on fact and the best science available, not political considerations or outcome-based desires.

Charles D. Stimson is Senior Legal Fellow and Manager in the National Security Law Program in the Edwin Meese III Center for Legal and Judicial Studies, of the Institute for Constitutional Government, at The Heritage Foundation.