Office of Reporting, Research, Analytics and Implementation (ORRAI)

Implementation works through collaboration and consensus-building to inform, plan, implement, and evaluate research for all program areas. It is a sequential process which includes a collaborative workgroup model, training development, pilots, statewide training, follow up and technical assistance. The timeline from research to completed implementation can take approximately 12 to 24 months depending on the complexity of the research and size of the training audience.


Goals

The goal of the ORRAI Research Implementation team is to ensure that DHS data, research, and analytics:

  • Strengthen decision-making and outcome and metric tracking;
  • Inform policies, procedures and operational models;
  • Are conducted collaboratively through partnerships with DHS leadership, staff from all levels, experts, and other key stakeholders;
  • Are well-communicated throughout the DHS community;
  • Are well-implemented through training and technical assistance to staff;
  • Are supported and sustainable through evaluation and designated resources when applicable.

Research Implementation Steps

STEP 1 - Research:

During the research phase we work with stakeholders to define the scope and operational impact, prepare and document the project, and identify the impact.

STEP 2 - Workgroup:

We utilize representative workgroups throughout the research process to: evaluate what we are learning; determine the optimal uses; develop strategies for implementation; and provide feedback on training development and delivery.

STEP 3 - Development:

In developing tools, processes, or models that will be used by employees in their work, we focus on useful and user-friendly products by developing curriculum for stakeholders at multiple levels, utilizing the content knowledge of experts within the system, and by considering diversity, equity, and inclusion principles.

STEP 4 - Training:

Our training phase includes: utilizing workgroup recommendations for training; addressing site specific strengths, barriers, challenges, and goals; and gathering and documenting feedback from trainings to inform future projects and adapt future trainings.

STEP 5 - Support and Maintenance:

We monitor the outcomes and implementation of research through:

  • Continuous on-site technical assistance, case specific consultation and stakeholder support.
  • Outcome monitoring, data collection and evaluation related to implementation.
  • Communication with stakeholders regarding the impact of implementation.
  • Maintenance of implementation plan with status and progress updates to stakeholders.

Project Highlights

Summary

Child Welfare: Capacity Project

Identifying Capacity Needs for Children within the Oregon Child Welfare System

Placements for children and youth in the Oregon foster care system have been dictated by placement availability with limited recognition of child needs and provider capability. This research will estimate the number of placement beds (e.g. foster care, proctor care, residential treatment, etc.) necessary to optimally serve children in substitute care. The Implementation team worked with a panel of casework experts to recommend placement types for a random sample of children and youth entering substitute care. Researchers will then use statistical techniques to identify the best placement for each of these same youth, based on optimal child outcomes. The differences in these two results will refine estimates of placement capacity to create the optimal continuum of care.

Child Welfare Screening Predictive Analytics Tool

ORRAI’s Research and Implementation units partnered with Child Welfare to launch an innovative new tool that utilizes predictive analytics to support child abuse allegation screening decisions. The Screening Predictive Analytics Tool uses Oregon historical child welfare outcomes and predictive models to generate probability scores that assist in decision-making about whether to assign a report to the child abuse hotline for further investigation. The predictive models combine the current report information with historical data from OR-Kids, the statewide DHS Child Welfare data system. The models use machine learning to leverage more than a hundred distinct variables (i.e., individual pieces of information) into thousands of combinations to generate the estimates.

Implementation convened Child Welfare leadership and staff and members of the Office of Equity and Multicultural Services (OEMS), the Office of Information Services (OIS), and the Office of Training, Investigations, and Safety (OTIS) in a series of workgroups that informed the development of the predictive model, implementation of the tool into OR-Kids and the screening process, troubleshooting and testing, communication to stakeholders, and staff training. The process was also supported by partnerships with Action for Child Protection and Casey Family Programs. Read more about the model and pilot implementation of the tool at the Oregon Child Abuse Hotline (ORCAH) in the Safety at Screening Research Brief.pdf and the Safety at Screening Tool Development and Execution Report.pdf.

Because predictive equations are created with historical data, they have the potential to perpetuate biases embedded in the data. An additional, important component of the Safety at Screening Tool is the fairness correction procedure that minimizes bias while preserving accuracy. Read more in the Fairness Correction Manuscript.pdf

(More Items...)

 

OregonDHS_Safety-at-Screening-Research-Brief-v1.071519.pdfOregonDHS_Safety-at-Screening-Research-Brief-v1.071519.pdf
  
Summary
  
  
  

Care for Older Oregonians

ORRAI’s reporting and research teams partnered with Institute on Aging at Portland State University to create two publications in 2018: Resident and Community Characteristics Report on Assisted Living, Residential Care, Memory Care, 2018 and Resident and Community Characteristics Report on Adult Foster Homes, 2018.

1/1/2018Reporting/Data Warehouse
  

Nursing Facility Care in Oregon

Reporting also collaborated with Oregon State University College of Public Health and Human Sciences to produce The State of Nursing Facilities in Oregon in, 2016 to assist in local and statewide planning and policy-making efforts in long-term care services.

1/1/2016Reporting/Data Warehouse
  

Child Welfare: Capacity Project

Identifying Capacity Needs for Children within the Oregon Child Welfare System

Placements for children and youth in the Oregon foster care system have been dictated by placement availability with limited recognition of child needs and provider capability. This research will estimate the number of placement beds (e.g. foster care, proctor care, residential treatment, etc.) necessary to optimally serve children in substitute care. The Implementation team worked with a panel of casework experts to recommend placement types for a random sample of children and youth entering substitute care. Researchers will then use statistical techniques to identify the best placement for each of these same youth, based on optimal child outcomes. The differences in these two results will refine estimates of placement capacity to create the optimal continuum of care.

1/1/2019Research Implementation
  

Child Welfare Screening Predictive Analytics Tool

ORRAI’s Research and Implementation units partnered with Child Welfare to launch an innovative new tool that utilizes predictive analytics to support child abuse allegation screening decisions. The Screening Predictive Analytics Tool uses Oregon historical child welfare outcomes and predictive models to generate probability scores that assist in decision-making about whether to assign a report to the child abuse hotline for further investigation. The predictive models combine the current report information with historical data from OR-Kids, the statewide DHS Child Welfare data system. The models use machine learning to leverage more than a hundred distinct variables (i.e., individual pieces of information) into thousands of combinations to generate the estimates.

Implementation convened Child Welfare leadership and staff and members of the Office of Equity and Multicultural Services (OEMS), the Office of Information Services (OIS), and the Office of Training, Investigations, and Safety (OTIS) in a series of workgroups that informed the development of the predictive model, implementation of the tool into OR-Kids and the screening process, troubleshooting and testing, communication to stakeholders, and staff training. The process was also supported by partnerships with Action for Child Protection and Casey Family Programs. Read more about the model and pilot implementation of the tool at the Oregon Child Abuse Hotline (ORCAH) in the Safety at Screening Research Brief.pdf and the Safety at Screening Tool Development and Execution Report.pdf.

Because predictive equations are created with historical data, they have the potential to perpetuate biases embedded in the data. An additional, important component of the Safety at Screening Tool is the fairness correction procedure that minimizes bias while preserving accuracy. Read more in the Fairness Correction Manuscript.pdf

1/1/2018Research Implementation
  

Social Complexity in Medicaid Youth

Literature has shown that social and environmental factors present in a child’s life greatly influence health outcomes as an adult, and that the effects of these experiences are cumulative. Some of these social determinants of health are identifiable through administrative data on service utilization within DHS. This project, in partnership with Oregon Health Authority and the Oregon Pediatric Improvement Partnership, seeks to identify available administrative data sources for social risk factors and provide an index of social risk to Oregon’s coordinated care organizations so they can provide additional services to children most at-risk.

Learn more about research and analytics across DHS programs and state agencies.

1/1/2019Research/Analytics
  

Child Welfare Screening Predictive Analytics Tool

ORRAI’s Research and Implementation units partnered with Child Welfare to launch an innovative new tool that utilizes predictive analytics to support child abuse allegation screening decisions. The Screening Predictive Analytics Tool uses Oregon historical child welfare outcomes and predictive models to generate probability scores that assist in decision-making about whether to assign a report to the child abuse hotline for further investigation. The predictive models combine the current report information with historical data from OR-Kids, the statewide DHS Child Welfare data system. The models use machine learning to leverage more than a hundred distinct variables (i.e., individual pieces of information) into thousands of combinations to generate the estimates.
 
Implementation convened Child Welfare leadership and staff and members of the Office of Equity and Multicultural Services (OEMS), the Office of Information Services (OIS), and the Office of Training, Investigations, and Safety (OTIS) in a series of workgroups that informed the development of the predictive model, implementation of the tool into OR-Kids and the screening process, troubleshooting and testing, communication to stakeholders, and staff training. The process was also supported by partnerships with Action for Child Protection and Casey Family Programs. Read more about the model and pilot implementation of the tool at the Oregon Child Abuse Hotline (ORCAH) in the Safety at Screening Research Brief.pdf and the Safety at Screening Tool Development and Execution Report.pdf.
Because predictive equations are created with historical data, they have the potential to perpetuate biases embedded in the data.
 
An additional, important component of the Safety at Screening Tool is the fairness correction procedure that minimizes bias while preserving accuracy. Read more in the Fairness Correction Manuscript.pdf
1/1/2018Research/Analytics