Implementation works through collaboration and consensus-building to inform, plan, implement, and evaluate research for all program areas. It is a sequential process which includes a collaborative workgroup model, training development, pilots, statewide training, follow up and technical assistance. The timeline from research to completed implementation can take approximately 12 to 24 months depending on the complexity of the research and size of the training audience.
The goal of the ORRAI Research Implementation team is to ensure that DHS data, research, and analytics:
- Strengthen decision-making and outcome and metric tracking;
- Inform policies, procedures and operational models;
- Are conducted collaboratively through partnerships with DHS leadership, staff from all levels, experts, and other key stakeholders;
- Are well-communicated throughout the DHS community;
- Are well-implemented through training and technical assistance to staff;
- Are supported and sustainable through evaluation and designated resources when applicable.
Research Implementation Steps
STEP 1 - Research:
During the research phase we work with stakeholders to define the scope and operational impact, prepare and document the project, and identify the impact.
STEP 2 - Workgroup:
We utilize representative workgroups throughout the research process to: evaluate what we are learning; determine the optimal uses; develop strategies for implementation; and provide feedback on training development and delivery.
STEP 3 - Development:
In developing tools, processes, or models that will be used by employees in their work, we focus on useful and user-friendly products by developing curriculum for stakeholders at multiple levels, utilizing the content knowledge of experts within the system, and by considering diversity, equity, and inclusion principles.
STEP 4 - Training:
Our training phase includes: utilizing workgroup recommendations for training; addressing site specific strengths, barriers, challenges, and goals; and gathering and documenting feedback from trainings to inform future projects and adapt future trainings.
STEP 5 - Support and Maintenance:
We monitor the outcomes and implementation of research through:
- Continuous on-site technical assistance, case specific consultation and stakeholder support.
- Outcome monitoring, data collection and evaluation related to implementation.
- Communication with stakeholders regarding the impact of implementation.
- Maintenance of implementation plan with status and progress updates to stakeholders.
Child Welfare: Capacity Project
Identifying Capacity Needs for Children within the Oregon Child Welfare System
Placements for children and youth in the Oregon foster care system have been dictated by placement availability with limited recognition of child needs and provider capability. This research will estimate the number of placement beds (e.g. foster care, proctor care, residential treatment, etc.) necessary to optimally serve children in substitute care. The Implementation team worked with a panel of casework experts to recommend placement types for a random sample of children and youth entering substitute care. Researchers will then use statistical techniques to identify the best placement for each of these same youth, based on optimal child outcomes. The differences in these two results will refine estimates of placement capacity to create the optimal continuum of care.
Child Welfare Screening Predictive Analytics Tool
ORRAI’s Research and Implementation units partnered with Child Welfare to launch an innovative new tool that utilizes predictive analytics to support child abuse allegation screening decisions. The Screening Predictive Analytics Tool uses Oregon historical child welfare outcomes and predictive models to generate probability scores that assist in decision-making about whether to assign a report to the child abuse hotline for further investigation. The predictive models combine the current report information with historical data from OR-Kids, the statewide DHS Child Welfare data system. The models use machine learning to leverage more than a hundred distinct variables (i.e., individual pieces of information) into thousands of combinations to generate the estimates.
Implementation convened Child Welfare leadership and staff and members of the Office of Equity and Multicultural Services (OEMS), the Office of Information Services (OIS), and the Office of Training, Investigations, and Safety (OTIS) in a series of workgroups that informed the development of the predictive model, implementation of the tool into OR-Kids and the screening process, troubleshooting and testing, communication to stakeholders, and staff training. The process was also supported by partnerships with Action for Child Protection and Casey Family Programs. Read more about the model and pilot implementation of the tool at the Oregon Child Abuse Hotline (ORCAH) in the Safety at Screening Research Brief.pdf and the Safety at Screening Tool Development and Execution Report.pdf.
Because predictive equations are created with historical data, they have the potential to perpetuate biases embedded in the data. An additional, important component of the Safety at Screening Tool is the fairness correction procedure that minimizes bias while preserving accuracy. Read more in the Fairness Correction Manuscript.pdf