Annual Strategic Research Plan

County of San Diego, Office of Evaluation, Performance, and Analytics
July 2023
Download PDF
A colorful mural on the side of a building in a park, with pieces that read "diversity is a fact" and "inclusion is an act!!!"
Three children in the foreground and multiple children in the background shoveling in a garden
Picture frame sitting on a shelf that says home sweet home
Contents

Preface

Is a $500 monthly rent subsidy effective in preventing homelessness among low-income older adults? What about the effectiveness of a $500 monthly cash transfer in preventing involvement in the foster care system among families at high risk of such involvement? Can we use County contracts to nudge businesses into paying their employees’ wage theft judgements if they want to do business with the County?
These are some of the questions the County of San Diego will try to answer through the six research and evaluation projects summarized in our first Annual Strategic Research Plan:
  • Evaluation of the Recovery Action Fund for Tomorrow Program
  • Evaluation of the Shallow Rent Subsidy Program for Older Adults
  • Economic Analysis of Increasing the Neurodivergent Workforce in the Public Sector
  • Using Contractual Incentives to Improve Recovery of Wage Theft Judgements
  • Evaluation of the Family Income for Empowerment Program
  • Evaluation of the Immigrant Rights Legal Defense Program
These six projects are only the starting point. The plan also identifies 33 priority questions that will guide our research and evaluation efforts over the next several years. Policy areas represented among these questions include decarbonization, infrastructure investments, childcare, early care and education, preventative regulatory programs, behavioral health, social isolation, and justice reform, among many others. These priority questions will be revisited annually; over time, our research and evaluation work will cover an increasingly higher proportion of the hundreds of programs and services delivered by the County.
This strategic and structured approach to building evidence seeks to ensure that our analytical resources and research partnerships focus on high priority issues that will help the County become more effective and efficient. The plan will also increase transparency and accountability around our efforts to build and use evidence. Furthermore, the development of this plan is part of a larger effort to support the use and building of evidence across all County departments. Related initiatives include the development of an inventory of all County programs; an assessment of the County’s current capacity to conduct program evaluation and performance measurement; the development of training curricula on program evaluation for County employees; and the establishment of an external advisory committee and a technical expert panel that will provide technical assistance in our research and evaluation projects.
Altogether, these efforts will help us further develop and sustain a culture of learning and continuous improvement, so we can provide better services and create better policies to improve the lives of the residents of the San Diego region.
RICARDO BASURTO-DÁVILA, PHD MS
Chief Evaluation Officer
Office of Evaluation, Performance and Analytics

Introduction

On May 18, 2021 (16), the San Diego County Board of Supervisors approved recommendations on Launching a Comprehensive Approach to Evidence-Based Policy Making and Establishing the Office of Evaluation, Performance, and Analytics (OEPA). This action instructed OEPA to develop the County’s Annual Strategic Research Plan. The plan would identify the County’s most important research and policy questions and determine which key service areas and program models need evaluation. 
The plan was developed in collaboration with OEPA’s Internal Advisory Group (IAG), which includes representatives from the County’s four Business Groups: Finance and General Government, Health and Human Services Agency, Land Use and Environment, and Public Safety. The public gave input as part of community engagement efforts for the County’s budget, and academic and technical experts also offered advice.
To develop the research plan, OEPA and the IAG first enlisted subject matter experts from all four County business groups to create a Learning Agenda for the County, which identifies priority questions for each of the County’s strategic initiatives: sustainability, equity, empower, community, and justice. The Learning Agenda (shown in Appendix A) will provide strategic direction for OEPA and County departments to plan investments in analytical capacity and establish external research partnerships.
The next step identified Learning Activities, the research and evaluation projects to work on within the next year. In general, learning activities are related to existing learning agenda questions, but there could be exceptions, such as evaluations mandated by program funders or directed by the Board. In some cases, more than one learning activity may be necessary to answer one learning agenda question. OEPA is identified as the lead in most research and evaluation projects in this document but learning activities are not exclusive to OEPA. Future research plans will reflect how OEPA is helping departments conduct research and evaluation for their own projects.  
The Annual Strategic Research Plan is a living document. As County priorities evolve, and we identify and develop new research and evaluation projects, the learning agenda and learning activities will be updated.

Learning Activities

Evaluation of Recovery Action Fund for Tomorrow Program (RAFT)
Project Description
RAFT will provide a one-time cash payment of $4,000 to 2,250 eligible applicants identified as low-income families and seniors living in areas disproportionately affected by the COVID-19 pandemic. The program’s main goals are to increase economic security, well-being, and—for a subset of applicants who demonstrate a high risk—prevent homelessness.
The proposed study will evaluate the impact of unconditional cash payments and help the County make future decisions about similar investments.
Evaluative/Research Question
  1. What impact did RAFT have on economic security?
  2. What impact did RAFT have on housing stability?
  3. What impact did RAFT have on individual or family well-being?
  4. Did the program promote equitable well-being?
  5. How did program beneficiaries spend the money?
Evaluation of Shallow Subsidy Program for Older Adults (SRSP)
Project Description
Homelessness is one of the County’s most critical policy challenges. The 2020 Point in Time Count showed that one out of four San Diego adults experiencing homelessness is over the age of 55.
A lottery will select 222 low-income older adults who are rent-burdened. SRSP will pay monthly $500 subsidies to the landlords of these selected applicants for 18 months. Evaluation of SRSP will determine whether the program is effective in reducing housing instability and preventing homelessness.
Evaluative/Research Questions
  1. To what extent did the SRSP affect housing stability?
  2. To what extent did the SRSP affect entry into homelessness?
Economic Analysis of Increasing the Neurodivergent Workforce in the Public Sector
Project Description
Autism affects one in 44 people in the U.S. and one in 26 in California, including tens of thousands in San Diego County. Autistic people have been largely under-represented in the workforce despite many having skills employers need. This study, initiated per County’s Board of Supervisors (June 14, 2022, item 23) will estimate the costs and benefits of increasing the neurodivergent workforce in the public sector and include impacts to  social services, housing and health.
Evaluative/Research Questions
  1. What would be the costs of increasing the neurodivergent workforce in the public sector?
  2. What would be the economic benefits of increasing the neurodivergent workforce in the public sector?
Evaluation of Family Income for Empowerment Program (FIEP)
Project Description
Children living in low-income households run a higher risk of maltreatment and entering foster care. FIEP is a guaranteed income pilot that will give direct cash payments to low-income families that have been involved with the County’s child welfare system. FIEP will distribute $500 monthly to 485 families selected through a lottery for a period of two years. The proposed evaluation will involve multiple studies to assess the program’s impact on foster care involvement, cost savings to the County, and whether it was implemented as intended.
Evaluative/Research Questions
  1. To what extent did FIEP impact new reports, investigations, or substantiated findings of child abuse or neglect?
  2. To what extent did FIEP affect indicators of protective factors, well-being, housing and financial security, etc.?
  3. What was FIEP’s economic impact on the County of San Diego?
Project Description
Individuals in immigration court who cannot afford an attorney do not have a right to government-appointed counsel, even though many immigrants face deportation to life-threatening conditions or family separation. Immigrants navigating the complexities of immigration law may not get the relief they are entitled to under immigration law. IRLDP provides attorneys to detained individuals in deportation proceedings, aiming to promote due process, reduce the workload of immigration judges and save taxpayer dollars by moving people through the system efficiently and reducing detention related costs. The proposed study will estimate the impact of IRLDP on case outcomes, assess cost savings to the County or the federal government, and identify opportunities to improve implementation.
Evaluative/Research Questions
  1. What is the impact of IRLDP on case dispositions, release from detention facilities, case length, and failure to appear in court?
  2. What are the economic costs and benefits of the program, and who realizes these benefits, compared to the alternative of not providing counsel?
  3. Are there any implementation challenges that could be addressed through process improvements?
  4. Did the rate of representation amongst detained immigrants in San Diego increase after implementation of IRLDP?
Using Contractual Incentives to Improve Recovery of Unpaid Wage Theft Judgements
Project Description
Nearly 1,000 workers in San Diego County have not been paid over $13 million in the past six years of lost wages, even after prevailing in a wage claim against their employer. This study will determine the overlap between businesses that contract with the County and those that have unpaid wage theft judgements. Next, we will design an evaluation to estimate the impact of recent changes to County contracts that require vendors to disclose if they are under investigation or have judgements against them of unpaid wage theft.
Evaluative/Research Questions
  1. To what extent do County contractors have wage theft judgements against them that have not been paid?
  2. What is the impact of using contractual incentives to improve recovery of wage theft?

Funding Status and Estimated Timeline

  1. Budget is under development.
  2. Two evaluation components are funded; we are currently exploring funding options for the rest.
  3. The RAND Corporation is interested in conducting this evaluation; they are currently exploring funding options.

Appendix A: Learning Agenda

SUSTAINABILITY
S-1: What climate action measures and decarbonization strategies will produce the most return on investment for greenhouse gas reductions, improved air quality, green jobs and increased equity?
S-2: In what ways can the impact of conservation efforts (e.g., Multiple Species Conservation Program) on biodiversity (natural) and community (social/people) be quantified?
S-3: a) How are the commitments made in the Departmental Sustainability Plans aligned with the goals referenced in the Regional Decarbonization Framework (RDF)? b) Is the County on track to reach regional, State, and federal goals for impact category (materials, energy, water, biodiversity, emissions, waste) which are also referenced in the RDF?
S-4: a) What are the short- and long-term costs and benefits of investing in the County's aging infrastructure such as roads, flood control and water quality?  b) How does infrastructure investment compare to the costs and benefits of delaying or neglecting such investments? c) What impact does delaying infrastructure investment have on underserved communities and the creation of high quality, well-paying jobs?
S-5: To what extent do alternative delivery strategies for green stormwater infrastructure projects such as design-build and community-based public-private partnerships impact the speed, quality, and co-benefits delivered (e.g., social, economic, and environmental) in comparison to traditional delivery methods like design-bid-build?
EQUITY
EQ-1: a) How should the County measure equity and changes in equity? b) To what extent is the use of the equity lens by County programs impacting equity in underserved or disadvantaged communities? c) How can the County incorporate or advance the use of equity in the design, delivery and evaluation of programs and services?
EQ-2: a) To what extent is the County’s procurement process contributing to equitable economic growth and development? b) What are alternative procurement models that could support a stronger positive impact on equitable economic growth and development?
EQ-3: How has the two-year Childcare Blueprint changed the County childcare system?
EQ-4: a) To what extent are County investments in early care and education programs resulting in high quality programs? b) To what extent are these investments impacting the long-term development of children? c) Which investment factors contribute the most to desired student and family outcomes?
EQ-5: a) What programs or strategies are most effective at preventing homelessness? b) What programs or strategies are most effective at reducing homelessness?
EQ-6: a) How can the effectiveness of Homeless Solutions and Equitable Communities (HSEC) community engagement efforts be evaluated? b) To what extent do each of the HSEC community engagement channels contribute to the measures of effectiveness?
EQ-7: To what extent are County projects funded through the Centers for Disease Control and Prevention Health Disparities Grant producing the desired outcomes?
EQ-8: a) To what extent did County outreach efforts increase participation of seniors aged 50 or older, regardless of immigration status, in the Medi-Cal Program? b) To what extent were there differences in results between outreach channels? c) What factors may be responsible for differences in the effectiveness of channels?
EQ-9: What measures or strategies can the County take to increase business compliance with employment laws and regulations?
EQ-10: What would be the impact of increasing the involvement of people with disabilities in County services, programs, and activities?
EQ-11: a) What is the demand for medical care across all regions of the County for low-acuity patients who are usually transported to hospitals, regardless of insurance status? b) Are there available resources in all regions of the County to satisfy this demand? c) What actions can the County take to reduce the number of these patients transported to hospitals?
EMPOWER
EM-1: a) What County data is of greatest interest to the community? b) To what extent is the data of interest available, accessible and usable?
EM-2: a) In what ways can the results of preventive regulatory (e.g., environmental health) programs be quantified?  b) To what extent do preventive regulatory programs relate to cost savings to the County. c) What are effective ways to use results and savings data for funding allocation? d) Are there any barriers that hinder the public’s understanding or ability to submit complaints of regulatory violations?  e) Do those or other barriers create inequities in the enforcement of preventive regulations?
EM-3: a) To what extent do County programs include lived experience in program design and implementation? b) What high impact approaches could expand the inclusion of lived experience in program design and implementation?
EM-4: a) To what extent are recent legislative changes affecting the operations and outcomes of the Office of Revenue and Recovery? b) Within the context of the effect of these legislative changes, what alternative methods to the operations of the Office of Revenue and Recovery are most likely to achieve best value to the County and community?
EM-5: How can the Office of Labor Standards and Enforcement use data and programs from across the county to expand the reach of its activities, and improve the recovery of unpaid wage theft judgements (i.e., wages that the State Labor Commissioner has already determined are owed to workers) in San Diego County?
EM-6: a) Does the County have any programs or services that create unnecessary administrative burdens—such as complex, confusing or time-consuming processes—for the people who try to access them? b) What measures can the County take to reduce these administrative burdens?
EM-7: a) To what degree does the County experience problems with employee recruitment, hiring and retention? b) What measures or strategies can the County take to address these problems?
COMMUNITY
C-1: To what extent is the Behavioral Health Services’ Continuum of Care affecting the community and the lives of the individuals served?
C-2: a) To what extent does the Library High School Program impact graduates’ future formal education and career advancement or increased earnings? b) To what extent are there differences in these outcomes between different subgroups? c) If differences exist, what may account for those differences?
C-3: a) What population subgroups in the County experience the highest degrees of social isolation affecting their mental wellbeing? b) Which strategies to address social isolation (e.g., technology training, friendly visits, phone calls, etc.) are most effective at reducing isolation and promoting mental well-being?
C-4: What percentage of eligible people are being referred to or are receiving services from the Medical Care Services department?
C-5: To what extent is increased investment in prevention services impacting the rate of child abuse and neglect?
C-6: a) To what degree do different population groups distrust County government? b) What measures or strategies could the County take to increase trust?

JUSTICE
J-1: How can the County use alternatives to incarceration approaches to reduce the number of people involved in the justice system?
J-2: a) To what extent has the Public Defender’s Immigrant Rights Legal Defense Program impacted access to legal representation for detained—or on alternatives to detention (ATD)—immigrants facing removal proceedings in San Diego County? b) To what extent are detained/ATD immigrants experiencing greater success (released from custody and/or avoiding deportation) in their removal proceedings due to representation by the Immigrant Rights Legal Defense Program?
J-3: How can the Probation Department use training to reduce use of force incidents, youth altercations, and employee and youth injuries?
J-4: a) To what extent do investments in Crisis Stabilization Units (CSUs) impact mental health and law enforcement resources? b) To what extent do investments in CSUs impact short- and long-term outcomes for individuals contacted by law enforcement in mental health crises?

Appendix B: From Learning Agenda to Learning Activities

To ensure that the County’s research and evaluation efforts are aligned to our strategic priorities, OEPA has established a process to develop Learning Activities from Learning Agenda questions.
Box one says learning agenda with arrow pointing to box 2, pipeline of potential projects. Box 2 has arrow pointing to box 3, learning activities
Continuous Engagement with Business Groups
OEPA will designate lead points of contact with each of the County’s business groups. They will engage regularly with business groups and their departments and will identify opportunities to discuss potential research or evaluation projects of relevance to questions in the learning agenda.
There will be other pathways to identify potential projects. For example, Board direction or requirements from program funding contracts may directly lead to the identification of research or evaluation projects related to the learning agenda.
Pipeline of Potential Research and Evaluation Projects
Once a potential project is identified, OEPA will work with the relevant partners to assess the project’s feasibility and define the minimum required elements to get the project ready for implementation. These elements include project description, research/evaluation questions, necessary data, analysis methodology, project lead, and estimated budget. When funding is required, the project will remain in the potential project pipeline until the funding is secured or the team has reasonable certainty that the project will be funded within a 12-month period.
Learning Activities
Once a project has been deemed feasible, all the required elements mentioned above are defined and the team agrees that the project is “ready to go”, the project will become a learning activity for the County and be added to the Annual Strategic Research Plan.