In the face of proliferating data sources and research technologies, budget cutbacks, and the growing prominence of performance-based programs and evidence-based policy-making, there has been an increase in demand for faster, cheaper program evaluation. To address this need, the Center for Improving Research Evidence (CIRE) at Mathematica Policy Research has produced the Evidence Insight video series featuring four innovative research methods by different Mathematica researchers. The goal of the series is to make policy makers, researchers and practitioners aware of the strengths, limitations and contexts of these new evaluation techniques and enable continuous improvement of program delivery and effectiveness.
Summaries of the transcribed video of each of these techniques follow:
Rapid-Cycle Evaluation: Determining What Works in Less Time
Small changes to program operations and service delivery can improve the efficiency and effectiveness of the program. This video delivered by Alex Resch of Mathematica, describes how rapid-cycle evaluation can quickly determine the impact of these changes. Such evaluations rely predominantly on existing program data and rigorous research methods to measure impacts of changes on program delivery, to identify causal links between the changes and the outcomes, and to focus on results that can be observed in a short time. Because rapid-cycle evaluations proceed relatively quickly, the results give decision makers and practitioners the timely and reliable evidence needed for continual program improvement.
CIRE #EvidenceInsight Video Series – Rapid Cycle Evaluation from Mathematica Policy Research on Vimeo.
Bayesian Methods: A Faster, Probabilistic Approach to Research Design
Bayesian methods allow researchers to rigorously use external data to supplement traditional data collection sources and techniques to gain insights about programs and policies that would not have been possible with the traditional evaluation techniques alone. In this video, Lauren Vollmer of Mathematica explains how these methods produce precise probabilities, enabling policymakers to target programs and policies to meet the needs of specific audiences. Moreover, the increased precision enables detection of meaningful affects even when sample sizes are small, thus reducing the cost of data collection. And the ability to draw conclusions in probabilistic terms along a spectrum offers researchers and policymakers greater opportunity to apply subject-matter expertise in determining whether effects are meaningful in context. While there are costs involved, such as sometimes unreliable external data and computational complexity, these obstacles have been met by the development of comprehensive guidelines and sensitivity checks as well as advances in computation that enable faster and less computationally draining Bayesian analysis.
CIRE #EvidenceInsight Video Series – Bayesian Methods from Mathematica Policy Research on Vimeo.
Adaptive Randomization: A Fresh Perspective on Traditional Research Design
Randomized control trials (RCTs) are considered to be the gold standard for providing valid and rigorous evidence of program evidence by controlling for non-program factors such as the characteristics of program participants that possibly contributed to observed outcomes. However, they can be time-consuming and expensive to implement. In this video, Mariel Finucane and Ignacio Martinez discuss adaptive randomization, an innovative extension of traditional RCTs. Faster and cheaper than RCTs, adaptive randomization enables researchers to adjust their methods in response to accumulating evidence without losing the methodological principles of randomization. This new research method, when combined with access to rich datasets such as electronic health records, contributes, in efficient and rigorous fashion, to our understanding of how specific program elements can be refined to better meet the needs of particular subgroups.
CIRE #EvidenceInsight Video Series – Adaptive Randomization from Mathematica Policy Research on Vimeo.
Predictive Analytics: Transforming Decision Making in Three Steps
Predictive analytics is an emerging research field in which data and evidence are used to predict outcomes and inform decision making. In this video, Phil Killewald of Mathematica breaks the predictive process down into three steps:
- Define the problem: gain thorough knowledge of the extent and quality relevant data bases and of agency activities; and pose evaluation questions to address shortcomings in program effectiveness;
- Design the predictive model: gather historical data on past activities where a shortcoming is to be addressed; design, build and evaluate a number of predictive models using these data; and test the model using separate data in order to identify the best performing models; and
- Make the predictions: use the best performing model to predict outcomes with the real-world data where the true outcomes are not yet known, make the model available as early as possible, and periodically assess its accuracy and update it as necessary or as new data become available.
When applied to policy-relevant research questions, predictive analytics can be a valuable tool for organizations seeking to improve their services and their decision making processes by identifying what works for whom under what conditions on the basis of evidence-based outcome predictions.
CIRE #EvidenceInsight Video Series – Predictive Analytics from Mathematica Policy Research on Vimeo.
This information should increase awareness of and amenability of these new techniques among not only researchers, but also practitioners seeking to improve the effectiveness of the outcomes they deliver.
Hello! I realize this is kind of off-topic however I had to
ask. Does operating a well-established blog such as yours require a large amount of work?
I’m brand new to writing a blog but I do write in my journal on a daily basis.
I’d like to start a blog so I will be able to share
my own experience and thoughts online. Please let me know if you have any
kind of ideas or tips for new aspiring blog owners. Appreciate it!
Thanks for finally talking about > Mathematica's #EvidenceInsight Video Series: Cutting-Edge Methods in Policy Research – BC
Centre for Employment ExcellenceBC Centre for Employment Excellence < Liked it!
hello!,I love your writing very much! percentage we keep in touch more approximately your post on AOL?
I need an expert on this area to solve my problem. May be that is you!
Having a look forward to peer you.
Post writing is also a excitement, if you be acquainted with after that you can write or else it
is complicated to write.
Hey there are using WordPress for your site platform?
I’m new to the blog world but I’m trying to get started and set up
my own. Do you require any html coding expertise to make your own blog?
Any help would be really appreciated!
There is certainly a great deal to know about
this issue. I like all of the points you have
made.
Hello colleagues, good post and good urging commented at this place,
I am really enjoying by these.
Hi! I’ve been following your website for a while now and finally got the courage to go ahead and give you a shout out from Atascocita Tx!
Just wanted to say keep up the good job!
I just like the valuable information you supply
in your articles. I’ll bookmark your weblog and take a look at again right here frequently.
I’m relatively certain I’ll learn many new stuff right here!
Good luck for the next!
I was wondering if you ever considered changing the structure
of your website? Its very well written; I love what youve got to say.
But maybe you could a little more in the way of content so people could connect with it better.
Youve got an awful lot of text for only having 1 or 2 pictures.
Maybe you could space it out better?
Do you have a spam issue on this website;
I also am a blogger, and I was wondering your situation; many of
us have developed some nice methods and we are looking to exchange
techniques with other folks, why not shoot me an email if interested.
Thanks designed for sharing such a pleasant thought,
article is nice, thats why i have read it entirely
Appreciate this post. Let me try it out.
It’s amazing to pay a quick visit this web page and reading the views
of all friends on the topic of this article,
while I am also zealous of getting experience.
Hello there, You have done an incredible job. I’ll definitely digg it and personally
recommend to my friends. I’m confident they’ll be benefited from
this site.
naturally like your website but you have to check the
spelling on quite a few of your posts. A number
of them are rife with spelling problems and I in finding it very troublesome
to tell the reality on the other hand I will
surely come back again.
Hi there, I discovered your web site by way of Google while searching for a comparable
topic, your web site got here up, it seems good.
I’ve bookmarked it in my google bookmarks.
Hi there, just become aware of your weblog via Google, and located that it’s really informative.
I’m going to be careful for brussels. I’ll be grateful if you proceed this in future.
Many other people might be benefited from your writing. Cheers!
Why visitors still use to read news papers when in this technological globe all is accessible
on web?
I’m gone to say to my little brother, that he should also go to see this webpage on regular basis to obtain updated from latest gossip.
adreamoftrains best hosting
This blog was… how do you say it? Relevant!! Finally I have found something that helped me.
Thank you!
This is my first time visit at here and i am actually
impressed to read everthing at alone place.
You can definitely see your skills within the article you write.
The arena hopes for even more passionate writers like you
who are not afraid to say how they believe. Always follow your heart.
I could not refrain from commenting. Well written!
Currently it sounds like Movable Type is the top blogging platform available right now.
(from what I’ve read) Is that what you’re using on your blog?
Normally I do not read post on blogs, but I would like to say that
this write-up very compelled me to try and do
so! Your writing taste has been amazed me. Thanks, very great
article.
Everyone loves it whenever people get together and share opinions.
Great site, keep it up!
I don’t even know how I ended up here, but I thought this post
was great. I don’t know who you are but certainly you are going to a famous blogger if you
are not already 😉 Cheers!
Hello i am kavin, its my first time to commenting anyplace, when i read this
post i thought i could also make comment due to this good post.
Great goods from you, man. I’ve bear in mind your stuff prior to and you are simply too fantastic.
I actually like what you’ve got right here, really like
what you’re stating and the way in which through which you assert it.
You’re making it enjoyable and you still care for to
stay it smart. I cant wait to read much more from you.
This is really a great site.