Mathematica’s #EvidenceInsight Video Series: Cutting-Edge Methods in Policy Research

In the face of proliferating data sources and research technologies, budget cutbacks, and the growing prominence of performance-based programs and evidence-based policy-making, there has been an increase in demand for faster, cheaper program evaluation. To address this need, the Center for Improving Research Evidence (CIRE) at Mathematica Policy Research has produced the Evidence Insight video series featuring four innovative research methods by different Mathematica researchers.  The goal of the series is to make policy makers, researchers and practitioners aware of the strengths, limitations and contexts of these new evaluation techniques and enable continuous improvement of program delivery and effectiveness.

Summaries of the transcribed video of each of these techniques follow:

Rapid-Cycle Evaluation: Determining What Works in Less Time

Small changes to program operations and service delivery can improve the efficiency and effectiveness of the program. This video delivered by Alex Resch of Mathematica, describes how rapid-cycle evaluation can quickly determine the impact of these changes. Such evaluations rely predominantly on existing program data and rigorous research methods to measure impacts of changes on program delivery, to identify causal links between the changes and the outcomes, and to focus on results that can be observed in a short time. Because rapid-cycle evaluations proceed relatively quickly, the results give decision makers and practitioners the timely and reliable evidence needed for continual program improvement.

CIRE #EvidenceInsight Video Series – Rapid Cycle Evaluation from Mathematica Policy Research on Vimeo.

Bayesian Methods: A Faster, Probabilistic Approach to Research Design

Bayesian methods allow researchers to rigorously use external data to supplement traditional data collection sources and techniques to gain insights about programs and policies that would not have been possible with the traditional evaluation techniques alone. In this video, Lauren Vollmer of Mathematica explains how these methods produce precise probabilities, enabling policymakers to target programs and policies to meet the needs of specific audiences. Moreover, the increased precision enables detection of meaningful affects even when sample sizes are small, thus reducing the cost of data collection. And the ability to draw conclusions in probabilistic terms along a spectrum offers researchers and policymakers greater opportunity to apply subject-matter expertise in determining whether effects are meaningful in context. While there are costs involved, such as sometimes unreliable external data and computational complexity, these obstacles have been met by the development of comprehensive guidelines and sensitivity checks as well as advances in computation that enable faster and less computationally draining Bayesian analysis.

CIRE #EvidenceInsight Video Series – Bayesian Methods from Mathematica Policy Research on Vimeo.

Adaptive Randomization: A Fresh Perspective on Traditional Research Design

Randomized control trials (RCTs) are considered to be the gold standard for providing valid and rigorous evidence of program evidence by controlling for non-program factors such as the characteristics of program participants that possibly contributed to observed outcomes. However, they can be time-consuming and expensive to implement. In this video, Mariel Finucane and Ignacio Martinez discuss adaptive randomization, an innovative extension of traditional RCTs. Faster and cheaper than RCTs, adaptive randomization enables researchers to adjust their methods in response to accumulating evidence without losing the methodological principles of randomization. This new research method, when combined with access to rich datasets such as electronic health records, contributes, in efficient and rigorous fashion, to our understanding of how specific program elements can be refined to better meet the needs of particular subgroups.

CIRE #EvidenceInsight Video Series – Adaptive Randomization from Mathematica Policy Research on Vimeo.

Predictive Analytics: Transforming Decision Making in Three Steps

Predictive analytics is an emerging research field in which data and evidence are used to predict outcomes and inform decision making. In this video, Phil Killewald of Mathematica breaks the predictive process down into three steps:

  • Define the problem: gain thorough knowledge of the extent and quality relevant data bases and of agency activities; and pose evaluation questions to address shortcomings in program effectiveness;
  • Design the predictive model: gather historical data on past activities where a shortcoming is to be addressed; design, build and evaluate a number of predictive models using these data; and test the model using separate data in order to identify the best performing models; and
  • Make the predictions: use the best performing model to predict outcomes with the real-world data where the true outcomes are not yet known, make the model available as early as possible, and periodically assess its accuracy and update it as necessary or as new data become available.

When applied to policy-relevant research questions, predictive analytics can be a valuable tool for organizations seeking to improve their services and their decision making processes by identifying what works for whom under what conditions on the basis of evidence-based outcome predictions.

CIRE #EvidenceInsight Video Series – Predictive Analytics from Mathematica Policy Research on Vimeo.

This information should increase awareness of and amenability of these new techniques among not only researchers, but also practitioners seeking to improve the effectiveness of the outcomes they deliver.

157 Comments

  1. Hello! I realize this is kind of off-topic however I had to
    ask. Does operating a well-established blog such as yours require a large amount of work?
    I’m brand new to writing a blog but I do write in my journal on a daily basis.
    I’d like to start a blog so I will be able to share
    my own experience and thoughts online. Please let me know if you have any
    kind of ideas or tips for new aspiring blog owners. Appreciate it!

  2. Hey there are using WordPress for your site platform?

    I’m new to the blog world but I’m trying to get started and set up
    my own. Do you require any html coding expertise to make your own blog?
    Any help would be really appreciated!

  3. I just like the valuable information you supply
    in your articles. I’ll bookmark your weblog and take a look at again right here frequently.
    I’m relatively certain I’ll learn many new stuff right here!
    Good luck for the next!

  4. I was wondering if you ever considered changing the structure
    of your website? Its very well written; I love what youve got to say.
    But maybe you could a little more in the way of content so people could connect with it better.
    Youve got an awful lot of text for only having 1 or 2 pictures.

    Maybe you could space it out better?

  5. Do you have a spam issue on this website;
    I also am a blogger, and I was wondering your situation; many of
    us have developed some nice methods and we are looking to exchange
    techniques with other folks, why not shoot me an email if interested.

  6. naturally like your website but you have to check the
    spelling on quite a few of your posts. A number
    of them are rife with spelling problems and I in finding it very troublesome
    to tell the reality on the other hand I will
    surely come back again.

  7. Hi there, I discovered your web site by way of Google while searching for a comparable
    topic, your web site got here up, it seems good.
    I’ve bookmarked it in my google bookmarks.

    Hi there, just become aware of your weblog via Google, and located that it’s really informative.
    I’m going to be careful for brussels. I’ll be grateful if you proceed this in future.
    Many other people might be benefited from your writing. Cheers!

  8. You can definitely see your skills within the article you write.
    The arena hopes for even more passionate writers like you
    who are not afraid to say how they believe. Always follow your heart.

  9. Normally I do not read post on blogs, but I would like to say that
    this write-up very compelled me to try and do
    so! Your writing taste has been amazed me. Thanks, very great
    article.

  10. Great goods from you, man. I’ve bear in mind your stuff prior to and you are simply too fantastic.
    I actually like what you’ve got right here, really like
    what you’re stating and the way in which through which you assert it.
    You’re making it enjoyable and you still care for to
    stay it smart. I cant wait to read much more from you.
    This is really a great site.

  11. Awesome blog you have here but I was wondering if you knew of any message boards that cover the same topics talked about in this article?
    I’d really love to be a part of group where I can get responses from other knowledgeable individuals that share the same interest.
    If you have any recommendations, please let me know.
    Many thanks!

  12. great submit, very informative. I’m wondering why the other specialists of
    this sector do not notice this. You should continue your writing.
    I am sure, you have a great readers’ base already!

  13. Hi there, I discovered your website by the use of
    Google whilst looking for a similar subject, your website got here
    up, it appears to be like great. I have bookmarked it in my google bookmarks.

    Hi there, just changed into aware of your weblog through Google, and found that it’s
    really informative. I’m going to be careful for brussels.

    I’ll be grateful for those who proceed this in future. Many other people
    can be benefited out of your writing. Cheers!

  14. Today, I went to the beach with my children. I found a sea shell and gave it to my 4 year old
    daughter and said “You can hear the ocean if you put this to your ear.” She
    put the shell to her ear and screamed. There was a hermit crab inside and it pinched her ear.
    She never wants to go back! LoL I know this is entirely off topic but I had
    to tell someone!

  15. Excellent beat ! I would like to apprentice while you amend your web site, how could i subscribe for a blog website?
    The account aided me a acceptable deal. I had been tiny bit acquainted of this your broadcast offered bright clear idea y2yxvvfw cheap
    flights

  16. Awesome blog! Do you have any tips and hints for aspiring writers?

    I’m hoping to start my own blog soon but I’m a little lost on everything.
    Would you suggest starting with a free platform like WordPress or go for a paid option? There are so many options out there that
    I’m completely confused .. Any tips? Thank you!

    3aN8IMa cheap flights

  17. Heya! I realize this is kind of off-topic however I had to ask.
    Does building a well-established blog such as yours take a massive amount work?
    I’m brand new to blogging but I do write in my journal on a daily basis.
    I’d like to start a blog so I can easily share my experience and feelings online.
    Please let me know if you have any kind of ideas or tips for brand new aspiring bloggers.
    Appreciate it! cheap flights 34pIoq5

  18. I needed to thank you for this fantastic read!! I definitely enjoyed every little bit of it.

    I’ve got you saved as a favorite to check out new things you post… 3gqLYTc cheap flights

  19. Hmm is anyone else experiencing problems with the images on this blog
    loading? I’m trying to find out if its a problem on my end or
    if it’s the blog. Any suggestions would be greatly appreciated.

  20. We stumbled over here from a different website and thought I should check things out.
    I like what I see so now i am following you.

    Look forward to looking at your web page yet again.

  21. Hey! I understand this is kind of off-topic but I needed to ask.

    Does managing a well-established blog like yours require a lot of
    work? I’m brand new to operating a blog however I do
    write in my journal daily. I’d like to start a blog so I
    can easily share my personal experience and thoughts online.
    Please let me know if you have any ideas or tips for new aspiring blog owners.
    Appreciate it!

  22. Its like you read my mind! You appear to know a lot about this, like you wrote the
    book in it or something. I think that you can do with some pics to drive the message home
    a bit, but instead of that, this is excellent blog. A fantastic read.
    I’ll certainly be back.

  23. I know this if off topic but I’m looking into starting my own blog and was curious what all is needed to get set up?
    I’m assuming having a blog like yours would cost a pretty penny?
    I’m not very internet savvy so I’m not 100% sure.
    Any suggestions or advice would be greatly appreciated.
    Appreciate it

  24. Howdy I am so glad I found your site, I really found you by accident,
    while I was looking on Google for something else, Regardless I am here now and would just like to say
    many thanks for a incredible post and a all round entertaining blog (I also love the theme/design), I don’t have
    time to look over it all at the moment but I have saved it and
    also added your RSS feeds, so when I have time I will be back to read a great deal more, Please
    do keep up the awesome work.

  25. Can I simply say what a relief to uncover someone that truly knows
    what they are talking about on the net. You certainly know how to bring an issue
    to light and make it important. A lot more people ought to look at this and understand this side of your story.

    I was surprised you’re not more popular since you most certainly have the gift.

  26. Thank you, I have just been looking for information approximately
    this topic for a while and yours is the greatest
    I’ve discovered till now. However, what in regards to the bottom line?

    Are you positive concerning the source?

  27. I don’t even know how I ended up here, but I thought this post
    was good. I do not know who you are but certainly you’re going to a famous blogger if you aren’t already 😉 Cheers!

  28. Great post. I was checking constantly this blog and I am inspired!

    Extremely helpful information specifically the last section 🙂 I take care of such
    info a lot. I used to be seeking this certain info for a very lengthy time.
    Thank you and best of luck.

  29. Hi there! I could have sworn I’ve been to this
    blog before but after checking through some of the post I realized it’s new to me.
    Nonetheless, I’m definitely happy I found it and I’ll be book-marking and checking back often!

  30. Howdy! This is kind of off topic but I need some guidance from an established blog.
    Is it tough to set up your own blog? I’m not very
    techincal but I can figure things out pretty quick.
    I’m thinking about creating my own but I’m not sure where to begin. Do you have any points or suggestions?
    With thanks

  31. Fantastic goods from you, man. I’ve understand your stuff previous to and you are just too
    magnificent. I really like what you have acquired here, really like what you are stating and the way in which you say it.
    You make it entertaining and you still care for to keep it sensible.
    I cant wait to read far more from you. This is actually a terrific web site.

  32. I was curious if you ever thought of changing the structure of your site?
    Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better.
    Youve got an awful lot of text for only having 1 or
    2 images. Maybe you could space it out better?

  33. I’m extremely impressed with your writing skills
    and also with the layout on your weblog. Is this a paid theme
    or did you modify it yourself? Anyway keep up the nice quality writing, it is rare to see a nice blog like this one today.

  34. Hmm is anyone else experiencing problems with the
    pictures on this blog loading? I’m trying to figure out if its a problem
    on my end or if it’s the blog. Any responses would be greatly appreciated.

  35. Great beat ! I would like to apprentice even as you amend your website, how could i subscribe for a weblog site?
    The account aided me a acceptable deal. I have been a little bit familiar
    of this your broadcast offered brilliant clear idea

  36. An interesting discussion is worth comment. I believe that
    you ought to write more on this issue, it may not be a taboo matter but generally folks don’t speak
    about such issues. To the next! Kind regards!!

  37. What you published made a bunch of sense. But, think on this, what if you
    wrote a catchier post title? I mean, I don’t want to tell you
    how to run your website, but suppose you added a post title
    that grabbed people’s attention? I mean Mathematica's #EvidenceInsight Video Series: Cutting-Edge Methods in Policy Research – BC Centre for Employment ExcellenceBC
    Centre for Employment Excellence is kinda vanilla. You might glance at Yahoo’s front
    page and watch how they create post headlines to get people to click.
    You might add a video or a related pic or
    two to get people interested about everything’ve got to
    say. Just my opinion, it could bring your website a little bit more interesting.

  38. Greetings from Carolina! I’m bored at work so I decided to browse your site on my iphone during lunch break.
    I really like the knowledge you present here and
    can’t wait to take a look when I get home.
    I’m amazed at how quick your blog loaded on my phone ..

    I’m not even using WIFI, just 3G .. Anyhow, amazing site!

  39. Wow that was unusual. I just wrote an extremely long comment
    but after I clicked submit my comment didn’t appear. Grrrr…
    well I’m not writing all that over again. Anyways,
    just wanted to say superb blog!

  40. I blog frequently and I genuinely thank you for your information. The article has truly peaked
    my interest. I’m going to bookmark your website and
    keep checking for new details about once a week.
    I subscribed to your Feed as well.

  41. An impressive share! I’ve just forwarded this onto a colleague who has been doing a little homework on this.
    And he actually ordered me lunch simply because I found it for him…
    lol. So allow me to reword this…. Thanks for the meal!!
    But yeah, thanx for spending time to discuss this issue here on your web site.

  42. Hey I know this is off topic but I was wondering if you knew of any
    widgets I could add to my blog that automatically tweet my newest
    twitter updates. I’ve been looking for a plug-in like this for quite some time and was hoping maybe
    you would have some experience with something like this.
    Please let me know if you run into anything.
    I truly enjoy reading your blog and I look forward to your new updates.

  43. Hi there! Quick question that’s totally off topic. Do you know how to make your site mobile friendly?
    My blog looks weird when viewing from my iphone 4.
    I’m trying to find a template or plugin that might be able to resolve this problem.
    If you have any suggestions, please share.
    With thanks!

  44. Superb website you have here but I was curious if you knew of any message boards that cover the same topics discussed in this article?
    I’d really love to be a part of online community where I can get
    suggestions from other knowledgeable people that share the same interest.
    If you have any suggestions, please let me know.
    Cheers!

  45. Have you ever thought about adding a little bit more
    than just your articles? I mean, what you say is valuable and everything.
    However think about if you added some great images or videos to give your posts more, “pop”!
    Your content is excellent but with pics and video clips, this blog could undeniably be one of the
    very best in its niche. Wonderful blog!

  46. I got this web site from my buddy who told me on the topic of this web site and now this time I am browsing this web page and reading very informative posts here.

  47. Hey I am so glad I found your site, I really found you by error, while I was browsing on Yahoo for something
    else, Regardless I am here now and would just like to say thanks for a
    fantastic post and a all round entertaining blog (I also love the theme/design),
    I don’t have time to browse it all at the moment but I have book-marked it and also added your RSS feeds, so
    when I have time I will be back to read a great deal more, Please do keep up the fantastic work.

  48. It is the best time to make some plans for the future and it’s time to be happy.

    I’ve read this post and if I could I wish to suggest you few interesting
    things or tips. Perhaps you can write next articles referring to this article.
    I desire to read even more things about it!

  49. Good day! I could have sworn I’ve been to this site before but after browsing through some of
    the post I realized it’s new to me. Anyhow, I’m definitely glad I
    found it and I’ll be bookmarking and checking back often!

  50. Every weekend i used to pay a quick visit this web site, because i want
    enjoyment, for the reason that this this website conations actually fastidious funny data too.

  51. Howdy! This is my first comment here so
    I just wanted to give a quick shout out and tell
    you I really enjoy reading your posts. Can you
    recommend any other blogs/websites/forums that cover the same subjects?

    Appreciate it!

  52. Hey there this is kind of of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML.
    I’m starting a blog soon but have no coding know-how so I wanted to
    get guidance from someone with experience. Any help would be
    greatly appreciated!

  53. Hi just wanted to give you a brief heads up and let you know a few of the pictures
    aren’t loading correctly. I’m not sure why but I think its a linking issue.

    I’ve tried it in two different internet browsers and both show the same outcome.

  54. Does your website have a contact page? I’m having trouble locating it but, I’d like to send you an email.

    I’ve got some ideas for your blog you might be interested in hearing.
    Either way, great blog and I look forward to seeing it
    grow over time.

  55. May I simply say what a comfort to discover somebody who actually understands what they’re discussing on the net.
    You definitely realize how to bring an issue to light and make it important.
    More people have to read this and understand this
    side of your story. I was surprised that you are not more popular
    since you certainly have the gift.

  56. I’ve been surfing on-line more than three hours these days, but I by no means found any attention-grabbing article like yours.

    It is pretty value enough for me. Personally, if all webmasters and bloggers made excellent content material
    as you did, the web can be a lot more useful than ever before.

  57. Nice post. I was checking constantly this weblog and I’m inspired!
    Very useful info specifically the remaining phase 🙂 I maintain such information much.
    I used to be seeking this certain information for a long time.
    Thanks and good luck.

  58. What i don’t understood is in reality how you are no longer really a
    lot more neatly-favored than you may be right now. You are very intelligent.
    You already know therefore considerably in the case of this subject, made me for my part consider it from so many varied angles.
    Its like women and men don’t seem to be fascinated until it’s something to accomplish
    with Lady gaga! Your individual stuffs outstanding.
    All the time take care of it up!

  59. Attractive portion of content. I simply stumbled upon your
    blog and in accession capital to claim that I get actually loved account your
    blog posts. Anyway I’ll be subscribing for your augment or even I success you get
    entry to consistently rapidly.

  60. My developer is trying to persuade me to move to .net from PHP.
    I have always disliked the idea because of the costs.
    But he’s tryiong none the less. I’ve been using WordPress on various websites
    for about a year and am nervous about switching
    to another platform. I have heard good things about
    blogengine.net. Is there a way I can import all my wordpress posts into it?
    Any kind of help would be really appreciated!

  61. May I simply just say what a comfort to uncover somebody who
    truly understands what they’re discussing over the internet.
    You definitely understand how to bring an issue to light and make it important.
    More people ought to look at this and understand this side of the story.

    I can’t believe you’re not more popular since you definitely have the gift.

  62. The other day, while I was at work, my sister stole my apple ipad and tested to see
    if it can survive a thirty foot drop, just so she can be a youtube sensation. My iPad is now destroyed
    and she has 83 views. I know this is entirely off topic but I had to share it with
    someone!

  63. Nice post. I learn something totally new and challenging on websites I stumbleupon on a daily basis.
    It will always be interesting to read through content from other authors and use a
    little something from other websites.

  64. When I originally commented I clicked the “Notify me when new comments are added” checkbox and now
    each time a comment is added I get three e-mails with the same
    comment. Is there any way you can remove me from that service?

    Thanks!

  65. Woah! I’m really digging the template/theme
    of this blog. It’s simple, yet effective. A lot of times it’s difficult to get that “perfect balance” between superb usability and visual appearance.
    I must say you have done a superb job with this.
    Additionally, the blog loads extremely quick for me on Safari.
    Excellent Blog!

  66. I was recommended this website through my cousin. I am now not sure whether or not this put
    up is written by means of him as nobody else recognise such
    specific about my difficulty. You are wonderful!

    Thanks!

  67. First of all I want to say wonderful blog! I had a quick question which I’d like
    to ask if you don’t mind. I was curious to know how you center yourself and clear your head prior
    to writing. I’ve had a difficult time clearing my thoughts
    in getting my thoughts out. I do enjoy writing but it just seems like the first 10 to
    15 minutes are generally wasted just trying to
    figure out how to begin. Any suggestions or tips? Kudos!

  68. Hi there, just became alert to your blog through Google,
    and found that it is truly informative. I’m going to watch out for brussels.

    I will be grateful if you continue this in future.
    Lots of people will be benefited from your writing.

    Cheers!

  69. Hi! Do you know if they make any plugins to assist with SEO?
    I’m trying to get my blog to rank for some targeted keywords but I’m
    not seeing very good gains. If you know of any please share.

    Many thanks!

  70. Undeniably believe that which you said. Your favorite reason appeared to be
    on the web the easiest thing to be aware of.
    I say to you, I definitely get annoyed while people think about worries
    that they just don’t know about. You managed to hit the nail
    upon the top and also defined out the whole thing without having side-effects , people could take a signal.
    Will likely be back to get more. Thanks

  71. naturally like your web-site but you need to test the spelling on quite a few of your posts.
    A number of them are rife with spelling problems
    and I to find it very bothersome to inform the truth on the other hand I’ll surely come again again.

  72. Hello this is somewhat of off topic but I was wanting to know if blogs
    use WYSIWYG editors or if you have to manually code
    with HTML. I’m starting a blog soon but have no coding skills so
    I wanted to get guidance from someone with experience.
    Any help would be greatly appreciated!

Leave a Reply to adreamoftrains web hosting service Cancel reply

Your email address will not be published. Required fields are marked *