1 / 12

Learning from Disagreeing Demonstrators

Learning from Disagreeing Demonstrators. Bruno N. da Silva University of British Columbia bnds@cs.ubc.ca. Motivation. Some traditional cases of Learning from Demonstration assume a human expert In some (subjective) tasks, there might not be a single expert How to drive from point A to B.

keelia
Download Presentation

Learning from Disagreeing Demonstrators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Learning from Disagreeing Demonstrators Bruno N. da Silva University of British Columbia bnds@cs.ubc.ca

  2. Motivation • Some traditional cases of Learning from Demonstration assume a human expert • In some (subjective) tasks, there might not be a single expert • How to drive from point A to B

  3. Motivation • In general, these tasks involve more than one feature • e.g. in the driving domain, want to optimize travel time and number of crashes • Different contexts lead to different tradeoffs between features • Idiosyncratic demonstrators do not reflect on their routine approach to the problem

  4. Problem definition • How can we integrate idiosyncratic (disagreeing) demonstrations to form a homogeneous and effective policy?

  5. Solution • We extend the framework presented by Argall et al, 2007 • Traditional demonstrations in the first stage • Robot execution and human critique in the second stage • Robot collects critiques • Robot updates policy

  6. The 1st stage of the mechanism

  7. The 2nd stage of the mechanism

  8. A little more concretely… • The first stage can be interpreted as a set of datapoints (pm,an,c) • Perception pm • Actionan • Confidence on the mappingc • The criticism will affect the confidence • If praise the execution, increase c • If knock the execution, decrease c

  9. But let’s not be naïve • If demonstrators “lie” in the demonstration, they would “lie” in the criticism • Therefore, associate a reputation riwith each demonstration di • And update the confidence level carefully • c := c + ri * f(feedback)

  10. Adjusting reputation ranks • And adjust ribased on (lack of) improvement from di’s feedback • ri := ri +  * evaluation(feedback) • evaluation(.) can be interpreted as a Pareto improvement from the feedback

  11. Current investigations • Policy conversion? • Rate of conversion? • What are the long term effects on human demonstrators? • Frustration? • Repudiation? • Will critiques really be mindful?

  12. Thanks! • Questions?

More Related