FRC Scouting and Competitive Assessment

1. FRC Scouting and Competitive Assessment Jim Zondag - Chrysler Isaac Rife – BAE Systems 4/11/2013

2. About the Presenters Jim Zondag: Apps Development Manager - Chrysler Team 33 Team Leader FRC mentor since 1998. Isaac Rife: Sr. Mobility Systems Engineer - BAE Systems Team 33 Mechanical Mentor, Chief Scout. FRC mentor since 2002.

3. Quote of the Day “If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself,you will succumb in every battle” - Sun Tzu – The Art of War

4. Why Scout? • Scouting is very important, and is often undervalued. • Good quality, accurate scouting is one of the key ingredients to a successful FRC competition weekend. • Successful teams put considerable effort into scouting. • Without scouting, proper strategy developmentis almost impossible. • Good scouting info will allow all members of your alliances to make the most out of every match. • A good pick list is very important for Alliance Selection. • Even if you think your team will not be picked, ALWAYS make a pick list.

5. What is Scouting? • Scouting is as much about your own team as it is about the other teams. • The goal of scouting is to figure out how your team can best work with your two alliance partners against your three opponents, and to help produce a well thought out Pick-List. • Scouting is the process of collecting objective and subjective data on all of the teams at the event (including your own team). • There are several different types of scouting • Pre-scouting – before the event. • Pit Scouting - static review at events. • Match Scouting – observation and data collection of matches • Analytical methods – Statistical analysis of match results / data feeds.

6. Pre-Scouting • Advance preparation before the event. • This can often be accomplished by one person. • Software/Macros will help tremendously, if you know what you want, you can automate almost all of this task. • Compile results for all teams: • Match scores, Win/Loss, Ranking position, Draft position, Eliminations results, Judged Awards. • Despite the fact that we build new robots each year, results from prior years can be amazingly accurate. • There is very strong correlation between past performance and future results for almost all teams.

7. Pit Scouting • Also known as “Static Review” • Tour the pits and review all the machines. • Take notes on key attributes: Type of drivetrain, features, wheel type, etc. • Talk to the teams: Most teams love to tell you all about their robots. Ask them if they have made recent improvements. • Always check the facts teams tell you against actual performance on the field. Most teams have a tendency to be optimistic about their own performance. • Always involve your students in these efforts. They will learn a lot about the different ways that teams solve problems.

8. Match Scouting • Proper match scouting takes considerable effort and organization. You must watch EVERY match. • The exact data you collect will change each year with the game design. • We typically use a group of 7 students to do our data collection; one for each robot per match, plus a scouting leader / coordinator. • Goal is to collect objective and subjective data on each robot in each match. • Be sure to collect penalty data. • A good data set allows you to evaluate teams for both capability AND consistency. • Some teams collaborate together to do data collection.

9. Match Scouting • Many teams have trouble with scouting….students hate it and thus do a poor job. They have to want to do it. • Match scouting is a very tiring and sometimes boring task. Despite this, NEVER treat scouting like a lower tier task, it is one of the most important things your team will do at an event. • Recommend that students work in shifts. • We give scouting a lot of esteem on our team. We praise the scouts often for the quality of their data and pick list. • Our student scouting leader is an earned position, and students actually compete for this title. • This greatly improves the willingness of our students to do a good job.

10. Data Entry • Methods vary from pencil and paper lists, to databases, to iPad apps. • As with robot design, choose your scouting method based on your team’s ability and comfort level. • There is no one right answer, you must do what will work best for your team. • Personally, I find computers to be somewhat of a distraction in the match scouting process. Technology is great, but often comes with additional failure modes and new avenues for goofing off. • The batteries never die on a paper list. • We use a customized scouting sheet, redesigned each year. The key to paper scouting is a compact, easy to read and sort data set. • We use computerized methods in both the pre-scouting and analytical

11. Match Scouting Go through the schedule and highlight your matches. Then watch partners and opponents to help formulate a match strategy.

12. Data Acquisition Sheets Inspired by FRC 3929

13. Starting a Pick-List Always have a full team list handy when making a pick-list. Sort team by Yes, Maybe, or No. These three levels make it pretty easy to sort. You need at least 24 Yes and Maybes combined, or you have to move some over from the No pile… Yes: Definitely a team that will make Elims. Maybe : A team you are not 100% sure of early on. No: A team that will not make elims. Yes+Maybe>24 (23 + your team)

14. Making a Pick-List Always have a full team list handy when making a pick-list. It is important to have a full list of 24 teams (23 plus your team) as well as any possible additional teams (in case you and your first pick have conflicts). We also use DNP as a section for Do Not Pick. This can be because they are broken, get penalties, or just have a conflicting strategy.

15. Analytical Methods • Numerical Analysis is a big part of most successful teams’ scouting efforts. • By using the published match results from past matches and events, future match results can be predicted with over 80% accuracy in most cases. • For a team with limited resources, these methods can go a long way to reduce the burden of manual data collection. • Several automated methods exist for obtaining this data in real-time: • The Team 2834 Scouting Database • The FRC Tracker App • The FIRST OPR App. • FRC Twitter Feed

16. OPR and CCWM • What is OPR? (Offensive Power Rating) • OPR is a computed number for the offensive contribution of a given team. The higher your OPR number, the more points your robot is probably scoring. • The math assumes that your robot’s potential is the same in every matchat an event. • The OPR value range is game specific, depending on the points available in the game. • OPR will generally not indicate defensive specialists well. • CCWM (Calculate Contribution to Winning Margin) does a better job at highlighting good defense teams. • The Twitter Feed now provides separate results per match for Auton, Teleop and Climbing, allowing individualized OPRs for each section of the game to be computed for each team. • Never forget that these numbers are leading indicators, and should never be trusted as being an absolute measure of team performance.

17. “It’s not all about the Robots” • Most scouting is focused on the capabilities of the machines. • Robots are designed by people, built by people, maintained by people, and operated by people. The human factor in all of this is huge. • If you do not know the people on the other teams, your ability to do comprehensive competitive assessment will be limited. • Get out and meet the other coaches and drivers. Every team is unique. Once you understand how they operate, you can better determine the best ways to play with and against them.

18. Q & A • We are happy to answer any and all questions on this or other FRC topics. Good Luck to all teams at the 2013 Michigan State Championship! Thanks to teams 1114 and 2834 for some of the material in this presentation .