Flight Test Principles Applied to First Robotics: Simple Practices to Help Your Team Succeed

Flight Test Principles Applied to First Robotics: Simple Practices to Help Your Team Succeed Steven R. Jacobson NASA Dryden Flight Research Center Flight Controls and Dynamics Branch steve.jacobson@nasa.gov

Professional Background BS Aeronautical Engineering Embry-Riddle Aeronautical University 1992 MS Aeronautics and Astronautics Purdue University 2000 Daedalus Research, Inc. 1993 – 1997 Design and flight test of small UAVs NASA Dryden Flight Research Center 1997 – present. Development and flight test of control systems for vehicles ranging from 12’ UAVs to X-37 Steve Jacobson’s Background Autonomous Formation Flight X-37 Approach and Landing Test Vehicle F-15 Inner Loop Thrust Vectoring Networked UAV Teams (Aerial robots) Advanced Aeroelastic Wing

Similarities between FIRST LEGO League (FLL) and First Robotics (FR) Short time span Student Mentor involvement Coaching/Mentoring strategies very similar Kids have energy that needs focused in the right direction The more mentors, the merrier Differences between FLL and HS FLL is intended to let the students do all of the work First HS robotics is students and mentors working side by side. Similarities and Differences Between Coaching FLL and High School Robotics While most of my experience is with Mentoring FLL, The principles in this presentation apply to both FIRST LEGO League and FIRST High School robotics

FIRST Background Mentor for Team 585, Tehachapi High School 2002, 2003, 2005 First LEGO League 2002 City Sights Golden Hills School Robotic Eagles 5th and 6th grade team 2nd place at Coppertop regional Qualifier 6th place at LEGOLand 2003 Mission Mars Valley Oaks Charter School 4th, 5th and 6th grade team 1st place at LA qualifier 4th place at LEGOLand 2004 No Limits Valley Oaks Charter School 4th, 5th and 6th grade team 1st place at Newhall qualifier 2nd place at LEGOLand Steve Jacobson’s Background

Motivation for This Presentation Each year • My teams have been mostly rookies • Most of my FLL teams have consisted of 4th and 5th grade students • I start from scratch at the beginning of the season • I run my teams with the philosophy • The kids come up with the ideas • The kids design, build and program • Let the kids make the mistakes and learn the lessons that are obvious to the coaches • Our robots are not very sophisticated • My teams have performed well in competition against older teams. • I am amazed at what these kids can accomplish in FIRST. • There are accusations of coaches “doing all the work” • Each year is ask myself “What is different between my teams and other teams that makes us successful?”

Presentation Abstract Many robotics teams are coached by people that have a passion for the sport but lack a technical background. Many coaches and mentors have a technical background, but may not have expertise in conducting systems integration, test and operations. Flight testing aircraft and preparing a robot for competition are very similar activities. Both require complex planning, testing, and tying up of loose ends. The flight test community has developed processes for accomplishing flight test and these processes can be applied to FIRST robotics. These include the systems engineering process, configuration management, risk/hazard analysis, system integration, system testing, and operations. When applied they can make an ordinary team into a great team. This paper is intended to help team leaders with technical and non-technical backgrounds prepare there teams at each step in the season. It maps the technical processes applied by the flight test community into simple concepts that can be understood by students and mentors alike. Most of the examples come from my FLL robotics experience but can be applied to high school robotics as well.

Johnson Space Center Training • JSC is NASA’s Lead center for manned space flight • New project managers at JSC are assigned to mentor a FIRST High School Robotics team • Why? • Six week systems engineering cycle • Complex problem with infinite solutions • Limited budget • Untrained and unskilled workforce • If a new project manager can pull this job off with these constraints, they can see what it takes to get any job done at NASA • NASA Dryden analog: Working with first and applying simple engineering principles in a short time frame has helped me become a better engineer.

Phases of a Project • Project Planning • Design/Building • Testing • Operations/Execution Systems Engineering Vee

Phases of a Project • Project Planning • Design/Building • Testing • Operations/Execution

NASA Flight Test leadership Team leadership and management skills are the most important skills for the leader Intimate knowledge of the technical details is required, but not the most important. Empower the technical leaders to get the job done FIRST team Leaders The RCA winners are the teams that have strong student leadership. Team members that focus mostly on planning, delegating and making sure that the details get done. Require your parents to get involved FIRST Mentors (Technical leaders) Understand the technology FLL: LEGO structures and Programming FR: C, Fabrication, Pneumatics, etc… Communicate this knowledge to the students Know the game Know the rules Assign a mentor to rule checking Project Planning: Team Leadership

Project Planning: Trained Workforce FIRST Trained Workforce: Train the students in the technology • FR: Get your team members to trained before the season starts (C programming, CAD, Pneumatics, drive trains etc.) • Use past years robots or fall to get the students comfortable with the technology • Find a school in your region that offers FIRST Robotics training in the fall • Start a robotics course at your high school • Teach your students how to use sensors • Go over the game play and rules with your team so everyone understands them. • Go for the awards not based on robot performance

Project Planning: Trained Workforce FIRST Trained Workforce: Train the students in the technology • FIRST LEGO League training: Go over the technology with the students • Use the links on the FIRST web site http://www.usfirst.org/jrobtcs/flg_tm.htm • Team building • Mindstorms and Robolab Programming • Teach them how to build robust structures with LEGOs • Burn 10 copies of the programming CD and give it to each team member • Sensors are your best friend in FLL • Battery voltage and motor performance vary from hour to hour • Sensors increase repeatability of your robot • Rotation sensor is the most useful sensor in the kit. • Rotation sensor for turns: Put the rotation sensor on the side that is on the outside turn most of the time

Project Planning: Requirements development “Begin with the end in mind” Stephen Covey

Tools for developing requirements

Requirement example for FLL Missions

Requirements example: results • Integrated forklift • Attachable rakes hooks and other mechanisms • Integrated rotation sensor • Removable light sensor • Robot required only removal of mechanisms during competition. • Mechanisms were designed to used for multiple missions The Requirements design took three practices. All requirements came From the students and before we build the robot we knew exactly what needed to go on it.

Project Planning: Risk Analysis Risk Level Payoff

Design/building phase • Subsystem Requirements Development • Design • Trade studies/prototyping • Configuration Management

Design/Building: Subsystem Requirements • Continue on with the requirements development as in the first phase for the subsystem • Continually look at the interfaces for requirements changes • Have regular “standup” meetings to manage the configuration • Trade studies: • Allow competing designs and let the team members decide which is the best design • Allow the team members to make mistakes. That is how they learn.

Design/Building: Configuration Management • Make it well understood at the beginning of the project that if you have any changes to your part of the design that you inform the rest of the team. • Example: A change in a mechanism could effect the robot base, sensor requirements or a software change • FIRST Robotics: • Hold a regular Configure Control Board (CCB): Discuss any proposed configuration changes (Major or minor) at each meeting or each time a change is desired • All groups (stakeholders should be represented) • FIRST LEGO League: • Instruct team members that they should communicate changes to coaches and other members • Use a configuration control box where you keep all of the mechanisms (Prevent unintentional disassembly) • Take digital photographs of the robot and mechanisms to document the construction • Backup programs before you make a change and after every practice

Phases of a Project • Project Planning • Design/Building • Testing • Operations/Execution Flight research separates “the real from the imagined,” and makes known the “overlooked and unexpected.” – Dr. Hugh L. Dryden

Testing, Testing, Testing • For every hour of flight test there are hundreds or thousands of ground tests • Testing is where you learn the most about your strengths and weaknesses • With enough time you can make modifications to improve your robot • Testing increases the confidence of your team in themselves and their robot • Plan to start testing with 1-2 weeks to spare • Go to a scrimmage match Testing is probably one of the most important steps in development. It is usually overlooked and never given enough time.

Testing: Guidelines • Write a test plan • Perform end-to-end testing of your robot • Robustness testing: Intentionally try to break the robot to see if it will withstand the rigors of competition • Perform testing In the order you intend to operate it • First Robotics: Test the autonomous mode followed by the manual mode • FIRST LEGO League: Test in the order you intend to operate. • Statistical Process Control • Run a mission 10-15 times without changing anything • Take notes on the successes and failures • Determine the failure modes based on observations • Modify the robot and repeat the tests • Regression testing: If you change one thing, however minor, retest everything all over

Operations and Execution: Flight Cards • Flight cards include • Checklists for setup before competition • Steps in which to execute the missions • Contingency plans • Flight cards should be used • To prepare for a competition • During practice at the competition • Only as a reference in case you “Freeze”

Operations and Execution: General Guidelines • Practice, Practice, Practice • The flight test analogy is simulation • Resist the urge to change spontaneously • “Brief what you fly and fly what you brief”

Conclusion • Many analogies between getting an airplane ready to fly and a robot ready to play. • Not every principle may be applicable to your team, but simple discipline in these areas will help your team. • Most important areas (If you had to only do a few things) • Make sure you have mentors that can teach the technology • Let the kids “stub their toes” • Spend adequate time on requirements development early on • Configuration management • Test, Test, Test • Use “Flight cards” • Practice, Practice, Practice

Flight Test Principles Applied to First Robotics: Simple Practices to Help Your Team Succeed