1 / 52

Organization of Report

NASA CUSTOMER SURVEY SUMMARY REPORT Prepared by Science Applications International Corporation (SAIC) Arlington, Virginia August 1998. Organization of Report. Population Approach Introductory Questions ASTT Program Three Pillars and Ten Goals Level of Satisfaction with NASA’s ASTT Program

aolani
Download Presentation

Organization of Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NASA CUSTOMER SURVEYSUMMARY REPORTPrepared by Science Applications International Corporation (SAIC)Arlington, VirginiaAugust 1998

  2. Organization of Report Population Approach Introductory Questions ASTT Program Three Pillars and Ten Goals Level of Satisfaction with NASA’s ASTT Program • Facilities and Services • Technology Transfer • Program Planning Level of Satisfaction with NASA Resource Management Effectiveness in Meeting National Needs Closing Questions – Executive Level Only

  3. Survey Population Data Executive Mid-Level Manager Survey Population 101 72 Customer Groups Direct Industry 63 65 Indirect Industry 38 7 Benchmark Companies 50 59 Aviation Advisory Committee Members 11 0 Universities 18 3 Aviation Associations 8 3

  4. Industry Groups DIRECT INDUSTRY • Large Civil Transport Manufacturers • High Performance Aircraft Manufacturers • Aircraft Engine Manufacturers • Rotorcraft Manufacturers • GA/Commercial/Sport/Business/Other Aircraft Manufacturers • Launch Vehicle Manufacturers • Launch Vehicle Engine/Propellant Manufacturers • Aircraft Instrument/Avionics Manufacturers • Specialized Aerospace Services • Department of Defense Entities • Other Federal Agencies • INDIRECT INDUSTRY • Aviation Metal/Metal Component Manufacturers • Non-Metallic Aviation Materials Manufacturers • Air Carriers • Satellite Manufacturers and Other Launch Service Customers • Universities (Major ASTT Grantees) • Aviation Associations • Aerospace Analysts and Advocacy Groups

  5. Survey Approach Quantitative Questions Statistical Analysis (Mean, Standard Deviation) Ratings from 1 to 10 • 1 to 4 - low • 5 to 7 - medium • 8 to 10 - high Qualitative Questions (open-ended) Followed several Quantitative Questions YES/NO Questions Sorting Items into Categories Comparisons with 92 and 95 Surveys

  6. Survey Approach Summary Executive Mid-Level Manager Method of Interview Personal Visit 86 14 Telephone 15 58 Average Interview Duration (Minutes) 56 41 Level of Understanding (1 to 10 scale) 7.7 7.9 (Interviewer Assessment) Level of Interest (1 to 10 scale) 8.2 7.2 (Interviewer Assessment) ASTT Mission Area to Which Answers Apply Aeronautics 63 47 Space 14 15 Aeronautics and Space 24 10

  7. Overall Ratings by Customer Group Category Mean Rank Aeronautics Advisory Committee 7.02 1 Universities 6.85 2 Government 6.69 3 Associations/Publications 6.58 4 Mid-Level Managers 6.55 5 Benchmark Companies 6.53 6 Direct Industry 6.50 7 Executives 6.41 8 Indirect Industry 6.36 9 Overall 6.47

  8. 8.2 9.7 1. What is your overall level of interest in the ASTT Program? • 96 percent of the total respondents said they had a high or medium level of interest in the ASTT Program. This level was up from the 1995 survey when the total was 92 percent. • 79 percent of the total said they had a high level of interest. This compares to 75 percent in the 1995 survey. • NASA’s Aviation Advisory Committee (AAC) had the highest level of interest at 100 percent in the high category. • University interest dropped from 100 percent in 1995 to 90 percent in 1998. • The group that had the lowest level of interest in ASTT was Indirect Industry. • This question had the highest mean value of all quantitative questions in the 1998 survey. Mean 8.9 7.6 8.2

  9. 1b. Select all the NASA mission areas that greatly interest you. • Critical Technology (Base R&T) was selected as the area of greatest interest by 73 percent of the respondents. • Other areas of high interest were Advanced Subsonic Transport (57percent) and High-Speed Research (51 percent). • High Performance Aircraft (45 percent), High Performance Computing (45 percent), Space Transportation Technology (41 percent) and General Aviation (41 percent) all had moderate interest. • Hypersonics (33 percent ) and Rotorcraft (33 percent) had the lowest level of interest in the specifically named categories. a. Advanced Subsonic Transport b. High Speed Research c. High Performance Aircraft d. Hypersonics e. High Performance Computing f. Critical Technology (R&T Base) g. Facilities and Services h. Space Transportation Technology i. General Aviation j. Rotorcraft k. Other

  10. 6.4 8.8 6.8 4.9 7.0 2. How well do you understand the mission of the ASTT Program? • The level of understanding of the ASTT Program for all respondents dropped from 55 percent in 1995 to 44 percent in 1998 in the “well” category. The mean value also dropped from 7.0 in 1995 to 6.4 in 1998. • AAC Members had the highest level of understanding with a mean value of 8.8. • Benchmark Customers had the second highest level of understanding with a mean value of 6.8. • The group with the lowest level of understanding was Association/Publication with a mean value of 4.9. • This question’s overall mean score of 6.4 ranked 14th of 30 quantitative questions. Mean

  11. 5.8 7.0 6.1 4.5 6.1 3. How well are the current mission areas of NASA’s ASTT Program aligned to your organization’s needs? • Government rated NASA’s ASTT Program mission areas best aligned to their needs with a mean value of 7.0. • The Benchmark Companies and Direct Industry rated the mission areas next well aligned with a mean value of 6.1. • Total scores were close to 1995’s scores, only dropping one percent from 26 percent to 25 percent in well aligned, but the poorly aligned category grew from 23 percent to 27 percent. • Associations/Publications rated the goals least aligned with none scoring in the “well” category. • This question’s overall mean score of 5.8 ranked 27th of 30 quantitative questions. Mean

  12. a. Advanced Subsonic Transport b. High Speed Research c. High Performance Aircraft d. Hypersonics e. High Performance Computing f. Critical Technology (R&T Base) g. Facilities and Services h. Space Transportation Technology i. General Aviation j. Rotorcraft k. Other 3b. What current mission areas of NASA’s ASTT Program are most important to your organization? • Organizational interest closely paralleled individual interest (Question 1b) but at a level approximately 8 to 10 percent less. • Critical Technology (Base R&T) showed the most interest with 61 percent of the respondents. • Another area of significant interest was Advanced Subsonic Transport (49 percent). • High Speed Research (39 percent), High Performance Computing (38 percent), High Performance Aircraft (34 percent), Space Transportation Technology (34 percent), Facilities and Services (32 percent), General Aviation (30 percent), and Rotorcraft (30 percent) all had moderate interest. • Hypersonics (23 percent ) had the lowest level of interest in the specifically named categories.

  13. Mean 6.0 6.7 6.1 5.6 6.3 4. How well is NASA’s ASTT program accomplishing its mission? • 63 percent of the total respondent group rated their general satisfaction in the middle range and most individual groups followed this pattern. • The mean value for the 98 Survey (6.0) dropped by 0.3 as compared to the 95 Survey (6.3). • Government Customers gave the highest rating with a mean value of 6.7. • Mean values for Benchmark Customers was 6.1 which is slightly higher than the total population. • Lower scores for accomplishing mission are consistent with lower scores for understanding of ASTT mission (Question 2). • This question’s overall mean score of 6.0 ranked 24th of 30 quantitative questions.

  14. a. Advanced Subsonic Transport b. High Speed Research c. High Performance Aircraft d. Hypersonics e. High Performance Computing f. Critical Technology (R&T Base) g. Facilities and Services h. Space Transportation Technology i. General Aviation j. Rotorcraft k. Other 4b. What current mission areas is NASA’s ASTT Program accomplishing the best? • Respondents typically only selected one or two items, unlike the previous selection questions where they picked several choices. • Four mission areas were the most common selection; High Speed Research (31 percent), Advanced Subsonic Transport (29 percent), Critical Technology (R&T Base) (22 percent), and Space Transportation Technology (21 percent). • High Performance Computing (12 percent) received a moderate rating. • General Aviation (8 percent); High Performance Aircraft, Facilities and Services and Rotorcraft (all at 7 percent) and Hypersonics (5 percent) are accomplished least well according to the respondents.

  15. Mean 6.1 8.7 6.8 4.8 4.4 5. How familiar were you with the Pillars and Goals before this survey? • Over two-thirds of the respondents were somewhat to very familiar with the Pillars and Goals before the survey. • Those most familiar were the AAC Members (91 percent) followed by Benchmark Companies at 49 percent. • The groups least familiar were Associations/ Publications with 62 percent being unfamiliar and Indirect Industry with 51 percent. • This question’s overall mean score of 6.1 ranked 21st of 30 quantitative questions. • There was no comparable question on the 95 or 92 surveys.

  16. 5b. How did you learn about the Pillars and Goals? • Most respondents learned about the Pillars and Goals through interaction with NASA, either through working with NASA or from Committee work. These are indicated in the “Other” column. • The next most frequent way of learning about the Pillars and Goals was from the NASA Administrator’s Speech. • The least frequent way of learning about the Pillars and Goals was from the Internet. a. NASA Administrator’s Speech b. Pamphlet Mailed to You c. Letter Introducing the Survey d. From the Internet e. Other

  17. Mean 6.6 7.1 6.9 6.4 6.1 6. How well are the Goals aligned to your organization’s needs? • Mean values of ratings for this question were closely bunched ranging from a high of 7.1 for AAC Members to a low of 6.1 for Associations/ Publications • 42 percent of respondents ranked the Goals as well-aligned to their organization. • This question’s overall mean score of 6.6 ranked 10th of 30 quantitative questions.

  18. 6b. Which goals are best aligned to your organization’s needs? Many respondents identified specific Pillars or Goals; some identified subsets of Pillars or Goals.

  19. 6c. Which important Goal, from your organization’s perspective, is missing? Respondents provided various ideas regarding goals that should be/could be added. • Work with DoD where commonality applies -- [need] Goal to share and integrate cost-effective R&D with DoD • Greatly concerned about future of subsonic transport R&T. Basic is being neglected. • Development of future rotorcraft concepts..., working with FAA to develop infrastructure for civil transportation. • No materials research for airframe technology for general aviation. • Aeronautics and Earth Science not connected -- NASA ignores internal customers. • High performance military aircraft. • Next generation avionics. • FANS, satellite communications technology, composite wing, wet wing. • Reduce cost to lower Earth orbit applies to only part of mission -- need Goals that cover launch to orbit mission, transfer orbits and on issues with space probe. • Future engineering education should be in the Goals; important to next generation. • Economic viability of the contemplated technologies. • Human factors research. • Product quality. • Fundamental flow physics. • Infrastructure development. • There isn’t anything missing.

  20. Mean 7.9 8.1 8.1 7.3 7.1 7. How well are the Goals aligned to national needs? • Total respondents overwhelmingly ranked NASA’s Goals as being very well aligned to national needs scoring 66 percent in the “well” category. Total “well” and “moderately well” rating was 97 percent. • Benchmark Customers and Universities ranked this question very high with 70 percent in the “well” category. • The lowest “aligned well” rating was Government at 54 percent, which also had the second highest “poorly aligned” rating of 15 percent. • Mean values closely followed the percentile ratings. • This question’s overall mean score of 7.9 ranked 2nd of 30 quantitative questions.

  21. 7b. Which goals are best aligned to national needs? Most respondents identified specific Pillars or Goals; some respondents identified subsets of Pillars or Goals. These results closely parallel the results of Question 6b regarding the alignment to organizational needs. 7c. What important Goal, from a national perspective, is missing? Many of the suggestions for National Goals were similar to those provided in Question 6c in response to missing Goals from the Customer’s organizational needs.

  22. 6.4 6.9 6.9 6.3 6.3 8. Overall, how well are NASA’s current programs aligned to the Goals? Poorly Moderately Well Mean • Overall, the respondents thought that NASA’s current programs were moderately well aligned to the Goals. However, within the Customer Groups, the rankings for this question were nearly the reverse of those observed in most other questions. • The highest rankings were given by the Associations/Publications and the Universities with a mean value of 6.9. • The lowest rankings within the customer groups were given by the Benchmark Customers and the Direct Industry Customers. They each had a mean value of 6.3. • This question’s overall mean score of 6.4 ranked near the middle at 16th of 30 quantitative questions. 98 All 15 58 28 98 Assoc/Pub 10 50 40 98 Universities 11 42 47 98 Direct 15 61 24 Industry 98 Benchmark 13 64 23 0% 20% 40% 60% 80% 100%

  23. 8b. Which current ASTT programs are best aligned to the goals? Most respondents provided answers that related to specific Pillars, Goals, or ASTT Programs.

  24. 6.0 7.0 6.5 5.4 4.9 6.8 9. What is your level of satisfaction with NASA’s ASTT Facilities and Services? Mean • The level of satisfaction with ASTT facilities and services dropped significantly as compared to the 1995 survey. The mean value in 1995 was 6.8, in this survey it dropped to 6.0 • Government Customers showed the greatest satisfaction with facilities and services with a mean value of 7.0; the AAC had the second highest mean value of 6.5. • Associations and Publications had the lowest mean value of 4.9; Indirect Industry had the next lowest mean value at 5.4. • This question’s overall mean score of 6.0 ranked moderately low at 22nd of 30 quantitative questions.

  25. a. Charge for using the facility or service b. Level of technical expertise c. State-of-the-art facilities d. Timelines of the results e. Convenience of the facility or service f. Ownership of the intellectual property g. Familiarity and/or trust h. Availability of the facility or service i. Location of the facility or service j. Quality of the data k. Protection of the data l. Computational quality m. Ease of technology transfer n. Ownership by the U. S. Government 9b. Which NASA facility and service issues are most important to your organization? Facility or Service Issue Percent 1. Quality of the Data 75 2. Level of Technical Expertise 68 3. Timeliness of the Results 63 High 4. Availability of Facility or Service 60 5. State-of-the-Art Facilities 58 6. Protection of the Data 42 7. Ease of Technology Transfer 39 8. Familiarity and/or Trust 38 Moderate 9. Charge for Using Facility or Service 36 10. Ownership by the U.S. Government 35 11. Computational Quality 32 12. Convenience of Facility or Service 19 13. Ownership by U. S. Government 10 Low 14. Location of the Facility or Service 6

  26. 10. Which methods of technology transfer work best for your organization? 92 Survey Mean Rank 4 7.01 1 7.49 6 6.81 7.07 2 6.85 5 7.03 3 5.98 7 These results differ somewhat from results of the 1992 survey. In that survey, customers were asked to give the various technology transfer methods a ranking from 1 to 10 with 1 being unfavorable and 10 being favorable. These results are shown in the right-hand columns.

  27. 6.0 7.3 6.7 5.9 5.8 5.6 11. How would you rate NASA’s program planning? Mean • Program planning was rated higher in this survey (mean = 6.0) than in the 95 survey (mean = 5.6). • Program planning was rated much higher in this survey than in the 92 survey when 53 percent of the respondents indicated that NASA’s program planning was inadequate. • This question was rated higher by Associations and Publications (mean = 7.3) and Universities (mean = 6.7). It was rated lower by Benchmark Customers (mean = 5.9) and lowest by Government Customers (5.8) • 58 percent of respondents answered “Yes” to the question, “Does NASA adequately involve your organization in its program planning for ASTT?” • 62 percent of respondents answered “Yes” to the question, “Does NASA plan research programs to provide results in a form, structure and format useful to you?” • This question’s overall mean score of 6.0 ranked quite low at 23nd of 30 quantitative questions.

  28. 6.3 7.4 7.3 6.4 6.3 6.2 12. How would you rate NASA’s Outreach and External Communication? Mean • Outreach and External Communication was rated slightly higher in this survey (mean = 6.3) than in the 95 survey (mean = 6.2). • The AAC (mean = 7.4) and University groups (mean = 7.3) rated this question high. Benchmark Customers (mean = 6.4) and Direct Industry Customers (mean = 6.3) rated the question lower than the overall mean of 6.3. • 68 percent of respondents answered “Yes” to the question, “Does NASA ask for your input on plans and issues important to your organization?” • 67 percent of respondents answered “Yes” to the question, “Does NASA use your input?” • 65 percent of respondents answered “Yes” to the question, “Does NASA provide feedback on plans/issues important to your organization?” • This question’s overall mean score of 6.0 ranked near the middle at 17th of 30 quantitative questions.

  29. 6.6 7.0 6.3 6.4 13. What is your overall level of satisfaction with NASA? Mean • The Customer’s Overall Level of Satisfaction with NASA was rated slightly higher in this survey (mean = 6.6) than in the 95 survey (mean = 6.4). • There was a significant difference (p<0.05) in the mean values of the Executives (mean = 6.3) and the Mid-Level Managers (mean = 7.0) on this question. • This question’s overall mean score of 6.6 ranked near the top one-third of the questions at 11th of 30 quantitative questions. 13b. In what areas are you most satisfied with NASA? • Increasing attentiveness to the needs of U. S. industry; improved communication ... in executing contracts. • Good job getting input from industry; good facilities and people. • Quality work, quality reports, results and availability. • Headquarters' interest in having a positive national impact through technology development. 13c. In what areas are you least satisfied with NASA? • Inability to do meaningful multi-year planning. • Inability to change with changing needs of industry. • Technology transfer - cost to industry; complex paperwork; “nightmare” multi-page licenses. • NASA competition with industry.

  30. 14. What is your overall level of satisfaction with NASA’s ASTT products? Mean • The Customer’s Overall Level of Satisfaction with NASA’s ASTT Products was rated moderately high. Overall, 85 percent of the respondents rated their satisfaction as “medium” or “high.” • University Customers (mean = 7.1) and AAC Customers (mean = 6.9) had a higher rating than the overall population. Direct Industry Customers (mean = 6.3) had a lower rating than the overall population. • This question’s overall mean score of 6.4 ranked in the middle at 15th of 30 quantitative questions. 6.4 7.1 6.9 6.3 • 14b. With what products were you most satisfied? • Responses Responses • High Speed Research/Travel & High Speed Civil Transport 23 Technical Reports/Symposia 13 Computational Fluid Dynamics/Computational Analysis 17 Use of NASA Facilities 12 • Research and Technology Base 15 Design and Analysis Tools 10 • Advanced Subsonic Transport/Subsonic Research 14 Reduce Noise 10 • Experimental Aircraft (X-33, X-34, X-36, Hyper-X, Future-X) 13 Rotorcraft/Tiltrotor 9 • 14c. What products were you least satisfied with? • Those that don’t have a cooperative character... • Full-cost [funding] resulting in budget cuts and diminished expectations. • Technology transfer between larger corporations and NASA and government (as on HSR program).

  31. Mean 6.1 15. How does your organization’s overall level of satisfaction with NASA compare with its satisfaction 3 to 6 years ago (before the last surveys)? • The Customer’s felt that their satisfaction with NASA had improved over the last 3 to 6 years. They rated this question slightly higher in the current survey (mean = 6.5) than in the 95 survey (mean = 6.3). • 85 percent of respondents rated this question as “medium” or “high.” Benchmark Customers (mean = 6.6) provided slightly higher ratings, and University Customers (mean = 6.1) provided slightly lower answers than the overall population. • This question’s overall mean score of 6.5 ranked slightly above the middle at 13th of 30 quantitative questions. 6.5 6.6 6.3 • 15b. In what area(s) has your level of satisfaction increased? • Alignment of research to the Pillars; six years ago there was inefficiency and lack of focus. • Shift from micro-management to a partnership relationship. • Involvement and visibility of NASA plans and goals. • Just in being part of their activities; we’re in partnership with them; our relationship has grown by leaps and bounds. • 15c. In what area(s) has your level of satisfaction decreased? • Systems integration emphasis, computational resources, and test facility readiness. • Availability of wind tunnel test facilities; reduced emphasis on R&T base for aeronautics. • I think the most important is the condition of NASA’s deteriorating facility support. • Difficulty in dealing with NASA as a customer.

  32. 16. NASA provides key laboratory, computing, and test facilities. Mean • The respondents strongly felt that NASA provides key facilities. • 93 percent of respondents rated this question as “partially agree” or “agree” and 62 percent rated it “agree.” Government Customers (mean = 8.2) and University Customers (mean = 8.1) provided high ratings, and Benchmark Customers (mean = 7.5) provided slightly lower answers than the overall population. • This question’s overall mean score of 7.6 ranked near the top at 3rd of 30 quantitative questions. 7.6 8.2 8.1 7.5 • 16b. What NASA facilities do you view as being essential to the conduct of leading edge R&D? • The overwhelming answer to this question was wind tunnels, followed by computational resources. The comments below characterize these and other comments. • Modern wind tunnel test facilities and world class computational resources. • Space test facilities at Stennis Center. • Long duration test and space exposure tests. • Airborne test beds, simulation and flight facilities.

  33. 17. NASA downsizing has affected its ability to support its customers. • The respondents were equally divided in response to this question. • 65 percent of respondents rated this question as “partially agree” or “agree” and 33 percent rated it “agree.” AAC Customers (mean = 7.0) and University Customers (mean = 6.4) provided high ratings, while Benchmark Customers (mean = 5.9) were more inclined to disagree than the overall population. • This question’s overall mean score of 5.8 ranked near the bottom at 26th of 30 quantitative questions indicating significant disagreement that downsizing has affected NASA’s ability to support customers. Mean 5.8 7.0 6.4 5.9 • 17b. In what ways, if any, has NASA downsizing affected your organization? • Some respondents felt that NASA downsizing had a negative effect: • Foundational research is not getting done. • Primarily, loss of key NASA expertise and missing expertise is not being replaced. • We are starting to reduce our alliance on technologies developed by NASA. • Other respondents felt that NASA downsizing had positive effects: • Very positive effect - less micro-management - more of a partnership. • Haven’t downsized enough. • Helped since NASA is now less competitive with industry. • Its been positive; it’s made them take a hard look at programs and ... it helped in that they are more focused.

  34. 18. NASA is a key contributor to the education of scientists and engineers. • The respondents strongly felt that NASA is a key contributor to the education of scientists and engineers. The respondents of the current survey (mean = 7.0) gave higher ratings to this question than did the respondents of the 95 survey (mean = 6.7). • 84 percent of respondents rated this question as “partially agree” or “agree” and 52 percent rated it “agree.” University Customers (mean = 8.0) provided high ratings, and Benchmark Customers (mean = 6.9) provided slightly lower ratings than the overall population. • This question’s overall mean score of 7.0 ranked near the top at 4th of 30 quantitative questions Mean 7.0 8.0 6.9 6.7 • 18b. In what ways can NASA improve the education and training of our next generation of scientists and engineers? • The main themes were more funding, cooperative programs, and more internships. Also NASA involvement in university, secondary and elementary schools was identified as being important. • Strengthen university relationships that include industry. • Put more money into foundational research in universities best equipped to do that type of research. • Re-instituting university programs programs where NASA provides seed money to university engineering programs. • Be more involved in [recommending] curricula of engineering schools. Have more open house events to stimulate younger students. • Public relations programs aimed at younger folks to kindle their interest early on. • Target younger children, even in elementary grades.

  35. 19. NASA cooperates with other organizations to make cost-effective use of limited R&D resources, skills and facilities. • The respondents were mixed in their rating of NASA’s cooperation with other organizations. The respondents of the current survey (mean = 6.3) gave slightly higher ratings to this question than did the respondents of the 95 survey (mean = 6.1). • 78 percent of respondents rated this question as “partially agree” or “agree” and 35 percent rated it “agree.” AAC Customers (mean = 7.0) provided higher ratings than the overall population and Benchmark Customers (mean = 6.3) provided ratings equal to the overall population. • This question’s overall mean score of 6.3 ranked in the lower half at 19th of 30 quantitative questions Mean 6.3 7.0 6.3 6.1 • 19b. In what ways can NASA improve cooperation with other organizations? • More communication and coordination was the primary theme of the responses. • Talk to and coordinate with DoD -- and don’t just talk -- [perform] some deeds. • Expand levels of cooperation with other government agencies as well as corporate world; develop and participate in [joint] technical programs. • By helping its personnel to understand the realities and mechanisms of the commercial marketplace. • Improve communication; become more aware of industry programs to create cooperative programs. • [Be] more pro-active in searching for organizations to cooperate with.

  36. 20. The ASTT program effectively balances fundamental (R&T base) research with its focused-program research. • The respondents were mixed in their rating of NASA’s balance of fundamental versus focused-program research. Government Customers (mean = 7.0) provided much higher ratings than the overall population. • This question’s overall mean score of 5.9 ranked in the lower half at 25th of 30 quantitative questions. • Respondents of the current survey called for significantly more fundamental research than did the respondents of the 92 survey. Mean 5.9 20b. Should NASA pursue more fundamental research or focused-program research, and why? Some thought balance was good; others presented arguments for both fundamental and focused research. 7.0 • Fundamental research; if fundamental research is done, the focused-program research will be done better. • [NASA] should be looking at 5 to 10 year technology; universities should be looking at 10 to 15 year technology; industry should look at 2 to 5 year technology. • More fundamental research; industry better understands market need; leave focused-program to industry. • More focused-program [research] because it is a more effective use of resources. • Focused-program research to become more efficient -- pull research rather than push research. • More focused-program research, because it offers more incentive for industry involvement that would lead to commercial products.

  37. 21. The ASTT program effectively balances analytical, computational, ground-based experimental and flight research. • The respondents generally approved of NASA’s mix of research methods. The overall mean was quite high at 6.8. • There was some variation within the customer groups. AAC Customers (mean = 7.6) provided higher ratings than the overall population; Benchmark Customers (mean = 6.9) provided slightly greater than the ratings of the overall population; Associations and Publications gave a low rating. • This question’s overall mean score of 6.8 ranked quite high (in the upper quarter) at 6th of 30 quantitative questions Mean 6.8 7.6 6.9 5.9 • 21b. Should NASA change its mix of analytical, computational, ground-based experimental and flight research, and why? • Most respondents felt the balance was good now, but the mix needs to be constantly scrutinized. There was some call for more flight testing. • Yes, not enough flight research for aeronautics; maintain ground-based experimental facilities. • More emphasis on flight research [ will produce] more rapid progress. • In general, they are pretty well balanced, but caution is necessary on flight research to make sure the costs/benefits are scrutinized. • The mix is fine and doesn’t need to be changed.

  38. 22. The ASTT engineers and scientists play a key role in the development of innovative or enhanced technical solutions. • The respondents generally felt that NASA’s engineers and scientists play a key role in finding technical solutions. The overall mean was quite high at 7.0. • There was only slight variation within the customer groups as the mean values ranged from 6.9 for Benchmark Customers to 7.3 for University and Government Customers. • This question’s overall mean score of 7.0 ranked quite high (in the upper quarter) at 5th of 30 quantitative questions Mean 7.0 7.3 7.3 6.9 • 22b. In what areas have NASA engineers & scientists played a key role? • Human factors, structures, propulsion and computational fluid dynamics -- Space travel is the biggest • Atmospheric physics; earth physical science -- Development of high performance rocket propulsion • Lifting-body ceramic and metallic thermal protection systems; supercritical wing; coke-bottle design of aircraft • Composite technology; computer fluid analytical (CFA) tools; and noise research -- Atmospheric modeling • 22c. In what area have they not played a key role? • They’ve tried to play a role in system integration, but unsuccessfully -- Solution of critical operational problems • General aviation and airline safety -- Development of cost effective launch systems • Have not achieved leadership in information technology, for example, redundant/fault-tolerant information systems • Reducing cost of bringing advanced technology to market and reducing the time to certification of key technologies

  39. 23. How effective is NASA’s ASTT program in performing fundamental research? Mean • As compared with the 95 survey(which contained two question relating to performance of fundamental research) the 98 survey showed much lower mean values. The 98 survey had a mean of 6.3. The 98 survey had a mean score of 7.1 for effectiveness and a 7.5 score for contributions to national needs. • The customer groups presented varied results in rating the effectiveness NASA’s ASTT Program in performing fundamental research. The mean values ranged from 6.0 for Benchmark Customers to 7.8 for Associations and Publications. • This question’s overall mean score of 6.3 ranked in the lower half at 18th of 30 quantitative questions. 6.3 7.8 6.7 6.0 7.1 7.5

  40. 24. How effective is NASA’s ASTT Program in improving national security? • As compared with the 95 survey the 98 survey scored much lower mean values. The 98 survey had a mean of 5.1. The 98 survey had a mean score of 6.0 for contribution to national needs. • The customer groups presented very low results in rating the effectiveness NASA’s ASTT Program in improving national security. The mean values ranged from 4.5 for Mid-Level Managers to 5.4 for AAC Members. • The statistical data showed there was a significant difference in the mean values of the Executive Customers (mean = 5.5) and the Mid-Level Managers (mean = 4.5). For this question, there is a high likelihood that the two customer groups represent different population groups. • This question’s overall mean score of 5.1 ranked lowest in the entire survey at 30th of 30 quantitative questions. Mean 5.1 5.4 4.8 5.5 4.5 6.0

  41. 25. How effective is NASA’s ASTT Program in developing/ transferring technologies for economic competitiveness? Mean • As compared with the 95 survey the 98 survey scored much lower mean values. The 98 survey had a mean of 5.8 for effectiveness. The 95 survey had a mean score of 6.9 for effectiveness and 6.6 for contribution to national needs. • The customer groups had generally low mean values in rating the effectiveness NASA’s ASTT Program in developing/transferring technologies for economic competitiveness. The mean values ranged from 5.8 for Benchmark Customers to 6.5 for University Customers. • This question’s overall mean score of 5.8 ranked in the lowest quarter of the survey at 28th of 30 quantitative questions. 5.8 6.5 6.4 5.8 6.6 6.9

  42. 26. How effective is NASA’s ASTT Program in maintaining superiority of US aircraft and engines? Mean • As compared with the 95 survey the 98 survey scored slightly higher mean value. The 95 survey had a mean of 6.6 for national goals achievement. The 98 survey had a mean score of 6.7 for effectiveness. • The customer groups had moderately high mean values in rating the effectiveness NASA’s ASTT Program in maintaining the superiority of U. S. aircraft and engines. The mean values ranged from 6.1 for Government Customers to 7.4 for University Customers. • This question’s overall mean score of 6.7 ranked in the highest third of the survey at 9th of 30 quantitative questions. 6.7 7.4 7.1 6.7 6.1 6.6

  43. 27. How effective is NASA’s ASTT Program in developing a superior, affordable, global air transportation system? Mean • As compared with the 95 survey the 98 survey scored a lower mean value. The 95 survey had a mean of 6.2 for national goals achievement. The 98 survey had a mean score of 5.7 for effectiveness. • The customer groups had very low mean values in rating the effectiveness NASA’s ASTT Program in developing a superior, affordable global transportation system. The mean values ranged from 5.6 for Benchmark Customers to 6.7 for AAC Members. • This question’s overall mean score of 5.7 ranked very low in the survey at 29th of 30 quantitative questions. 5.7 6.7 6.5 6.4 5.6 6.2

  44. 28. How effective is NASA’s ASTT Program in developing superior, affordable, space launch systems? • As compared with the 95 survey the 98 survey scored a slightly lower mean value. The 95 survey had a mean of 6.4 for contributions to national needs. The 98 survey had a mean of 6.1 for effectiveness. • The customer groups had very mixed mean values rating the effectiveness in developing superior, affordable space launch systems. The mean values ranged from 5.1 for AAC Members to 6.6 for Mid-Level Managers. The AAC Members gave this question their lowest rating in the 98 survey. • The statistical data showed there was a significant difference in the mean values of the Executive Customers (mean = 5.8) and the Mid-Level Managers (mean = 6.6). For this question, there is a high likelihood that the two customer groups represent different population groups. • This question’s overall mean score of 6.1 ranked low in the survey at 20th of 30 quantitative questions. Mean 6.1 6.2 5.1 6.6 5.8 6.4 • There was one question on the 95 survey regarding contributions to national needs in space. The 95 survey question and the 98 survey question are slightly related in that regard and a comparison is presented herein.

  45. 29. How effective is NASA’s ASTT Program in ensuring long-term environmental compatibility of aerospace systems? Mean • There were two questions on the 95 survey dealing with environmental technology. One related to contribution to national needs and the other related to national goals achievement. • The 98 survey scored between the two 95 survey questions. The 95 survey had a mean of 6.4 for national goals achievement 6.8 for contribution to national needs. The 98 survey had a mean score of 6.7 for effectiveness. • The customer groups had generally high mean values in rating the effectiveness NASA’s ASTT Program in ensuring long-term environmental compatibility of aerospace systems. The mean values ranged from 6.6 for Benchmark Customers to 7.4 for University Customers. • This question’s overall mean score of 6.7 ranked in the upper third of the survey at 8th of 30 quantitative questions. 6.7 7.4 7.1 6.6 6.4 6.8

  46. 30. How effective is NASA’s ASTT Program in making partnerships with government, industry, and universities? Mean • There were two questions on the 95 survey dealing with partnerships. One related to effectiveness and the other related to national goals achievement. • The 98 survey scored between the two 95 survey questions. The 95 survey had a mean of 6.3 for national goals achievement and 7.0 for effectiveness. The 98 survey had a mean score of 6.8 for effectiveness. • The customer groups had generally high mean values in rating the effectiveness NASA’s ASTT Program making partnerships with government, industry and universities. The mean values were tightly bunched and ranged from 6.5 for Indirect Industry Customers Customers to 6.9 for several customer groups. • This question’s overall mean score of 6.8 ranked in the upper quarter of the survey at 7th of 30 quantitative questions. 6.8 6.9 6.9 6.7 6.3 7.0

  47. 31. How effective is the overall ASTT contribution to US R&D needs? • The respondents felt that NASA is an effective contributor to U. S. research and development needs. However, the respondents of the current survey (mean = 6.6) gave lower ratings to this question than did the respondents of the 95 survey (mean = 7.3). • 88 percent of respondents rated this question as “partially agree” or “agree” and 31 percent rated it “agree.” AAC Members (mean = 6.9) provided slightly higher ratings than the overall population. Benchmark Customers (mean = 6.6) provided ratings consistent with the overall population. • This question’s overall mean score of 6.6 ranked near the middle at 12th of 30 quantitative questions Mean 6.6 6.9 6.6 7.3 • 31b. What can ASTT do to increase its overall effectiveness in contributing to US R&D needs? • The main themes were more funding, more fundamental research, and working more closely with customers. • More stable, long term programs; more emphasis on validation. • NASA needs to excel in the use of critical research and development methods with both industry and other branches of the government, especially DoD and FAA. • Maintain a high level of communication in trying to understand industry’s needs. • Perform better market research to validate the need for the research. • Continue looking at business-based metrics; watch U. S. competitive needs.

  48. 32. Has NASA’s planning worsened or improved over the last few years? (Asked to Executive Customers only) • Most respondents (51 percent) felt that NASA’s planning had stayed the same over the last few years. However, more respondents (32 percent) felt that planning had improved as compared to those who felt that it had worsened (18 percent). • AAC Members (mean = 7.3) provided significantly higher ratings than the overall population (mean = 6.4). Benchmark Customers (mean = 6.6) provided ratings slightly higher than the overall population. Indirect Industry Customers (mean = 5.9) rated this question much lower than the overall mean. • This question was not included in the count of 30 quantitative rating questions because it was asked only of Executive Customers. However, the overall mean score of 6.4 ranked near the middle of the 30 quantitative questions Mean 6.4 7.3 6.6 5.9 • 32b. What can NASA do to improve its planning? • Provide a forum that goes beyond workshops and actually sets up programs -- smaller group of key industry participants to boil down ideas into programs; do more feasibility studies with industry before launching a program. • Be more responsive to industry’s needs -- not so much planning as it is attitude. • More planning in cooperation with DoD elements in aeronautics and space. • Better execution of plans. • Match planning to budget; avoid stop - starts in programs due to budget fluctuations. • Maintain commitments once established. • Narrow focus -- be better at fewer things.

  49. 33. Have you participated in a previous NASA Aero Survey? • 21 Executive Level respondents (out of a total of 101) participated in the 1992 survey. • 35 Executive Level respondents participated in the 1995 survey. • 13 Executive Level respondents participated in both prior surveys. • 33b. If you participated in the 1992 and/or the 1995 survey, did the results of the survey in any way change your organization’s relationship with NASA? If so, how? • The responses were mixed concerning changes in relationships with NASA. • Yes, between 1992 and 1995 industry became more involved in planning through workshops. • Yes, helped put more relevancy in NASA which led to Pillars and Goals. • Yes, increased efforts in joint industry/NASA planning. • Created an awareness of the advantages of working with NASA. • No apparent impact -- however, NASA should be praised for attempt. • No, I don’t know if I ever saw results of the survey.

  50. 34. Compared to previous NASA Aero Surveys (1992 and 1995), how would you rate this one? (Asked to Executive Customers only) • Most respondents (95 percent) rated the 98 survey as “medium” or “high.” • Indirect Industry Customers (mean = 7.1) provided slightly higher ratings than the overall population (mean = 6.9). Benchmark Customers (mean = 6.8) provided ratings slightly lower than the overall population. Government Customers (mean = 6.6) also rated this question lower than the overall mean. • This question was not included in the count of 30 quantitative rating questions because it was asked only of Executive Customers. However, the overall mean score of 6.9 ranked in the top quarter of the 30 quantitative questions Mean 6.9 7.1 6.8 6.6 • 34b. How can future surveys be improved? • Offer opportunity to get into specifics in different areas of ASTT. • Have Pillars and Goals ahead of time. • Need more questions on how NASA measures their effectiveness against customer need. • ... A few weeks before the survey, provide a list of fundamental areas that will be discussed. • Survey well done; it can’t be improved. • By dividing them into specific areas, such as: global civil aviation, space transportation, etc. • Continue the in-person interviews. • Emphasize more narrative questions and answers. There are techniques to analyze narrative answers.

More Related